From d6ec50102952a03c7b3a41bb3bfc7fd704b997d3 Mon Sep 17 00:00:00 2001 From: Jiekang Tian Date: Sun, 20 Sep 2026 15:14:22 +0800 Subject: [PATCH] refactor(nmr): migrate infrastructure to nmr 0.1.0 Replace in-house NMR readers and spectral kernels with the pinned library. Retain native datasets, calibration evidence, component semantics, and snapshot v1 across desktop and CLI workflows. Unify import and reset defaults, propagate processing failures, and cover sampling declarations, phase quality, NUS reconstruction, and offline snapshots with regression tests. Remove migration probes and synchronize English and Chinese documentation. Update rustls to 0.23.45 and rustls-webpki to 0.103.15 to resolve RUSTSEC-2026-0285 without adding an advisory exception. Validation: full cargo pr-check passes, including formatting, source sizes, dependency policy, default frontend builds, Clippy, and reference-backend tests (local sccache bypassed due to a cache-service permissions error). Documentation build passed in the prior validation; documentation is unchanged by the dependency fix. --- Cargo.lock | 28 +- Cargo.toml | 7 +- crates/app/Cargo.toml | 1 + crates/app/src/shot.rs | 9 +- crates/app/src/shot/craft_shot.rs | 5 +- crates/app/src/ui/canvas/craft_regions.rs | 11 +- crates/app/src/ui/canvas/craft_results.rs | 9 +- crates/app/src/ui/canvas/cursors.rs | 71 +- crates/app/src/ui/canvas/mod_tests.rs | 2 +- .../app/src/ui/canvas/reference_pick_tests.rs | 2 +- crates/app/src/ui/canvas/slices.rs | 30 +- crates/app/src/ui/clipboard_figure.rs | 2 +- crates/app/src/ui/command_exec.rs | 1 + crates/app/src/ui/commands.rs | 5 + crates/app/src/ui/commands/craft.rs | 22 +- crates/app/src/ui/commands/identity.rs | 2 + crates/app/src/ui/commands/roster.rs | 1 + crates/app/src/ui/commands_craft_tests.rs | 8 +- crates/app/src/ui/commands_tests.rs | 4 +- crates/app/src/ui/data_sheet.rs | 39 +- crates/app/src/ui/file_dialogs.rs | 40 +- crates/app/src/ui/file_dialogs/discovery.rs | 64 +- .../app/src/ui/file_dialogs/nmr_sampling.rs | 72 ++ crates/app/src/ui/menus.rs | 1 + crates/app/src/ui/mod.rs | 10 + .../src/ui/object_inspector/chart_gallery.rs | 43 +- .../src/ui/primary_sidebar/data_browser.rs | 25 +- crates/app/src/ui/properties/fixture.rs | 20 +- crates/app/src/ui/tools/craft.rs | 36 +- crates/app/src/ui/tools/craft/results.rs | 3 +- crates/app/src/ui/tools/craft/setup.rs | 37 +- crates/app/src/ui/tools/processing/mod.rs | 18 +- crates/app/src/ui/tools/processing/surface.rs | 8 +- crates/app/src/ui/tools/pseudo.rs | 149 ++-- crates/app/src/ui/tools/task_card_tests.rs | 11 +- crates/cli/Cargo.toml | 4 + crates/cli/src/craft.rs | 154 ++-- crates/cli/src/main.rs | 219 ++--- crates/cli/src/main_tests.rs | 145 ++++ crates/cli/tests/batch_cli.rs | 4 +- crates/cli/tests/nmr_craft.rs | 91 ++ crates/cli/tests/nmr_inspect.rs | 88 ++ crates/cli/tests/nmr_sampling.rs | 115 +++ crates/core/Cargo.toml | 1 + .../core/src/actions/app_impl/processing.rs | 20 +- crates/core/src/actions/mod.rs | 1 + crates/core/src/actions/processing_state.rs | 76 +- crates/core/src/actions/tests/align.rs | 24 +- crates/core/src/actions/tests/arithmetic.rs | 29 +- crates/core/src/actions/tests/linefit.rs | 8 +- crates/core/src/actions/tests/mod.rs | 17 +- crates/core/src/actions/tests/more.rs | 40 +- crates/core/src/actions/tests/multiplet.rs | 6 +- crates/core/src/actions/tests/scheme_apply.rs | 18 +- .../core/src/actions/tests/stable_identity.rs | 14 +- crates/core/src/actions/tests/stack.rs | 64 +- crates/core/src/actions/tests/symmetry.rs | 8 +- .../properties_tests_inbound_value.rs | 2 +- .../automation/properties_tests_outbound.rs | 27 +- .../automation/properties_tests_rejections.rs | 9 +- crates/core/src/automation/registry.rs | 4 +- crates/core/src/automation/tool_executors.rs | 5 +- crates/core/src/data_export.rs | 25 +- crates/core/src/data_export/nmr.rs | 37 + crates/core/src/data_export/tests.rs | 12 +- crates/core/src/data_export/write.rs | 32 +- crates/core/src/figures.rs | 57 +- crates/core/src/lib.rs | 8 +- crates/core/src/nmr_test_support.rs | 46 ++ .../src/project/acquisition_identity_tests.rs | 48 +- crates/core/src/project/cleanup_tests.rs | 8 +- crates/core/src/project/codec.rs | 233 +----- crates/core/src/project/codec_tests.rs | 32 +- crates/core/src/project/convert.rs | 215 ++--- crates/core/src/project/convert_dimensions.rs | 51 -- crates/core/src/project/convert_recipes.rs | 25 +- crates/core/src/project/craft_tests.rs | 24 +- crates/core/src/project/dto.rs | 22 - crates/core/src/project/field_catalog.rs | 2 +- .../core/src/project/field_encoding_tests.rs | 14 +- crates/core/src/project/lineage_tests.rs | 30 +- crates/core/src/project/linefit_tests.rs | 2 +- crates/core/src/project/mod.rs | 8 +- crates/core/src/project/multiplet_tests.rs | 4 +- crates/core/src/project/nmr_snapshot.rs | 131 +++ crates/core/src/project/nmr_snapshot_tests.rs | 415 ++++++++++ crates/core/src/project/pipeline_conv.rs | 60 +- .../core/src/project/pipeline_domain_tests.rs | 8 +- crates/core/src/project/pseudo_tests.rs | 24 +- crates/core/src/project/scheme.rs | 22 +- .../core/src/project/step_identity_tests.rs | 4 +- crates/core/src/project/symmetry_tests.rs | 2 +- crates/core/src/project/templates.rs | 23 +- crates/core/src/project/tests.rs | 34 +- .../core/src/properties/apodization_tests.rs | 2 +- crates/core/src/properties/bin.rs | 23 +- crates/core/src/properties/group_delay.rs | 6 +- .../core/src/properties/group_delay_tests.rs | 97 ++- crates/core/src/properties/ilt_tests.rs | 4 +- crates/core/src/properties/object_tests.rs | 6 +- .../core/src/properties/processing_common.rs | 32 +- .../src/properties/processing_test_support.rs | 27 +- crates/core/src/properties/provider_tests.rs | 16 +- crates/core/src/properties/step_enabled.rs | 6 +- crates/core/src/properties/tests.rs | 3 +- crates/core/src/properties/tests_fixture.rs | 8 +- crates/core/src/properties/zero_fill_tests.rs | 22 +- crates/core/src/state/app_impl.rs | 40 +- crates/core/src/state/app_impl_analysis.rs | 33 +- .../core/src/state/app_impl_analysis_tests.rs | 2 +- crates/core/src/state/app_impl_arithmetic.rs | 55 +- crates/core/src/state/app_impl_compute.rs | 80 +- .../core/src/state/app_impl_compute_tests.rs | 85 +- crates/core/src/state/app_impl_figures.rs | 60 +- crates/core/src/state/app_impl_io.rs | 280 +++---- crates/core/src/state/app_impl_io_tests.rs | 159 ++++ crates/core/src/state/app_impl_multiplet.rs | 11 +- crates/core/src/state/app_impl_slice.rs | 274 ++++--- crates/core/src/state/compute.rs | 78 +- crates/core/src/state/compute/tests.rs | 47 +- crates/core/src/state/compute_worker.rs | 67 +- crates/core/src/state/contour_budget_tests.rs | 43 +- crates/core/src/state/craft.rs | 53 +- crates/core/src/state/craft_fields.rs | 70 +- crates/core/src/state/data_import.rs | 200 +++++ crates/core/src/state/data_import_tests.rs | 131 +++ crates/core/src/state/dataset_trace.rs | 11 +- crates/core/src/state/datasets.rs | 410 ++++++--- .../core/src/state/datasets/nmr_defaults.rs | 66 ++ .../datasets/pseudo_display_binding_tests.rs | 10 +- .../core/src/state/datasets/pseudo_tests.rs | 116 +-- crates/core/src/state/datasets_2d_figure.rs | 20 +- crates/core/src/state/datasets_2d_maps.rs | 18 + crates/core/src/state/datasets_dispatch.rs | 49 +- crates/core/src/state/field.rs | 145 +--- crates/core/src/state/field_catalog.rs | 40 +- crates/core/src/state/field_metadata.rs | 77 ++ crates/core/src/state/field_nmr.rs | 61 ++ crates/core/src/state/field_payload.rs | 6 +- crates/core/src/state/field_runtime_tests.rs | 36 +- crates/core/src/state/field_tests.rs | 118 +-- crates/core/src/state/mod.rs | 10 +- crates/core/src/state/nmr_integrals.rs | 10 +- crates/core/src/state/nmr_integrals_2d.rs | 13 +- crates/core/src/state/nus.rs | 46 -- crates/core/src/state/peaks_tests.rs | 4 +- .../core/src/state/scientific_summary/mod.rs | 7 +- .../src/state/scientific_summary/resolver.rs | 50 +- .../core/src/state/trace_alignment_tests.rs | 41 +- crates/core/src/state/trace_provider.rs | 9 + crates/core/src/state/trace_provider_tests.rs | 20 +- crates/core/src/state/ui_state.rs | 6 + crates/core/src/state/ui_state_nmr_import.rs | 56 ++ crates/core/src/workflow.rs | 198 +++-- crates/core/src/workflow/dataset.rs | 69 +- crates/core/src/workflow/nmr.rs | 79 ++ crates/core/src/workflow_tests.rs | 42 +- crates/core/tests/nmr_auto_nus.rs | 247 ++++++ crates/core/tests/slice.rs | 15 +- crates/core/tests/slice2d.rs | 63 +- crates/io/Cargo.toml | 1 + crates/io/src/archive.rs | 15 +- crates/io/src/bruker.rs | 776 ------------------ crates/io/src/bruker/parser_tests.rs | 56 -- crates/io/src/bruker/processed.rs | 334 -------- crates/io/src/jcamp_dx.rs | 746 ----------------- crates/io/src/jcamp_dx/tests.rs | 123 --- crates/io/src/jeol.rs | 746 ----------------- crates/io/src/jeol/filter.rs | 51 -- crates/io/src/jeol/nus.rs | 120 --- crates/io/src/jeol/params.rs | 106 --- crates/io/src/jeol/ruler.rs | 127 --- crates/io/src/jeol/tests.rs | 718 ---------------- crates/io/src/lib.rs | 125 +-- crates/io/src/nmr_bridge.rs | 179 ++++ crates/io/src/nmr_bridge_snapshot.rs | 36 + crates/io/src/nmr_input.rs | 108 +++ crates/io/src/nmr_origin.rs | 66 -- crates/io/src/nmr_sampling.rs | 82 ++ crates/io/src/nmr_series.rs | 170 ++++ crates/io/src/nmr_series_input.rs | 278 +++++++ crates/io/src/nmr_view.rs | 294 +++++++ crates/io/src/varian.rs | 302 ------- crates/io/src/varian/fid.rs | 200 ----- crates/io/src/varian/procpar.rs | 187 ----- crates/io/src/varian/tests.rs | 296 ------- crates/io/tests/bruker_processed.rs | 131 --- crates/io/tests/fixtures/nmr/README.md | 30 + crates/io/tests/fixtures/nmr/bruker-1d/acqus | 13 + crates/io/tests/fixtures/nmr/bruker-1d/fid | Bin 0 -> 16 bytes .../tests/fixtures/nmr/bruker-1d/pdata/1/1r | Bin 0 -> 16 bytes .../fixtures/nmr/bruker-1d/pdata/1/procs | 10 + .../io/tests/fixtures/nmr/bruker-nus/acqu2s | 10 + crates/io/tests/fixtures/nmr/bruker-nus/acqus | 16 + .../io/tests/fixtures/nmr/bruker-nus/nuslist | 2 + crates/io/tests/fixtures/nmr/bruker-nus/ser | Bin 0 -> 64 bytes .../fixtures/nmr/bruker-states-tppi/acqu2s | 9 + .../fixtures/nmr/bruker-states-tppi/acqus | 16 + .../tests/fixtures/nmr/bruker-states-tppi/ser | Bin 0 -> 64 bytes .../tests/fixtures/nmr/bruker-states/acqu2s | 9 + .../io/tests/fixtures/nmr/bruker-states/acqus | 16 + .../io/tests/fixtures/nmr/bruker-states/ser | Bin 0 -> 64 bytes .../tests/fixtures/nmr/generate_jeol_nus.py | 33 + crates/io/tests/fixtures/nmr/jcamp-hz.dx | 15 + crates/io/tests/fixtures/nmr/jcamp-ppm.dx | 15 + crates/io/tests/fixtures/nmr/jeol-complex.jdf | Bin 0 -> 1440 bytes .../tests/fixtures/nmr/jeol-nus-missing.jdf | Bin 0 -> 2016 bytes .../fixtures/nmr/varian-short-header.fid/fid | Bin 0 -> 68 bytes .../nmr/varian-short-header.fid/procpar | 18 + .../fixtures/nmr/varian-v0-status.fid/fid | Bin 0 -> 68 bytes .../fixtures/nmr/varian-v0-status.fid/procpar | 18 + crates/io/tests/fixtures/nmr/varian.fid/fid | Bin 0 -> 68 bytes .../io/tests/fixtures/nmr/varian.fid/procpar | 18 + crates/io/tests/nmr_bridge.rs | 397 +++++++++ crates/io/tests/nmr_group_delay.rs | 105 +++ crates/io/tests/nmr_sampling.rs | 149 ++++ crates/io/tests/nmr_view.rs | 193 +++++ crates/processing/Cargo.toml | 2 +- crates/processing/src/arithmetic.rs | 185 ++--- crates/processing/src/autophase.rs | 478 ----------- crates/processing/src/baseline.rs | 355 -------- crates/processing/src/cleanup.rs | 360 -------- crates/processing/src/craft.rs | 25 +- crates/processing/src/craft/nmr_preview.rs | 111 +++ crates/processing/src/craft/preflight.rs | 72 +- crates/processing/src/craft/regions.rs | 16 +- crates/processing/src/craft/resolution.rs | 20 +- crates/processing/src/craft/stability.rs | 4 +- crates/processing/src/craft_tests.rs | 10 +- crates/processing/src/fft.rs | 425 ---------- crates/processing/src/fft2.rs | 553 ------------- crates/processing/src/lib.rs | 143 +--- crates/processing/src/nmr_bridge.rs | 568 +++++++++++++ crates/processing/src/nmr_bridge_phase.rs | 195 +++++ .../processing/src/nmr_bridge_staged_tests.rs | 302 +++++++ crates/processing/src/nmr_bridge_tests.rs | 448 ++++++++++ crates/processing/src/nmr_execution.rs | 113 +++ crates/processing/src/nmr_execution_2d.rs | 280 +++++++ crates/processing/src/nmr_test_execution.rs | 94 +++ crates/processing/src/nus.rs | 188 ----- crates/processing/src/phase.rs | 114 --- crates/processing/src/preview.rs | 68 -- crates/processing/src/slice.rs | 241 ++---- crates/processing/src/tests.rs | 29 +- crates/processing/src/twod.rs | 199 +---- crates/processing/src/xps.rs | 33 +- crates/processing/src/xps_signal.rs | 170 ++++ .../tests/auto_correction_quality.rs | 72 +- crates/processing/tests/nmr_axis_evidence.rs | 99 +++ crates/processing/tests/nmr_group_delay.rs | 104 +++ crates/processing/tests/nmr_nus.rs | 126 +++ crates/processing/tests/nmr_operations.rs | 138 ++++ crates/processing/tests/nmr_phase_quality.rs | 211 +++++ .../processing/tests/nmr_prepare_control.rs | 107 +++ crates/processing/tests/nmr_shared_complex.rs | 322 ++++++++ .../src/content/docs/guides/importing-data.md | 53 +- docs/src/content/docs/guides/processing.md | 84 +- docs/src/content/docs/reference/cli.md | 72 +- .../content/docs/reference/file-formats.md | 49 +- .../docs/zh-cn/guides/importing-data.md | 40 +- .../content/docs/zh-cn/guides/processing.md | 61 +- docs/src/content/docs/zh-cn/reference/cli.md | 64 +- .../docs/zh-cn/reference/file-formats.md | 35 +- 263 files changed, 11683 insertions(+), 11055 deletions(-) create mode 100644 crates/app/src/ui/file_dialogs/nmr_sampling.rs create mode 100644 crates/cli/src/main_tests.rs create mode 100644 crates/cli/tests/nmr_craft.rs create mode 100644 crates/cli/tests/nmr_inspect.rs create mode 100644 crates/cli/tests/nmr_sampling.rs create mode 100644 crates/core/src/data_export/nmr.rs create mode 100644 crates/core/src/nmr_test_support.rs delete mode 100644 crates/core/src/project/convert_dimensions.rs create mode 100644 crates/core/src/project/nmr_snapshot.rs create mode 100644 crates/core/src/project/nmr_snapshot_tests.rs create mode 100644 crates/core/src/state/app_impl_io_tests.rs create mode 100644 crates/core/src/state/data_import.rs create mode 100644 crates/core/src/state/data_import_tests.rs create mode 100644 crates/core/src/state/datasets/nmr_defaults.rs create mode 100644 crates/core/src/state/field_metadata.rs create mode 100644 crates/core/src/state/field_nmr.rs delete mode 100644 crates/core/src/state/nus.rs create mode 100644 crates/core/src/state/ui_state_nmr_import.rs create mode 100644 crates/core/src/workflow/nmr.rs create mode 100644 crates/core/tests/nmr_auto_nus.rs delete mode 100644 crates/io/src/bruker.rs delete mode 100644 crates/io/src/bruker/parser_tests.rs delete mode 100644 crates/io/src/bruker/processed.rs delete mode 100644 crates/io/src/jcamp_dx.rs delete mode 100644 crates/io/src/jcamp_dx/tests.rs delete mode 100644 crates/io/src/jeol.rs delete mode 100644 crates/io/src/jeol/filter.rs delete mode 100644 crates/io/src/jeol/nus.rs delete mode 100644 crates/io/src/jeol/params.rs delete mode 100644 crates/io/src/jeol/ruler.rs delete mode 100644 crates/io/src/jeol/tests.rs create mode 100644 crates/io/src/nmr_bridge.rs create mode 100644 crates/io/src/nmr_bridge_snapshot.rs create mode 100644 crates/io/src/nmr_input.rs delete mode 100644 crates/io/src/nmr_origin.rs create mode 100644 crates/io/src/nmr_sampling.rs create mode 100644 crates/io/src/nmr_series.rs create mode 100644 crates/io/src/nmr_series_input.rs create mode 100644 crates/io/src/nmr_view.rs delete mode 100644 crates/io/src/varian.rs delete mode 100644 crates/io/src/varian/fid.rs delete mode 100644 crates/io/src/varian/procpar.rs delete mode 100644 crates/io/src/varian/tests.rs delete mode 100644 crates/io/tests/bruker_processed.rs create mode 100644 crates/io/tests/fixtures/nmr/README.md create mode 100644 crates/io/tests/fixtures/nmr/bruker-1d/acqus create mode 100644 crates/io/tests/fixtures/nmr/bruker-1d/fid create mode 100644 crates/io/tests/fixtures/nmr/bruker-1d/pdata/1/1r create mode 100644 crates/io/tests/fixtures/nmr/bruker-1d/pdata/1/procs create mode 100644 crates/io/tests/fixtures/nmr/bruker-nus/acqu2s create mode 100644 crates/io/tests/fixtures/nmr/bruker-nus/acqus create mode 100644 crates/io/tests/fixtures/nmr/bruker-nus/nuslist create mode 100644 crates/io/tests/fixtures/nmr/bruker-nus/ser create mode 100644 crates/io/tests/fixtures/nmr/bruker-states-tppi/acqu2s create mode 100644 crates/io/tests/fixtures/nmr/bruker-states-tppi/acqus create mode 100644 crates/io/tests/fixtures/nmr/bruker-states-tppi/ser create mode 100644 crates/io/tests/fixtures/nmr/bruker-states/acqu2s create mode 100644 crates/io/tests/fixtures/nmr/bruker-states/acqus create mode 100644 crates/io/tests/fixtures/nmr/bruker-states/ser create mode 100644 crates/io/tests/fixtures/nmr/generate_jeol_nus.py create mode 100644 crates/io/tests/fixtures/nmr/jcamp-hz.dx create mode 100644 crates/io/tests/fixtures/nmr/jcamp-ppm.dx create mode 100644 crates/io/tests/fixtures/nmr/jeol-complex.jdf create mode 100644 crates/io/tests/fixtures/nmr/jeol-nus-missing.jdf create mode 100644 crates/io/tests/fixtures/nmr/varian-short-header.fid/fid create mode 100644 crates/io/tests/fixtures/nmr/varian-short-header.fid/procpar create mode 100644 crates/io/tests/fixtures/nmr/varian-v0-status.fid/fid create mode 100644 crates/io/tests/fixtures/nmr/varian-v0-status.fid/procpar create mode 100644 crates/io/tests/fixtures/nmr/varian.fid/fid create mode 100644 crates/io/tests/fixtures/nmr/varian.fid/procpar create mode 100644 crates/io/tests/nmr_bridge.rs create mode 100644 crates/io/tests/nmr_group_delay.rs create mode 100644 crates/io/tests/nmr_sampling.rs create mode 100644 crates/io/tests/nmr_view.rs delete mode 100644 crates/processing/src/autophase.rs delete mode 100644 crates/processing/src/baseline.rs delete mode 100644 crates/processing/src/cleanup.rs create mode 100644 crates/processing/src/craft/nmr_preview.rs delete mode 100644 crates/processing/src/fft.rs delete mode 100644 crates/processing/src/fft2.rs create mode 100644 crates/processing/src/nmr_bridge.rs create mode 100644 crates/processing/src/nmr_bridge_phase.rs create mode 100644 crates/processing/src/nmr_bridge_staged_tests.rs create mode 100644 crates/processing/src/nmr_bridge_tests.rs create mode 100644 crates/processing/src/nmr_execution.rs create mode 100644 crates/processing/src/nmr_execution_2d.rs create mode 100644 crates/processing/src/nmr_test_execution.rs delete mode 100644 crates/processing/src/nus.rs delete mode 100644 crates/processing/src/phase.rs delete mode 100644 crates/processing/src/preview.rs create mode 100644 crates/processing/src/xps_signal.rs create mode 100644 crates/processing/tests/nmr_axis_evidence.rs create mode 100644 crates/processing/tests/nmr_group_delay.rs create mode 100644 crates/processing/tests/nmr_nus.rs create mode 100644 crates/processing/tests/nmr_operations.rs create mode 100644 crates/processing/tests/nmr_phase_quality.rs create mode 100644 crates/processing/tests/nmr_prepare_control.rs create mode 100644 crates/processing/tests/nmr_shared_complex.rs diff --git a/Cargo.lock b/Cargo.lock index f81c5d07..6550dedf 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -3421,6 +3421,19 @@ dependencies = [ "jni-sys 0.3.1", ] +[[package]] +name = "nmr" +version = "0.1.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5749f9cb01773d2df50a3824c4591af5367da58daeaf9dd199d051a1358fda9c" +dependencies = [ + "num-complex", + "rustfft", + "sha2", + "tempfile", + "thiserror 2.0.18", +] + [[package]] name = "no-std-compat" version = "0.4.1" @@ -4136,6 +4149,7 @@ dependencies = [ "libc", "log", "muda", + "nmr", "num-complex", "plotx-analysis", "plotx-core", @@ -4170,10 +4184,12 @@ dependencies = [ name = "plotx-cli" version = "0.1.0" dependencies = [ + "nmr", "plotx-core", "plotx-io", "plotx-processing", "serde_json", + "tempfile", ] [[package]] @@ -4182,6 +4198,7 @@ version = "0.1.0" dependencies = [ "directories", "image", + "nmr", "num-complex", "pdf-writer", "plotx-analysis", @@ -4249,6 +4266,7 @@ dependencies = [ "flate2", "image", "memmap2", + "nmr", "num-complex", "quick-xml", "rust_xlsxwriter", @@ -4265,10 +4283,10 @@ dependencies = [ name = "plotx-processing" version = "0.1.0" dependencies = [ + "nmr", "num-complex", "plotx-analysis", "plotx-io", - "rustfft", "serde", "thiserror 2.0.18", ] @@ -4894,9 +4912,9 @@ dependencies = [ [[package]] name = "rustls" -version = "0.23.42" +version = "0.23.45" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "3c54fcab019b409d04215d3a17cb438fd7fbf192ee61461f20f4fe18704bc138" +checksum = "0d41d731c7d2f962d1ccc364cec258de3c0e93b38c2fb3ba97ac74513048d634" dependencies = [ "log", "once_cell", @@ -4918,9 +4936,9 @@ dependencies = [ [[package]] name = "rustls-webpki" -version = "0.103.13" +version = "0.103.15" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "61c429a8649f110dddef65e2a5ad240f747e85f7758a6bccc7e5777bd33f756e" +checksum = "f3c3cf1d8b1e7d4927e2d154c3fcb02979afb9939629c62cd9048d4f07b60ac2" dependencies = [ "ring", "rustls-pki-types", diff --git a/Cargo.toml b/Cargo.toml index 546408cd..f5b22ac0 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -38,10 +38,10 @@ publish = false repository = "https://github.com/nmrtist/plotx" [workspace.dependencies] +nmr = "=0.1.0" colorous = "1" num-complex = "0.4" nalgebra = { version = "0.35", default-features = false, features = ["std"] } -rustfft = "6.4" serde = { version = "1.0", features = ["derive", "rc"] } serde_json = { version = "1.0", features = ["float_roundtrip"] } semver = "1" @@ -123,7 +123,10 @@ opt-level = 3 [profile.dev.package.plotx-processing] opt-level = 3 -# rustfft is the only third-party dependency that needs optimized debug code. +[profile.dev.package.nmr] +opt-level = 3 + +# Optimize the shared FFT backend as well as its callers. [profile.dev.package.rustfft] opt-level = 2 diff --git a/crates/app/Cargo.toml b/crates/app/Cargo.toml index 8e2ab206..63dd75c9 100644 --- a/crates/app/Cargo.toml +++ b/crates/app/Cargo.toml @@ -17,6 +17,7 @@ default = [] datafusion = ["plotx-core/datafusion"] [dependencies] +nmr.workspace = true plotx-core.workspace = true plotx-analysis.workspace = true plotx-io.workspace = true diff --git a/crates/app/src/shot.rs b/crates/app/src/shot.rs index df996dc3..e1839851 100644 --- a/crates/app/src/shot.rs +++ b/crates/app/src/shot.rs @@ -516,7 +516,9 @@ fn setup(app: &mut PlotxApp) { let data = synthetic_fid(); let action = Action::insert_dataset_with_default_canvas( app, - Dataset::Nmr(Box::new(NmrDataset::load(data))), + Dataset::Nmr(Box::new( + NmrDataset::load(data).expect("the synthetic FID is valid"), + )), "Canvas 1 — synthetic".to_owned(), DEFAULT_CANVAS_SIZE_MM, ); @@ -553,7 +555,7 @@ fn line_fit(app: &mut PlotxApp, ctx: &egui::Context) -> Result<(), String> { fn symmetry_setup(app: &mut PlotxApp) -> Result<(), String> { *app = PlotxApp::new_with_settings(Settings::default()); - let mut dataset = Nmr2DDataset::load(synthetic_cosy()); + let mut dataset = Nmr2DDataset::load(synthetic_cosy()).expect("valid synthetic 2D acquisition"); let ids = dataset .peaks .add_pair( @@ -587,7 +589,8 @@ fn symmetry_setup(app: &mut PlotxApp) -> Result<(), String> { fn region_result(app: &mut PlotxApp) { *app = PlotxApp::new_with_settings(Settings::default()); - let mut dataset = Nmr2DDataset::load(synthetic_series()); + let mut dataset = + Nmr2DDataset::load(synthetic_series()).expect("valid synthetic 2D acquisition"); dataset.region_analysis.regions.push(Region { id: RegionId::new(0), lo: 4.65, diff --git a/crates/app/src/shot/craft_shot.rs b/crates/app/src/shot/craft_shot.rs index d1529130..2e9358c8 100644 --- a/crates/app/src/shot/craft_shot.rs +++ b/crates/app/src/shot/craft_shot.rs @@ -13,8 +13,9 @@ pub(super) fn setup(app: &mut PlotxApp, ctx: &egui::Context) -> Result<(), Strin .clone(); let mut params = CraftParams::conventional(); params.maximum_model_order = 8; - let invocation = CraftInvocation::acquisition(&data, params); - let result = process_craft_cancellable(&data, &invocation, &|| false) + let fid = data.craft_fid().map_err(|error| error.to_string())?; + let invocation = CraftInvocation::acquisition(&fid, params); + let result = process_craft_cancellable(&fid, &invocation, &|| false) .map_err(|error| format!("CRAFT screenshot analysis failed: {error}"))?; let nmr = app.doc.datasets[0] .as_nmr_mut() diff --git a/crates/app/src/ui/canvas/craft_regions.rs b/crates/app/src/ui/canvas/craft_regions.rs index 7cb25707..2c157a75 100644 --- a/crates/app/src/ui/canvas/craft_regions.rs +++ b/crates/app/src/ui/canvas/craft_regions.rs @@ -19,9 +19,14 @@ pub(crate) fn handle_craft_region_drag( } let acquired_bounds = nmr.spectrum().unwrap().ppm_bounds(); let dataset_id = nmr.resource_id; - let observe_freq = nmr.data.observe_freq_mhz.max(f64::MIN_POSITIVE); - let point_step = - nmr.data.spectral_width_hz.abs() / observe_freq / nmr.data.points.len().max(1) as f64; + let Some(reference) = nmr.craft_reference() else { + return; + }; + let Some(width) = nmr.data.axes()[0].spectral_width_hz else { + return; + }; + let observe_freq = reference.reference_frequency_mhz; + let point_step = width / observe_freq / nmr.data.len() as f64; let suggestions = app .session .ui diff --git a/crates/app/src/ui/canvas/craft_results.rs b/crates/app/src/ui/canvas/craft_results.rs index 8cf2b3c5..36ecefa8 100644 --- a/crates/app/src/ui/canvas/craft_results.rs +++ b/crates/app/src/ui/canvas/craft_results.rs @@ -50,7 +50,6 @@ pub(crate) fn handle_and_paint_craft_result( dataset, run, stored, - nmr, plot, figure, painter, @@ -119,7 +118,6 @@ struct CraftRangePaintContext<'a> { dataset: plotx_core::state::DatasetId, run: plotx_core::state::CraftRunId, stored: &'a plotx_core::state::StoredCraftRun, - nmr: &'a plotx_core::state::NmrDataset, plot: PlotRect, figure: &'a plotx_figure::Figure, painter: &'a egui::Painter, @@ -132,7 +130,6 @@ fn paint_craft_ranges(context: CraftRangePaintContext<'_>) { dataset, run, stored, - nmr, plot, figure, painter, @@ -143,7 +140,11 @@ fn paint_craft_ranges(context: CraftRangePaintContext<'_>) { .invocation .reference .effective_carrier_ppm(); - let observe = nmr.data.observe_freq_mhz; + let observe = stored + .provenance + .invocation + .reference + .reference_frequency_mhz; let modeling = stored .diagnostics .modeling_windows diff --git a/crates/app/src/ui/canvas/cursors.rs b/crates/app/src/ui/canvas/cursors.rs index 2ff4007d..0b979fa7 100644 --- a/crates/app/src/ui/canvas/cursors.rs +++ b/crates/app/src/ui/canvas/cursors.rs @@ -362,13 +362,29 @@ fn inspect_text(app: &PlotxApp, dataset: usize, point: CursorPoint) -> String { return String::new(); }; match data { - Dataset::Nmr(_) => format!("x {:.4} ppm · I {}", point.x, fmt_number(point.intensity)), - Dataset::Nmr2D(_) => format!( - "F2 {:.4} ppm · F1 {:.4} ppm · I {}", - point.x, - point.y.unwrap_or_default(), - fmt_number(point.intensity), - ), + Dataset::Nmr(n) => { + let unit = n + .spectrum() + .map_or("s", |s| plotx_processing::axis_unit_label(Some(s.unit))); + format!( + "x {:.4} {unit} · I {}", + point.x, + fmt_number(point.intensity) + ) + } + Dataset::Nmr2D(n) => { + let Processed2D::Ft(s) = &n.processed else { + return String::new(); + }; + format!( + "F2 {:.4} {} · F1 {:.4} {} · I {}", + point.x, + s.direct.unit_label(), + point.y.unwrap_or_default(), + s.indirect.unit_label(), + fmt_number(point.intensity) + ) + } _ => String::new(), } } @@ -380,24 +396,41 @@ fn delta_text(app: &PlotxApp, dataset: usize, delta: CursorDelta) -> String { let dx = delta.second.x - delta.first.x; let di = delta.second.intensity - delta.first.intensity; match data { - Dataset::Nmr(nmr) => format!( - "Δx {} ppm ({} Hz) · ΔI {}", - fmt_delta(dx), - fmt_delta(dx * nmr.data.observe_freq_mhz), - fmt_number(di), - ), + Dataset::Nmr(nmr) => { + if let Some(spectrum) = nmr.spectrum() { + if spectrum.unit == nmr::axis::AxisUnit::Hertz { + format!("Δx {} Hz · ΔI {}", fmt_delta(dx), fmt_number(di)) + } else { + let hz = nmr + .native_processed + .reference_frequency_mhz(0) + .map(|frequency| format!(" ({} Hz)", fmt_delta(dx * frequency))) + .unwrap_or_default(); + format!("Δx {} ppm{hz} · ΔI {}", fmt_delta(dx), fmt_number(di)) + } + } else { + format!("Δt {} s · ΔI {}", fmt_delta(dx), fmt_number(di)) + } + } Dataset::Nmr2D(nmr) => { let Processed2D::Ft(spectrum) = &nmr.processed else { return String::new(); }; let dy = delta.second.y.unwrap_or_default() - delta.first.y.unwrap_or_default(); + let axis_delta = |value: f64, meta: &plotx_processing::AxisMeta| { + let unit = match meta.unit { + Some(nmr::axis::AxisUnit::Second) => "s", + Some(nmr::axis::AxisUnit::Hertz) => "Hz", + Some(nmr::axis::AxisUnit::Ppm) => "ppm", + _ => "", + }; + format!("{} {unit}", fmt_delta(value)) + }; format!( - "ΔF2 {} ppm ({} Hz) · ΔF1 {} ppm ({} Hz) · ΔI {}", - fmt_delta(dx), - fmt_delta(dx * spectrum.direct.observe_freq_mhz), - fmt_delta(dy), - fmt_delta(dy * spectrum.indirect.observe_freq_mhz), - fmt_number(di), + "ΔF2 {} · ΔF1 {} · ΔI {}", + axis_delta(dx, &spectrum.direct), + axis_delta(dy, &spectrum.indirect), + fmt_number(di) ) } _ => String::new(), diff --git a/crates/app/src/ui/canvas/mod_tests.rs b/crates/app/src/ui/canvas/mod_tests.rs index 282a0482..1192ae75 100644 --- a/crates/app/src/ui/canvas/mod_tests.rs +++ b/crates/app/src/ui/canvas/mod_tests.rs @@ -137,7 +137,7 @@ fn phase_editor_open_drives_on_plot_pivot() { let mut app = PlotxApp::new(); app.doc .datasets - .push(Dataset::Nmr(Box::new(NmrDataset::load(data)))); + .push(Dataset::Nmr(Box::new(NmrDataset::load(data).unwrap()))); let mut canvas = CanvasDocument::new("page".to_owned(), [200.0, 200.0]); let id = canvas.allocate_object_id(); let obj = app.build_plot_object( diff --git a/crates/app/src/ui/canvas/reference_pick_tests.rs b/crates/app/src/ui/canvas/reference_pick_tests.rs index b199883f..d729f4f6 100644 --- a/crates/app/src/ui/canvas/reference_pick_tests.rs +++ b/crates/app/src/ui/canvas/reference_pick_tests.rs @@ -30,7 +30,7 @@ fn synthetic_app() -> PlotxApp { let mut app = PlotxApp::new(); app.doc .datasets - .push(Dataset::Nmr(Box::new(NmrDataset::load(data)))); + .push(Dataset::Nmr(Box::new(NmrDataset::load(data).unwrap()))); app.focus_single(0); app } diff --git a/crates/app/src/ui/canvas/slices.rs b/crates/app/src/ui/canvas/slices.rs index 48b4855b..5b421f75 100644 --- a/crates/app/src/ui/canvas/slices.rs +++ b/crates/app/src/ui/canvas/slices.rs @@ -110,14 +110,30 @@ pub(crate) fn paint_slice( }; // A Row cut runs along F2 (the plot's x-axis); a Column cut along F1 (y). - let (slice, along_x, mode) = match &n.processed { - Processed2D::Ft(s) => ( - s.slice(cursor.kind, cursor.index), - cursor.kind == SliceKind::Row, - DisplayMode::Real, - ), - Processed2D::Stack(s) => (s.slice(cursor.index), true, DisplayMode::Real), + let kind = if matches!(n.processed, Processed2D::Stack(_)) { + SliceKind::Row + } else { + cursor.kind + }; + let (_, slice) = match plotx_processing::slice::extract( + &n.native_processed, + kind, + plotx_processing::slice::Reduction::Slice(cursor.index), + ) { + Ok(output) => output, + Err(error) => { + painter.text( + Pos2::new(plot.left + 8.0, plot.top + 8.0), + egui::Align2::LEFT_TOP, + format!("Slice unavailable: {error}"), + egui::FontId::proportional(12.0), + SLICE_COLOR, + ); + return; + } }; + let along_x = kind == SliceKind::Row; + let mode = DisplayMode::Real; if let Some(position) = slice.position { if along_x { diff --git a/crates/app/src/ui/clipboard_figure.rs b/crates/app/src/ui/clipboard_figure.rs index 47afd819..1c7447c1 100644 --- a/crates/app/src/ui/clipboard_figure.rs +++ b/crates/app/src/ui/clipboard_figure.rs @@ -401,7 +401,7 @@ mod tests { let mut app = plotx_core::state::PlotxApp::new(); let action = Action::insert_dataset_with_default_canvas( &app, - Dataset::Nmr(Box::new(NmrDataset::load(data))), + Dataset::Nmr(Box::new(NmrDataset::load(data).unwrap())), "probe".to_owned(), DEFAULT_CANVAS_SIZE_MM, ); diff --git a/crates/app/src/ui/command_exec.rs b/crates/app/src/ui/command_exec.rs index 7931ba5c..1daf55ee 100644 --- a/crates/app/src/ui/command_exec.rs +++ b/crates/app/src/ui/command_exec.rs @@ -51,6 +51,7 @@ fn execute_inner( } CommandId::OpenFile => super::file_dialogs::open_file(app), CommandId::OpenFolder => super::file_dialogs::open_folder(app), + CommandId::ImportNmrSampling => super::file_dialogs::nmr_sampling::open(app), CommandId::RunBatchWorkflow => super::batch_workflow::AutomationUi::request_open(ctx), CommandId::RunScientificScript => { super::batch_workflow::AutomationUi::request_run_script(ctx) diff --git a/crates/app/src/ui/commands.rs b/crates/app/src/ui/commands.rs index 917cd73d..58231338 100644 --- a/crates/app/src/ui/commands.rs +++ b/crates/app/src/ui/commands.rs @@ -30,6 +30,7 @@ pub enum CommandId { CloseProject, OpenFile, OpenFolder, + ImportNmrSampling, RunBatchWorkflow, RunScientificScript, OpenRecent(usize), @@ -258,6 +259,10 @@ pub fn describe(app: &PlotxApp, id: CommandId) -> CommandDescriptor { app.session.ui.table_import_preview.is_none(), "Finish or cancel the current table import preview before importing another table.", ), + CommandId::ImportNmrSampling => requires( + app.session.ui.nmr_import.is_none(), + "Finish or cancel the current NMR import before importing another acquisition.", + ), CommandId::ImportImage | CommandId::ImportImageFirstFrame | CommandId::ImportImageWithoutMetadata diff --git a/crates/app/src/ui/commands/craft.rs b/crates/app/src/ui/commands/craft.rs index f4599f99..4f54ddb3 100644 --- a/crates/app/src/ui/commands/craft.rs +++ b/crates/app/src/ui/commands/craft.rs @@ -14,7 +14,7 @@ pub(super) fn gate(app: &PlotxApp, command: CommandId) -> Result<(), &'static st app.active_dataset().is_some_and(|index| { app.doc.datasets[index] .as_nmr() - .is_some_and(|nmr| nmr.data.domain == plotx_io::Domain::Time) + .is_some_and(|nmr| nmr.input_domain() == plotx_io::Domain::Time) }), "Select a one-dimensional time-domain NMR FID before opening CRAFT.", ), @@ -22,7 +22,7 @@ pub(super) fn gate(app: &PlotxApp, command: CommandId) -> Result<(), &'static st target.is_some_and(|index| { app.doc.datasets[index] .as_nmr() - .is_some_and(|nmr| nmr.data.domain == plotx_io::Domain::Time) + .is_some_and(|nmr| nmr.input_domain() == plotx_io::Domain::Time) }), "Open CRAFT for a one-dimensional time-domain NMR FID before running it.", ) @@ -32,15 +32,19 @@ pub(super) fn gate(app: &PlotxApp, command: CommandId) -> Result<(), &'static st if let Some(cache) = &app.session.ui.craft_resolution_cache && cache.dataset == nmr.resource_id && cache.dataset_epoch == app.session.dataset_epoch - && cache.reference == nmr.craft_reference() + && Some(cache.reference) == nmr.craft_reference() && cache.overrides == app.session.ui.craft_overrides && cache.parent_run == app.session.ui.craft_base_run { return cache.invocation.assessment.can_run(); } + let (Ok(data), Some(reference)) = (nmr.data.craft_fid(), nmr.craft_reference()) + else { + return false; + }; plotx_processing::craft::resolve_craft_invocation( - &nmr.data, - nmr.craft_reference(), + &data, + reference, &app.session.ui.craft_overrides, app.session .ui @@ -58,9 +62,13 @@ pub(super) fn gate(app: &PlotxApp, command: CommandId) -> Result<(), &'static st .and_then(|()| { let selected_count = target.map_or(0, |index| { let nmr = app.doc.datasets[index].as_nmr().unwrap(); + let (Ok(data), Some(reference)) = (nmr.data.craft_fid(), nmr.craft_reference()) + else { + return 0; + }; let invocation = plotx_processing::craft::resolve_craft_invocation( - &nmr.data, - nmr.craft_reference(), + &data, + reference, &app.session.ui.craft_overrides, app.session .ui diff --git a/crates/app/src/ui/commands/identity.rs b/crates/app/src/ui/commands/identity.rs index 732c95b5..a7727acb 100644 --- a/crates/app/src/ui/commands/identity.rs +++ b/crates/app/src/ui/commands/identity.rs @@ -51,6 +51,7 @@ pub(super) fn command_identity( CommandId::CloseProject => plain("Close Project", Some(icon::X)), CommandId::OpenFile => plain("Open File…", Some(icon::FILE)), CommandId::OpenFolder => plain("Open Folder…", Some(icon::FOLDER)), + CommandId::ImportNmrSampling => plain("Import NMR with Sampling Table…", Some(icon::FILE)), CommandId::RunBatchWorkflow => plain("Automation…", Some(icon::PLAY)), CommandId::RunScientificScript => plain("Run Scientific Script", Some(icon::PLAY)), CommandId::OpenRecent(i) => ( @@ -416,6 +417,7 @@ fn simple_stable_id(id: CommandId) -> &'static str { CommandId::CloseProject => "file.close_project", CommandId::OpenFile => "file.open_file", CommandId::OpenFolder => "file.open_folder", + CommandId::ImportNmrSampling => "file.import_nmr_sampling", CommandId::RunBatchWorkflow => "tools.automation", CommandId::RunScientificScript => "tools.run_scientific_script", CommandId::ImportTable => "file.import_table", diff --git a/crates/app/src/ui/commands/roster.rs b/crates/app/src/ui/commands/roster.rs index 24812cea..e4469162 100644 --- a/crates/app/src/ui/commands/roster.rs +++ b/crates/app/src/ui/commands/roster.rs @@ -17,6 +17,7 @@ pub(super) fn command_ids(recent_files: usize) -> Vec { CommandId::CloseProject, CommandId::OpenFile, CommandId::OpenFolder, + CommandId::ImportNmrSampling, CommandId::RunBatchWorkflow, CommandId::RunScientificScript, CommandId::ClearRecentFiles, diff --git a/crates/app/src/ui/commands_craft_tests.rs b/crates/app/src/ui/commands_craft_tests.rs index d54442ac..ede3acd0 100644 --- a/crates/app/src/ui/commands_craft_tests.rs +++ b/crates/app/src/ui/commands_craft_tests.rs @@ -28,7 +28,9 @@ fn craft_command_opens_a_task_for_the_original_time_domain_fid() { fn craft_warning_does_not_disable_run() { let mut app = app_with_nmr(); let nmr = app.doc.datasets[0].as_nmr_mut().unwrap(); - nmr.data.points.fill(num_complex::Complex64::new(0.0, 0.0)); + let mut input = nmr.data.craft_fid().unwrap(); + input.points.fill(num_complex::Complex64::new(0.0, 0.0)); + nmr.data = input.try_into().unwrap(); execute_without_clipboard(CommandId::Craft, &mut app, &egui::Context::default()); use_short_fixture_filter(&mut app); @@ -40,7 +42,9 @@ fn craft_warning_does_not_disable_run() { fn craft_hard_preflight_error_disables_run() { let mut app = app_with_nmr(); let nmr = app.doc.datasets[0].as_nmr_mut().unwrap(); - nmr.data.group_delay = nmr.data.points.len().saturating_sub(8) as f64; + let mut input = nmr.data.craft_fid().unwrap(); + input.group_delay = input.points.len().saturating_sub(8) as f64; + nmr.data = input.try_into().unwrap(); execute_without_clipboard(CommandId::Craft, &mut app, &egui::Context::default()); use_short_fixture_filter(&mut app); diff --git a/crates/app/src/ui/commands_tests.rs b/crates/app/src/ui/commands_tests.rs index 7a6a3bef..8fc69a93 100644 --- a/crates/app/src/ui/commands_tests.rs +++ b/crates/app/src/ui/commands_tests.rs @@ -70,7 +70,7 @@ pub(super) fn app_with_nmr() -> PlotxApp { }; let action = Action::insert_dataset_with_default_canvas( &app, - Dataset::Nmr(Box::new(NmrDataset::load(data))), + Dataset::Nmr(Box::new(NmrDataset::load(data).unwrap())), "Canvas — 1D NMR".to_owned(), DEFAULT_CANVAS_SIZE_MM, ); @@ -127,7 +127,7 @@ fn time_domain_nmr_hides_frequency_analysis_and_disables_spectral_commands() { | plotx_processing::StepKind::Invert ) }); - dataset.retransform(); + dataset.retransform().unwrap(); assert!( !app.doc.datasets[0] diff --git a/crates/app/src/ui/data_sheet.rs b/crates/app/src/ui/data_sheet.rs index 5c74352a..aed3ee4f 100644 --- a/crates/app/src/ui/data_sheet.rs +++ b/crates/app/src/ui/data_sheet.rs @@ -140,24 +140,16 @@ pub(super) fn data_sheet_window(app: &mut PlotxApp, ctx: &egui::Context) { pub(super) fn nmr2d_sheet(ui: &mut Ui, n: &plotx_core::state::Nmr2DDataset) { let d = &n.data; - ui.label(format!( - "{} × {} points · indirect quadrature {:?}", - d.cols, d.rows, d.quad - )); - ui.label(format!( - "Direct (F2): {} · {:.3} MHz · SW {:.0} Hz · carrier {:.2} ppm", - d.direct.nucleus, - d.direct.observe_freq_mhz, - d.direct.spectral_width_hz, - d.direct.carrier_ppm - )); - ui.label(format!( - "Indirect (F1): {} · {:.3} MHz · SW {:.0} Hz · carrier {:.2} ppm", - d.indirect.nucleus, - d.indirect.observe_freq_mhz, - d.indirect.spectral_width_hz, - d.indirect.carrier_ppm - )); + ui.label(format!("{} × {} points (F2 × F1)", d.cols, d.rows)); + for (name, axis) in [("Direct (F2)", &d.direct), ("Indirect (F1)", &d.indirect)] { + ui.label(format!("{name}: {} · {:?}", axis.nucleus, axis.domain)); + if let Some(frequency) = axis.observe_freq_mhz { + ui.label(format!("Observe frequency: {frequency:.3} MHz")); + } + if let Some(width) = axis.spectral_width_hz { + ui.label(format!("Spectral width: {width:.0} Hz")); + } + } if let Some(exp) = &d.experiment { ui.label(format!("Experiment hint: {exp}")); } @@ -167,8 +159,8 @@ pub(super) fn nmr2d_sheet(ui: &mut Ui, n: &plotx_core::state::Nmr2DDataset) { let (f2lo, f2hi) = s.f2_bounds(); let (f1lo, f1hi) = s.f1_bounds(); ui.label(format!( - "Contour spectrum {}×{} (F1×F2) — F2 {f2lo:.2}..{f2hi:.2} ppm, F1 {f1lo:.2}..{f1hi:.2} ppm", - s.f1_size, s.f2_size + "Contour spectrum {}×{} (F1×F2) — F2 {f2lo:.2}..{f2hi:.2} {}, F1 {f1lo:.2}..{f1hi:.2} {}", + s.f1_size, s.f2_size, s.direct.unit_label(), s.indirect.unit_label() )); } plotx_processing::Processed2D::Stack(s) => { @@ -182,7 +174,7 @@ pub(super) fn nmr2d_sheet(ui: &mut Ui, n: &plotx_core::state::Nmr2DDataset) { pub(super) fn nmr_sheet(ui: &mut Ui, n: &plotx_core::state::NmrDataset) { let len = n.processed.values().len(); - ui.label(format!("{} · {} pts", n.data.nucleus, len)); + ui.label(format!("{} · {} pts", n.data.nucleus(), len)); ui.separator(); let columns: Vec<(String, Vec)> = match &n.processed { @@ -195,7 +187,10 @@ pub(super) fn nmr_sheet(ui: &mut Ui, n: &plotx_core::state::NmrDataset) { ), ], plotx_processing::Processed1D::Frequency(spec) => vec![ - ("ppm".to_owned(), spec.ppm.clone()), + ( + plotx_processing::axis_unit_label(Some(spec.unit)).to_owned(), + spec.ppm.clone(), + ), ("Real".to_owned(), spec.real()), ( "Imag".to_owned(), diff --git a/crates/app/src/ui/file_dialogs.rs b/crates/app/src/ui/file_dialogs.rs index 91739898..e023679e 100644 --- a/crates/app/src/ui/file_dialogs.rs +++ b/crates/app/src/ui/file_dialogs.rs @@ -9,6 +9,7 @@ use plotx_core::state::ProcessingSchemeDialogState; mod delimited; mod discovery; pub(crate) mod image_import; +pub(crate) mod nmr_sampling; mod origin; mod path; mod preview; @@ -365,11 +366,16 @@ where } pub(crate) fn load_and_note(app: &mut PlotxApp, path: &std::path::Path) { - let before = app.doc.datasets.len(); - app.load_from(path); - if app.doc.datasets.len() > before { - app.note_recent_file(path); + if plotx_io::archive::is_zip(path) { + let before = app.doc.datasets.len(); + app.load_from(path); + if app.doc.datasets.len() > before { + app.note_recent_file(path); + } + return; } + let selected = path.to_owned(); + app.queue_data_import(selected.clone(), move || Ok(vec![selected])); } pub(crate) fn open_file(app: &mut PlotxApp) { @@ -440,30 +446,8 @@ pub(crate) fn open_folder(app: &mut PlotxApp) { /// flush every other entry out of the capped list. The folder is noted when /// any file of the batch loaded, not just the last one. fn open_folder_path(app: &mut PlotxApp, path: &std::path::Path) { - let before = app.doc.datasets.len(); - let mut data_files = Vec::new(); - discovery::collect_data_files(path, &mut data_files); - if data_files.is_empty() { - app.load_from(path); - } else { - data_files.sort(); - let companion_paths: std::collections::HashSet = data_files - .iter() - .filter(|file| { - file.extension() - .is_some_and(|ext| ext.eq_ignore_ascii_case("pfc")) - }) - .filter_map(|file| plotx_io::load_path(file).ok()) - .flat_map(|loaded| loaded.provenance.companion_paths) - .collect(); - data_files.retain(|file| !companion_paths.contains(file)); - for file in data_files { - app.load_from(&file); - } - } - if app.doc.datasets.len() > before { - app.note_recent_file(path); - } + let folder = path.to_owned(); + app.queue_data_import(folder.clone(), move || discovery::discover_folder(&folder)); } pub(crate) fn choose_export_path(settings: &ExportSettings) -> Option { diff --git a/crates/app/src/ui/file_dialogs/discovery.rs b/crates/app/src/ui/file_dialogs/discovery.rs index 3e998d5f..d3f9d69d 100644 --- a/crates/app/src/ui/file_dialogs/discovery.rs +++ b/crates/app/src/ui/file_dialogs/discovery.rs @@ -1,26 +1,18 @@ use std::path::{Path, PathBuf}; -pub(super) fn collect_data_files(folder: &Path, output: &mut Vec) { +pub(super) fn collect_data_files(folder: &Path, output: &mut Vec) -> std::io::Result<()> { // Vendor acquisition directories are atomic. Their payload files must // never be rediscovered as independent datasets. - if plotx_io::waters::is_masslynx_raw(folder) - || plotx_io::bruker::detect_processed(folder).is_some() - || plotx_io::bruker::is_bruker_dir(folder) - || plotx_io::varian::is_varian(folder) - { + if plotx_io::waters::is_masslynx_raw(folder) || plotx_io::nmr_bridge::is_candidate(folder) { output.push(folder.to_owned()); - return; + return Ok(()); } - let Ok(entries) = std::fs::read_dir(folder) else { - return; - }; - for entry in entries.flatten() { + for entry in std::fs::read_dir(folder)? { + let entry = entry?; let path = entry.path(); - let Ok(kind) = entry.file_type() else { - continue; - }; + let kind = entry.file_type()?; if kind.is_dir() && !kind.is_symlink() { - collect_data_files(&path, output); + collect_data_files(&path, output)?; } else if kind.is_file() { let extension = path .extension() @@ -33,11 +25,43 @@ pub(super) fn collect_data_files(folder: &Path, output: &mut Vec) { extension.eq_ignore_ascii_case("raw") && plotx_io::xrd::is_rigaku_raw(&path); let recognized_casaxps = extension.eq_ignore_ascii_case("txt") && plotx_io::xps::is_casaxps_text(&path); - if supported_extension || recognized_raw || recognized_casaxps { + if supported_extension + || recognized_raw + || recognized_casaxps + || plotx_io::nmr_bridge::is_candidate(&path) + { output.push(path); } } } + Ok(()) +} + +/// Keep companion files out of the batch while doing all discovery I/O off-thread. +pub(super) fn discover_folder(folder: &Path) -> Result, String> { + let mut files = Vec::new(); + collect_data_files(folder, &mut files).map_err(|error| error.to_string())?; + if files.is_empty() { + return Ok(vec![folder.to_owned()]); + } + files.sort(); + let mut companions = std::collections::HashSet::new(); + for file in &files { + if file + .extension() + .is_some_and(|ext| ext.eq_ignore_ascii_case("pfc")) + { + // The actual import will surface a failed probe as a per-file error. + match plotx_io::load_path(file) { + Ok(loaded) => companions.extend(loaded.provenance.companion_paths), + Err(error) => { + log::warn!("Companion discovery failed for {}: {error}", file.display()) + } + } + } + } + files.retain(|file| !companions.contains(file)); + Ok(files) } #[cfg(test)] @@ -54,7 +78,7 @@ mod tests { std::fs::write(root.join("_FUNC001.IDX"), vec![0; 22]).unwrap(); std::fs::write(root.join("_FUNC001.DAT"), []).unwrap(); let mut found = Vec::new(); - collect_data_files(&root, &mut found); + collect_data_files(&root, &mut found).unwrap(); assert_eq!(found.as_slice(), std::slice::from_ref(&root)); std::fs::remove_dir_all(root).unwrap(); } @@ -71,7 +95,7 @@ mod tests { std::fs::write(&unrelated, b"not an XRD file").unwrap(); let mut found = Vec::new(); - collect_data_files(&root, &mut found); + collect_data_files(&root, &mut found).unwrap(); assert_eq!(found, vec![xrd]); std::fs::remove_dir_all(root).unwrap(); @@ -87,7 +111,7 @@ mod tests { std::fs::write(dataset.join("fid"), [0; 32]).unwrap(); let mut found = Vec::new(); - collect_data_files(&root, &mut found); + collect_data_files(&root, &mut found).unwrap(); assert_eq!(found, vec![dataset]); std::fs::remove_dir_all(root).unwrap(); @@ -105,7 +129,7 @@ mod tests { std::fs::write(root.join("sample.timeseries.data"), b"data").unwrap(); let mut found = Vec::new(); - collect_data_files(&root, &mut found); + collect_data_files(&root, &mut found).unwrap(); assert_eq!(found, vec![wiff]); std::fs::remove_dir_all(root).unwrap(); diff --git a/crates/app/src/ui/file_dialogs/nmr_sampling.rs b/crates/app/src/ui/file_dialogs/nmr_sampling.rs new file mode 100644 index 00000000..7b4a7bdb --- /dev/null +++ b/crates/app/src/ui/file_dialogs/nmr_sampling.rs @@ -0,0 +1,72 @@ +use plotx_core::state::{NmrImportDraft, PlotxApp}; + +pub(crate) fn open(app: &mut PlotxApp) { + if let Some(path) = rfd::FileDialog::new() + .set_title("Select a Bruker ser or JEOL JDF acquisition") + .add_filter("NMR acquisition", &["ser", "jdf"]) + .add_filter("All files", &["*"]) + .pick_file() + { + app.session.ui.nmr_import = Some(NmrImportDraft::new(path)); + } +} + +pub(crate) fn window(app: &mut PlotxApp, ctx: &egui::Context) { + let Some(mut draft) = app.session.ui.nmr_import.take() else { + return; + }; + let mut import = false; + let mut cancel = false; + let modal = super::super::modal(ctx, "nmr_sampling_import", super::super::ModalKind::Dialog) + .show(ctx, |ui| { + ui.set_width(520.0); + ui.heading("Import NMR with sampling table"); + ui.label(draft.path.display().to_string()); + ui.label("For 2D Bruker NUS or JEOL reduced-grid acquisitions. Supply the original acquisition grid and observation order."); + ui.separator(); + egui::Grid::new("nmr_sampling_fields").num_columns(2).show(ui, |ui| { + ui.label("Table source / explanation"); + ui.text_edit_singleline(&mut draft.source); + ui.end_row(); + ui.label("Original indirect grid points"); + ui.text_edit_singleline(&mut draft.grid); + ui.end_row(); + ui.label("Lanes per observation"); + ui.text_edit_singleline(&mut draft.lanes); + ui.end_row(); + ui.label("Index base"); + ui.horizontal(|ui| { + ui.radio_value(&mut draft.one_based, Some(false), "Zero-based"); + ui.radio_value(&mut draft.one_based, Some(true), "One-based"); + }); + ui.end_row(); + }); + ui.label("Indirect index: one observation per line, including repeats"); + egui::ScrollArea::vertical().max_height(180.0).show(ui, |ui| { + ui.add(egui::TextEdit::multiline(&mut draft.rows).desired_rows(6).desired_width(f32::INFINITY)); + }); + ui.label("Each row includes all lanes. The table must agree with the acquisition and any embedded list. Repeated observations are preserved; IST rejects repeats."); + if let Some(error) = &draft.error { + ui.colored_label(ui.visuals().error_fg_color, error); + } + ui.horizontal(|ui| { + import = ui.button("Validate and import").clicked(); + cancel = ui.button("Cancel").clicked(); + }); + }); + if import { + match draft.declaration() { + Ok(declaration) => { + if app.load_nmr_with_sampling(&draft.path, declaration) { + app.note_recent_file(&draft.path); + return; + } + draft.error = Some(app.session.status.clone()); + } + Err(error) => draft.error = Some(error), + } + } + if !cancel && !modal.should_close() { + app.session.ui.nmr_import = Some(draft); + } +} diff --git a/crates/app/src/ui/menus.rs b/crates/app/src/ui/menus.rs index 19532d68..a88d8df7 100644 --- a/crates/app/src/ui/menus.rs +++ b/crates/app/src/ui/menus.rs @@ -54,6 +54,7 @@ pub(crate) fn menu_bar_spec() -> Vec<(&'static str, Vec)> { Separator, Command(CommandId::OpenFile), Command(CommandId::OpenFolder), + Command(CommandId::ImportNmrSampling), Command(CommandId::RunBatchWorkflow), Submenu( "Open Recent", diff --git a/crates/app/src/ui/mod.rs b/crates/app/src/ui/mod.rs index 4ddefd4b..4c04eca9 100644 --- a/crates/app/src/ui/mod.rs +++ b/crates/app/src/ui/mod.rs @@ -81,6 +81,9 @@ pub fn render( sync_chrome_theme(&ctx, app.settings.appearance.theme); clipboard_table_paste.begin_frame(app, &ctx); file_dialogs::image_import::poll(app, &ctx); + if app.poll_data_import() { + ctx.request_repaint_after(std::time::Duration::from_millis(16)); + } file_dialogs::image_import::large_image_consent_window(app, &ctx); if let Some(payload) = app.poll_data_export() { copy_table_export(&ctx, payload); @@ -116,6 +119,7 @@ pub fn render( || app.session.ui.export_options.is_some() || app.session.ui.data_export.is_some() || app.session.ui.table_import_preview.is_some() + || app.session.ui.nmr_import.is_some() || app.session.ui.settings_dialog.is_some() || batch_workflow.is_open(); if !modal_open { @@ -201,6 +205,7 @@ pub fn render( quit_confirm_window(app, &ctx); diagnostic_history_window(app, &ctx); file_dialogs::processing_scheme_window(app, &ctx); + file_dialogs::nmr_sampling::window(app, &ctx); processing_templates::processing_template_window(app, &ctx); arithmetic::spectrum_arithmetic_window(app, &ctx); align::align_spectra_window(app, &ctx); @@ -217,6 +222,11 @@ pub fn render( app.finish_pending_wheel_zoom(now, false); app.finish_pending_wheel_property(now, false); activity::observe(app); + // Menus and drops can enqueue after this frame's poll. Schedule the first + // worker poll even if the user stops moving the mouse immediately afterward. + if app.session.data_imports.is_pending() { + ctx.request_repaint_after(std::time::Duration::from_millis(16)); + } } fn project_window_title(app: &PlotxApp) -> String { diff --git a/crates/app/src/ui/object_inspector/chart_gallery.rs b/crates/app/src/ui/object_inspector/chart_gallery.rs index d6398b77..076aa904 100644 --- a/crates/app/src/ui/object_inspector/chart_gallery.rs +++ b/crates/app/src/ui/object_inspector/chart_gallery.rs @@ -193,21 +193,34 @@ mod tests { nucleus: nucleus.to_owned(), group_delay: 0.0, }; - let nmr = Dataset::Nmr2D(Box::new(Nmr2DDataset::load(plotx_io::NmrData2D { - data: vec![num_complex::Complex64::new(1.0, 0.0); 4], - rows: 2, - cols: 2, - domain: plotx_io::Domain::Frequency, - direct: dimension("1H"), - indirect: dimension("13C"), - quad: plotx_io::QuadMode::Complex, - indirect_conjugate: false, - experiment: None, - pseudo_axis: None, - diffusion: None, - nus: None, - source: "gallery NMR".to_owned(), - }))); + let nmr = Dataset::Nmr2D(Box::new( + Nmr2DDataset::load_with_pipeline( + plotx_io::NmrData2D { + data: vec![num_complex::Complex64::new(1.0, 0.0); 4], + rows: 2, + cols: 2, + domain: plotx_io::Domain::Frequency, + direct: dimension("1H"), + indirect: dimension("13C"), + quad: plotx_io::QuadMode::Complex, + indirect_conjugate: false, + experiment: None, + pseudo_axis: None, + diffusion: None, + nus: None, + source: "gallery NMR".to_owned(), + }, + Some(plotx_processing::Params2D { + layout: plotx_processing::Layout2D::Ft, + f2: plotx_processing::AxisPipeline { steps: Vec::new() }, + f1: plotx_processing::AxisPipeline { steps: Vec::new() }, + }), + Some(false), + None, + true, + ) + .unwrap(), + )); let nmr_plane = nmr .field_descriptors() .into_iter() diff --git a/crates/app/src/ui/primary_sidebar/data_browser.rs b/crates/app/src/ui/primary_sidebar/data_browser.rs index a654c954..7196acfe 100644 --- a/crates/app/src/ui/primary_sidebar/data_browser.rs +++ b/crates/app/src/ui/primary_sidebar/data_browser.rs @@ -420,16 +420,21 @@ mod tests { use plotx_io::{Domain, NmrData}; fn root(name: &str) -> Dataset { - let mut dataset = NmrDataset::load(NmrData { - points: vec![1.0.into(), 0.0.into()], - domain: Domain::Frequency, - spectral_width_hz: 1.0, - observe_freq_mhz: 1.0, - carrier_ppm: 0.0, - nucleus: "1H".into(), - source: name.into(), - group_delay: 0.0, - }); + let mut dataset = NmrDataset::load_with_pipeline( + NmrData { + points: vec![1.0.into(), 0.0.into()], + domain: Domain::Frequency, + spectral_width_hz: 1.0, + observe_freq_mhz: 1.0, + carrier_ppm: 0.0, + nucleus: "1H".into(), + source: name.into(), + group_delay: 0.0, + }, + Some(plotx_processing::AxisPipeline { steps: Vec::new() }), + Some(false), + ) + .unwrap(); dataset.name = Some(name.into()); Dataset::Nmr(Box::new(dataset)) } diff --git a/crates/app/src/ui/properties/fixture.rs b/crates/app/src/ui/properties/fixture.rs index 0d632529..57e51c67 100644 --- a/crates/app/src/ui/properties/fixture.rs +++ b/crates/app/src/ui/properties/fixture.rs @@ -23,7 +23,7 @@ fn nmr1d(domain: plotx_io::Domain) -> Dataset { source: "fixture".to_owned(), group_delay: 0.0, }; - Dataset::Nmr(Box::new(NmrDataset::load(data))) + Dataset::Nmr(Box::new(NmrDataset::load(data).unwrap())) } pub(crate) fn time_domain_1d() -> Dataset { @@ -64,9 +64,9 @@ fn nmr2d(source: &str) -> plotx_io::NmrData2D { /// One page holding `plots` contour plots of one 2D spectrum, all selected. pub(crate) fn contour_page(plots: usize) -> (PlotxApp, Vec) { let mut app = PlotxApp::new(); - app.doc - .datasets - .push(Dataset::Nmr2D(Box::new(Nmr2DDataset::load(nmr2d("panel"))))); + app.doc.datasets.push(Dataset::Nmr2D(Box::new( + Nmr2DDataset::load(nmr2d("panel")).unwrap(), + ))); let mut canvas = CanvasDocument::new("page".to_owned(), [200.0, 200.0]); let mut ids = Vec::new(); for index in 0..plots { @@ -89,11 +89,9 @@ pub(crate) fn contour_page(plots: usize) -> (PlotxApp, Vec) { /// A second dataset, so a navigation test can tell whether the data focus /// followed the object it landed on. pub(crate) fn add_dataset(app: &mut PlotxApp) -> usize { - app.doc - .datasets - .push(Dataset::Nmr2D(Box::new(Nmr2DDataset::load(nmr2d( - "second", - ))))); + app.doc.datasets.push(Dataset::Nmr2D(Box::new( + Nmr2DDataset::load(nmr2d("second")).unwrap(), + ))); app.doc.datasets.len() - 1 } @@ -159,12 +157,12 @@ pub(crate) fn set_lowest_level(app: &mut PlotxApp, object: ObjectId, multiplier: pub(crate) fn time_domain_2d() -> Dataset { let mut data = nmr2d("time domain"); data.domain = plotx_io::Domain::Time; - Dataset::Nmr2D(Box::new(Nmr2DDataset::load(data))) + Dataset::Nmr2D(Box::new(Nmr2DDataset::load(data).unwrap())) } pub(crate) fn homonuclear_frequency_2d() -> Dataset { let mut data = nmr2d("homonuclear frequency domain"); data.indirect = data.direct.clone(); data.experiment = Some("cosy".to_owned()); - Dataset::Nmr2D(Box::new(Nmr2DDataset::load(data))) + Dataset::Nmr2D(Box::new(Nmr2DDataset::load(data).unwrap())) } diff --git a/crates/app/src/ui/tools/craft.rs b/crates/app/src/ui/tools/craft.rs index 90a6a958..e9a34291 100644 --- a/crates/app/src/ui/tools/craft.rs +++ b/crates/app/src/ui/tools/craft.rs @@ -18,7 +18,7 @@ pub(crate) fn open_for_active(app: &mut PlotxApp) { let Some(nmr) = app.doc.datasets.get(index).and_then(Dataset::as_nmr) else { return; }; - if nmr.data.domain != plotx_io::Domain::Time { + if nmr.input_domain() != plotx_io::Domain::Time { return; } let dataset = nmr.resource_id; @@ -105,7 +105,13 @@ pub(crate) fn select_regions_on_canvas(app: &mut PlotxApp, index: usize) { .to_owned(); return; }; - let invocation = setup::resolved(app, index); + let invocation = match setup::resolved(app, index) { + Ok(invocation) => invocation, + Err(error) => { + app.session.status = error; + return; + } + }; if app.session.ui.craft_overrides.regions.is_none() { app.session.ui.craft_overrides.regions = Some( if invocation.sources.regions @@ -180,7 +186,7 @@ pub(crate) fn render_task(app: &mut PlotxApp, host: &mut Ui) { || !app.doc.datasets.get(index).is_some_and(|dataset| { dataset .as_nmr() - .is_some_and(|nmr| nmr.data.domain == plotx_io::Domain::Time) + .is_some_and(|nmr| nmr.input_domain() == plotx_io::Domain::Time) }) { return; @@ -322,7 +328,7 @@ mod tests { fn stored_run(data: &NmrData, params: CraftParams) -> StoredCraftRun { StoredCraftRun::from_result( CraftRunId(0), - data, + &data.clone().try_into().unwrap(), CraftInvocation::acquisition(data, params), None, CraftResult { @@ -347,13 +353,14 @@ mod tests { #[test] fn changing_target_rebuilds_draft_from_target_provenance() { - let first = NmrDataset::load(time_domain_data("first")); - let mut second = NmrDataset::load(time_domain_data("second")); + let first = NmrDataset::load(time_domain_data("first")).unwrap(); + let mut second = NmrDataset::load(time_domain_data("second")).unwrap(); let mut provenance_params = CraftParams::ssfp(); provenance_params.minimum_amplitude_to_noise = 8.5; - second - .craft_runs - .push(stored_run(&second.data, provenance_params.clone())); + second.craft_runs.push(stored_run( + &second.data.craft_fid().unwrap(), + provenance_params.clone(), + )); let mut app = PlotxApp::new_with_settings(plotx_core::settings::Settings::default()); app.doc.datasets.push(Dataset::Nmr(Box::new(first))); app.doc.datasets.push(Dataset::Nmr(Box::new(second))); @@ -371,7 +378,10 @@ mod tests { open_for_active(&mut app); assert_eq!(app.session.ui.craft_overrides, Default::default()); - assert_eq!(setup::resolved(&mut app, 1).params, provenance_params); + assert_eq!( + setup::resolved(&mut app, 1).unwrap().params, + provenance_params + ); assert_eq!(app.session.ui.craft_selected_run, Some(CraftRunId(0))); assert_eq!(app.session.ui.craft_task_page, CraftTaskPage::Results); } @@ -398,11 +408,11 @@ mod tests { + Complex64::from_polar(3.0, -std::f64::consts::TAU * 250.0 * time) }) .collect(); - let dataset = NmrDataset::load(data); + let dataset = NmrDataset::load(data).unwrap(); let invocation = plotx_processing::craft::resolve_craft_invocation( - &dataset.data, - dataset.craft_reference(), + &dataset.data.craft_fid().unwrap(), + dataset.craft_reference().unwrap(), &plotx_processing::craft::CraftParamOverrides { fir_filter_taps: Some(31), ..Default::default() diff --git a/crates/app/src/ui/tools/craft/results.rs b/crates/app/src/ui/tools/craft/results.rs index 9cd51fc7..b39d941f 100644 --- a/crates/app/src/ui/tools/craft/results.rs +++ b/crates/app/src/ui/tools/craft/results.rs @@ -277,7 +277,8 @@ fn reports( .changed(); ui.label(format!( "Hz ({:.5} ppm)", - definition.segment_width_hz / nmr.data.observe_freq_mhz + definition.segment_width_hz + / run.provenance.invocation.reference.reference_frequency_mhz )); }); let mut snapshot: CraftAmplitudeReport = serde_json::from_value(record.snapshot.clone()) diff --git a/crates/app/src/ui/tools/craft/setup.rs b/crates/app/src/ui/tools/craft/setup.rs index fc08d608..2949ba59 100644 --- a/crates/app/src/ui/tools/craft/setup.rs +++ b/crates/app/src/ui/tools/craft/setup.rs @@ -9,7 +9,13 @@ use plotx_processing::craft::{ use crate::ui::commands::CommandId; pub(super) fn show(app: &mut PlotxApp, index: usize, ui: &mut Ui) { - let invocation = resolved(app, index); + let invocation = match resolved(app, index) { + Ok(invocation) => invocation, + Err(error) => { + ui.colored_label(ui.visuals().error_fg_color, error); + return; + } + }; settings(app, index, &invocation, ui); ui.separator(); readiness(app.session.ui.craft_analysis_intent, &invocation, ui); @@ -17,10 +23,12 @@ pub(super) fn show(app: &mut PlotxApp, index: usize, ui: &mut Ui) { run_controls(app, index, ui); } -pub(super) fn resolved(app: &mut PlotxApp, index: usize) -> CraftInvocation { +pub(super) fn resolved(app: &mut PlotxApp, index: usize) -> Result { let nmr = app.doc.datasets[index].as_nmr().unwrap(); let dataset = nmr.resource_id; - let reference = nmr.craft_reference(); + let reference = nmr + .craft_reference() + .ok_or("CRAFT requires chemical-shift reference evidence")?; let parent_run = app.session.ui.craft_base_run; if let Some(cache) = &app.session.ui.craft_resolution_cache && cache.dataset == dataset @@ -29,10 +37,11 @@ pub(super) fn resolved(app: &mut PlotxApp, index: usize) -> CraftInvocation { && cache.overrides == app.session.ui.craft_overrides && cache.parent_run == parent_run { - return cache.invocation.clone(); + return Ok(cache.invocation.clone()); } + let data = nmr.data.craft_fid().map_err(|error| error.to_string())?; let invocation = resolve_craft_invocation( - &nmr.data, + &data, reference, &app.session.ui.craft_overrides, parent_run.and_then(|id| nmr.craft_run(id).map(|run| &run.provenance.invocation)), @@ -45,7 +54,7 @@ pub(super) fn resolved(app: &mut PlotxApp, index: usize) -> CraftInvocation { parent_run, invocation: invocation.clone(), }); - invocation + Ok(invocation) } fn readiness(intent: CraftAnalysisIntent, invocation: &CraftInvocation, ui: &mut Ui) { @@ -102,7 +111,7 @@ fn readiness(intent: CraftAnalysisIntent, invocation: &CraftInvocation, ui: &mut fn settings(app: &mut PlotxApp, index: usize, invocation: &CraftInvocation, ui: &mut Ui) { let nmr = app.doc.datasets[index].as_nmr().unwrap().clone(); - let reference = nmr.craft_reference(); + let reference = invocation.reference; ui.label(crate::typography::headline("1. Choose the analysis goal")); ui.horizontal_wrapped(|ui| { ui.selectable_value( @@ -155,7 +164,7 @@ fn settings(app: &mut PlotxApp, index: usize, invocation: &CraftInvocation, ui: ui.weak(format!( "Chemical-shift axis: acquisition {:.5} ppm · reference {:+.5} ppm · effective {:.5} ppm", - nmr.data.carrier_ppm, + reference.acquisition_carrier_ppm, reference.offset_ppm, reference.effective_carrier_ppm(), )); @@ -339,7 +348,7 @@ fn regions( regions.remove(position); changed = true; } - let half_width = 45.0 / nmr.data.observe_freq_mhz.max(f64::MIN_POSITIVE); + let half_width = 45.0 / invocation.reference.reference_frequency_mhz; let suggestions = invocation .assessment .clear_signals @@ -375,7 +384,7 @@ fn regions( }); } if ui.small_button("Add custom region").clicked() { - let center = nmr.craft_reference().effective_carrier_ppm(); + let center = invocation.reference.effective_carrier_ppm(); regions.push(CraftRegion::new( next_region_id(®ions), center - half_width, @@ -428,7 +437,13 @@ fn run_controls(app: &mut PlotxApp, index: usize, ui: &mut Ui) { _ => ui.label("Run"), }; super::command_button(app, CommandId::RunCraft, "Run CRAFT", true, ui); - let invocation = resolved(app, index); + let invocation = match resolved(app, index) { + Ok(invocation) => invocation, + Err(error) => { + ui.colored_label(ui.visuals().error_fg_color, error); + return; + } + }; if let Some(message) = invocation.assessment.first_blocking_message() { ui.colored_label(ui.visuals().error_fg_color, message); } diff --git a/crates/app/src/ui/tools/processing/mod.rs b/crates/app/src/ui/tools/processing/mod.rs index f47d16ae..6e7de553 100644 --- a/crates/app/src/ui/tools/processing/mod.rs +++ b/crates/app/src/ui/tools/processing/mod.rs @@ -172,8 +172,11 @@ fn move_entry( fn add_step_menu(app: &mut PlotxApp, di: usize, axis: PhaseAxis, ui: &mut Ui) { let dataset = &app.doc.datasets[di]; let input_domain = match dataset { - Dataset::Nmr(dataset) => dataset.data.domain, - Dataset::Nmr2D(dataset) => dataset.data.domain, + Dataset::Nmr(dataset) => dataset.input_domain(), + Dataset::Nmr2D(dataset) => match dataset.input_domain(axis) { + Ok(domain) => domain, + Err(_) => return, + }, Dataset::Table(_) | Dataset::Electrophysiology(_) | Dataset::Afm(_) @@ -273,7 +276,7 @@ fn default_bin_params(app: &PlotxApp, dataset: usize) -> BinParams { let Some(spectrum) = dataset.spectrum() else { return BinParams::DEFAULT; }; - let effective_minimum = 1.5 * plotx_processing::cleanup::axis_step(&spectrum.ppm); + let effective_minimum = 1.5 * spectrum.coordinate_spacing().unwrap_or(0.0); BinParams { width: BinParams::DEFAULT.width.max(effective_minimum.next_up()), ..BinParams::DEFAULT @@ -323,8 +326,11 @@ fn apply_row_op(app: &mut PlotxApp, di: usize, axis: PhaseAxis, id: StepId, op: }; let owner = dataset.resource_id(); let input_domain = match dataset { - Dataset::Nmr(dataset) => dataset.data.domain, - Dataset::Nmr2D(dataset) => dataset.data.domain, + Dataset::Nmr(dataset) => dataset.input_domain(), + Dataset::Nmr2D(dataset) => match dataset.input_domain(axis) { + Ok(domain) => domain, + Err(_) => return, + }, Dataset::Table(_) | Dataset::Electrophysiology(_) | Dataset::Afm(_) @@ -522,7 +528,7 @@ mod tests { nus: None, source: "default badge".to_owned(), }; - let mut dataset = Dataset::Nmr2D(Box::new(Nmr2DDataset::load(data))); + let mut dataset = Dataset::Nmr2D(Box::new(Nmr2DDataset::load(data).unwrap())); assert!(is_default_processing(&dataset)); dataset.as_nmr2d_mut().unwrap().group_delay_correct = false; assert!(!is_default_processing(&dataset)); diff --git a/crates/app/src/ui/tools/processing/surface.rs b/crates/app/src/ui/tools/processing/surface.rs index ac5b1d04..e16f6cc1 100644 --- a/crates/app/src/ui/tools/processing/surface.rs +++ b/crates/app/src/ui/tools/processing/surface.rs @@ -45,15 +45,15 @@ struct SurfaceShape { fn surface_shape(dataset: &Dataset) -> Option { let (input_domain, source) = match dataset { Dataset::Nmr(dataset) => ( - dataset.data.domain, - match dataset.data.domain { + dataset.input_domain(), + match dataset.input_domain() { Domain::Time => SourceShape::RawFid, Domain::Frequency => SourceShape::ImportedSpectrum, }, ), Dataset::Nmr2D(dataset) => ( - dataset.data.domain, - match dataset.data.domain { + dataset.input_domain(PhaseAxis::F2).ok()?, + match dataset.input_domain(PhaseAxis::F2).ok()? { Domain::Time => SourceShape::RawAcquisition2D, Domain::Frequency => SourceShape::ImportedSpectrum, }, diff --git a/crates/app/src/ui/tools/pseudo.rs b/crates/app/src/ui/tools/pseudo.rs index d3d8ee41..62faeb91 100644 --- a/crates/app/src/ui/tools/pseudo.rs +++ b/crates/app/src/ui/tools/pseudo.rs @@ -32,6 +32,12 @@ pub(super) fn experiment_group(app: &mut PlotxApp, di: usize, ui: &mut Ui) -> bo let n = app.doc.datasets[di].as_nmr2d().unwrap(); let layout = match n.params.layout { Layout2D::Ft => "Contour (true 2D FT)", + Layout2D::Stack + if n.native_processed.dataset().as_raw().is_some() && n.data.nus.is_some() => + { + "Acquired NUS observations (not reconstructed)" + } + Layout2D::Stack if n.data.nus.is_some() => "Stack (reconstructed NUS slices)", Layout2D::Stack => "Stack (pseudo-2D 1D slices)", }; ui.label(format!("Layout: {layout}")); @@ -48,7 +54,7 @@ pub(super) fn experiment_group(app: &mut PlotxApp, di: usize, ui: &mut Ui) -> bo if is_pseudo { ui.separator(); pseudo_group(app, di, ui); - } else if is_stack { + } else if is_stack && app.doc.datasets[di].as_nmr2d().unwrap().data.nus.is_none() { ui.separator(); ui.small( "This looks like a pseudo-2D array but no indirect-axis ruler \ @@ -65,99 +71,59 @@ pub(super) fn experiment_group(app: &mut PlotxApp, di: usize, ui: &mut Ui) -> bo false } -/// Non-uniform-sampling controls. The reader normally recovers JEOL schedules; -/// manual entry remains available for older or malformed files. +/// Reconstruction inputs apply to the imported sampling coordinates. fn nus_group(app: &mut PlotxApp, di: usize, ui: &mut Ui) { - let Some(nus) = app.doc.datasets[di] - .as_nmr2d() - .and_then(|n| n.data.nus.clone()) - else { + let Some(dataset) = app.doc.datasets[di].as_nmr2d() else { + return; + }; + let Some(nus) = &dataset.data.nus else { return; }; ui.separator(); ui.label(crate::typography::headline("Non-uniform sampling")); - let scheme = if nus.echo_antiecho { - "echo/anti-echo (P/N)" - } else { - "phase-modulated" - }; + if let Some(warning) = &dataset.reconstruction_warning { + ui.colored_label(ui.visuals().warn_fg_color, warning); + } ui.small(format!( - "{} scheduling — {} of {} F1 increments acquired ({}).", - nus.mode, nus.acquired, nus.grid, scheme, + "{} observations on a {}-point indirect grid.", + nus.acquired, nus.grid )); - - if nus.schedule.is_some() { - ui.small("Spectrum reconstructed from the available sampling list."); - } else { - ui.colored_label( - ui.visuals().warn_fg_color, - "No valid NUS schedule was found in the data file. Paste the sampling \ - list (space/comma separated) to reconstruct the spectrum.", - ); - } - - let text_id = ui.make_persistent_id(("nus_list", di)); - let base_id = ui.make_persistent_id(("nus_base", di)); - let err_id = ui.make_persistent_id(("nus_err", di)); - let mut text = ui.data_mut(|d| d.get_temp::(text_id).unwrap_or_default()); - let mut base = ui.data_mut(|d| d.get_temp::(base_id).unwrap_or(nus.idx_base)); - - ui.horizontal(|ui| { - ui.label("Index base"); - if ui.selectable_label(base == 1, "1-based").clicked() { - base = 1; - } - if ui.selectable_label(base == 0, "0-based").clicked() { - base = 0; - } - }); - ui.data_mut(|d| d.insert_temp(base_id, base)); - - let resp = ui.add( - egui::TextEdit::multiline(&mut text) - .hint_text("1 2 3 5 7 9 …") - .desired_rows(2) - .desired_width(f32::INFINITY), - ); - if resp.changed() { - ui.data_mut(|d| d.insert_temp(text_id, text.clone())); - } - - if ui - .add(Button::new(format!( - "Reconstruct ({} indices)", - nus.acquired - ))) - .clicked() - { - let result = match parse_indices(&text) { - Ok(values) => app.apply_nus_schedule(di, &values, base), - Err(e) => Err(e), - }; - let err = result.err().unwrap_or_default(); - ui.data_mut(|d| d.insert_temp::(err_id, err)); - } - let err = ui.data_mut(|d| d.get_temp::(err_id).unwrap_or_default()); - if !err.is_empty() { - ui.colored_label(ui.visuals().error_fg_color, err); - } -} - -fn parse_indices(text: &str) -> Result, String> { - let mut out = Vec::new(); - for tok in text - .split(|c: char| c.is_whitespace() || c == ',' || c == ';') - .filter(|s| !s.is_empty()) - { - let v: usize = tok - .parse() - .map_err(|_| format!("'{tok}' is not a whole number."))?; - out.push(v); + let mut request = dataset.nus_request.unwrap_or_default(); + let mut noise_known = request.noise_standard_deviation.is_some(); + let mut changed = ui + .checkbox(&mut noise_known, "Override automatic noise estimate") + .changed(); + let mut noise = request.noise_standard_deviation.unwrap_or(0.0); + changed |= ui + .add_enabled( + noise_known, + DragValue::new(&mut noise) + .range(0.0..=f64::MAX) + .prefix("Noise σ "), + ) + .changed(); + ui.small("Noise is estimated automatically from acquired data. An override uses the standard deviation after the current F2 recipe; zero explicitly asserts noiseless input."); + request.noise_standard_deviation = noise_known.then_some(noise); + changed |= ui + .add( + DragValue::new(&mut request.max_iterations) + .range(1..=2048) + .prefix("Maximum iterations "), + ) + .changed(); + let before = DatasetProcessingState::from_dataset(&app.doc.datasets[di]); + let mut after = before.clone(); + if let DatasetProcessingState::Nmr2D { nus_request, .. } = &mut after { + *nus_request = Some(request); } - if out.is_empty() { - return Err("Enter the sampling indices.".into()); + if changed && before != after { + app.execute_action(Action::update_dataset_processing( + app.doc.datasets[di].resource_id(), + before, + after, + )); } - Ok(out) + ui.small("NUS reconstruction runs automatically with the F2 FFT. The F1 FFT produces the second frequency axis."); } fn pseudo_group(app: &mut PlotxApp, di: usize, ui: &mut Ui) { @@ -302,6 +268,12 @@ fn pseudo_group(app: &mut PlotxApp, di: usize, ui: &mut Ui) { } } + let input_error = app.doc.datasets[di] + .as_nmr2d() + .and_then(|n| n.dosy_input_error()); + if let Some(error) = input_error { + ui.small(error); + } let progress = app .doc .datasets @@ -311,7 +283,10 @@ fn pseudo_group(app: &mut PlotxApp, di: usize, ui: &mut Ui) { if is_dosy && !is_ilt && ui - .add_enabled(progress.is_none(), Button::new("Build DOSY map")) + .add_enabled( + input_error.is_none() && progress.is_none(), + Button::new("Build DOSY map"), + ) .clicked() { app.request_dosy_map(di); @@ -320,7 +295,7 @@ fn pseudo_group(app: &mut PlotxApp, di: usize, ui: &mut Ui) { && is_ilt && ui .add_enabled( - is_gradient && progress.is_none(), + is_gradient && input_error.is_none() && progress.is_none(), Button::new("Build ILT DOSY map"), ) .clicked() diff --git a/crates/app/src/ui/tools/task_card_tests.rs b/crates/app/src/ui/tools/task_card_tests.rs index aa1a6f8e..0e60fed7 100644 --- a/crates/app/src/ui/tools/task_card_tests.rs +++ b/crates/app/src/ui/tools/task_card_tests.rs @@ -15,9 +15,14 @@ fn app_with_task(tab: TaskDockTab, collapsed: bool) -> PlotxApp { source: "test".into(), group_delay: 0.0, }; - app.doc - .datasets - .push(Dataset::Nmr(Box::new(NmrDataset::load(data)))); + app.doc.datasets.push(Dataset::Nmr(Box::new( + NmrDataset::load_with_pipeline( + data, + Some(plotx_processing::AxisPipeline { steps: Vec::new() }), + Some(false), + ) + .unwrap(), + ))); app.doc .canvases .push(CanvasDocument::new("p".into(), [100.0, 80.0])); diff --git a/crates/cli/Cargo.toml b/crates/cli/Cargo.toml index 6ecd7feb..e617d028 100644 --- a/crates/cli/Cargo.toml +++ b/crates/cli/Cargo.toml @@ -17,7 +17,11 @@ default = [] datafusion = ["plotx-core/datafusion"] [dependencies] +nmr.workspace = true plotx-core.workspace = true plotx-io.workspace = true plotx-processing.workspace = true serde_json.workspace = true + +[dev-dependencies] +tempfile.workspace = true diff --git a/crates/cli/src/craft.rs b/crates/cli/src/craft.rs index 9bfb6298..88923424 100644 --- a/crates/cli/src/craft.rs +++ b/crates/cli/src/craft.rs @@ -78,17 +78,23 @@ pub(super) fn run( } fn analyze(path: &Path, overrides: &CraftParamOverrides) -> Result { - let loaded = workflow::load_dataset(path).map_err(|error| error.to_string())?; - let dataset = loaded - .dataset - .as_nmr() - .ok_or_else(|| "CRAFT requires a one-dimensional NMR dataset".to_owned())?; - let invocation = resolve_craft_invocation( - &dataset.data, - plotx_processing::craft::CraftReference::acquisition(&dataset.data), - overrides, - None, + let input = plotx_io::nmr_bridge::read(path, &mut nmr::ExecutionContext::default()) + .map_err(|error| error.to_string())?; + let inspection = workflow::inspect_nmr_dataset(&input).map_err(|error| error.to_string())?; + let source = plotx_io::nmr_view::NmrSource::new(input).map_err(|error| error.to_string())?; + let data = source.craft_fid().map_err(|error| error.to_string())?; + let reference = source + .dataset() + .as_raw() + .and_then(|raw| raw.descriptor().axes().first()) + .and_then(|axis| axis.chemical_shift_reference()) + .ok_or("CRAFT requires chemical-shift reference evidence")?; + let reference = plotx_processing::craft::CraftReference::new( + reference.carrier_ppm(), + reference.reference_frequency_mhz(), + 0.0, ); + let invocation = resolve_craft_invocation(&data, reference, overrides, None); if !invocation.assessment.can_run() { return Err(invocation .assessment @@ -96,9 +102,15 @@ fn analyze(path: &Path, overrides: &CraftParamOverrides) -> Result Result values[0].norm() && magnitude >= values[2].norm()).then(|| { - json!({ - "chemical_shift_ppm": spectrum.ppm[index + 1], - "magnitude": magnitude, - }) + let spectrum = plotx_processing::craft::preview_spectrum( + &data, + invocation.reference, + invocation.derived_plan.effective_skip_points, + ) + .map_err(|error| error.to_string())?; + let mut fft_peaks = { + spectrum + .values + .windows(3) + .enumerate() + .filter_map(|(index, values)| { + let magnitude = values[1].norm(); + (magnitude > values[0].norm() && magnitude >= values[2].norm()).then(|| { + json!({ + "chemical_shift_ppm": spectrum.ppm[index + 1], + "magnitude": magnitude, }) }) - .collect::>() - }) - .unwrap_or_default(); + }) + .collect::>() + }; fft_peaks.sort_by(|left, right| { right["magnitude"] .as_f64() @@ -140,16 +155,14 @@ fn analyze(path: &Path, overrides: &CraftParamOverrides) -> Result Result>(); - peaks.sort_by(|left, right| { - right["magnitude"] - .as_f64() - .unwrap_or_default() - .total_cmp(&left["magnitude"].as_f64().unwrap_or_default()) - }); - peaks.truncate(5); - json!({ "region": region, "strongest_bins": peaks }) - }) - .collect::>() - }) - .unwrap_or_default(); + peaks.sort_by(|left, right| { + right["magnitude"] + .as_f64() + .unwrap_or_default() + .total_cmp(&left["magnitude"].as_f64().unwrap_or_default()) + }); + peaks.truncate(5); + json!({ "region": region, "strongest_bins": peaks }) + }) + .collect::>() + }; let region_amplitude_ratio = result.region_ratio.map(|ratio| ratio.value); Ok(json!({ "input": path, @@ -180,15 +192,16 @@ fn analyze(path: &Path, overrides: &CraftParamOverrides) -> Result Result Result, String> { - if !input.is_dir() || input.extension().is_some() || is_raw_acquisition(input) { + if !input.is_dir() || input.extension().is_some() || is_raw_acquisition(input)? { return Ok(vec![input.to_owned()]); } let mut children = std::fs::read_dir(input) .map_err(|error| format!("could not read {}: {error}", input.display()))? - .filter_map(Result::ok) - .map(|entry| entry.path()) - .filter(|path| path.is_dir() && is_raw_acquisition(path)) - .collect::>(); + .map(|entry| entry.map(|entry| entry.path())) + .collect::, _>>() + .map_err(|error| format!("could not enumerate {}: {error}", input.display()))? + .into_iter() + .try_fold(Vec::new(), |mut paths, path| { + if path.is_dir() && is_raw_acquisition(&path)? { + paths.push(path); + } + Ok::<_, String>(paths) + })?; children.sort(); if children.is_empty() { Err(format!( @@ -224,13 +243,12 @@ fn acquisition_inputs(input: &Path) -> Result, String> { } } -fn is_raw_acquisition(path: &Path) -> bool { - matches!( - plotx_io::detect_format(path), - Ok(plotx_io::DataFormat::Nmr( - plotx_io::NmrFormat::BrukerRaw | plotx_io::NmrFormat::VarianAgilentRaw, - )) - ) +fn is_raw_acquisition(path: &Path) -> Result { + match plotx_io::nmr_bridge::read_options().detect(path) { + Ok(format) => Ok(matches!(format, nmr::Format::Raw(_))), + Err(error) if error.kind() == nmr::ReadErrorKind::Unrecognized => Ok(false), + Err(error) => Err(format!("could not detect {}: {error}", path.display())), + } } #[cfg(test)] diff --git a/crates/cli/src/main.rs b/crates/cli/src/main.rs index f23d0788..bda30039 100644 --- a/crates/cli/src/main.rs +++ b/crates/cli/src/main.rs @@ -15,9 +15,9 @@ mod craft; const HELP: &str = r#"plotx-cli - headless PlotX workflows USAGE: - plotx-cli inspect [--json] + plotx-cli inspect [--json] [--sampling-declaration ] plotx-cli craft --output [--region ]... [--expected-ratio ]... - plotx-cli process --scheme --output [--format svg|pdf|png|tiff|jpeg] + plotx-cli process --scheme --output [--format svg|pdf|png|tiff|jpeg] [--sampling-declaration ] plotx-cli batch --workflow --manifest COMMANDS: @@ -63,12 +63,14 @@ enum Command { Inspect { input: PathBuf, json: bool, + sampling_declaration: Option, }, Process { input: PathBuf, scheme: PathBuf, output: PathBuf, format: OutputFormat, + sampling_declaration: Option, }, Craft { input: PathBuf, @@ -138,6 +140,7 @@ enum Flag { Manifest, Region, ExpectedRatio, + SamplingDeclaration, } impl Flag { @@ -152,6 +155,7 @@ impl Flag { Some("--manifest") => Ok(Some(Self::Manifest)), Some("--region") => Ok(Some(Self::Region)), Some("--expected-ratio") => Ok(Some(Self::ExpectedRatio)), + Some("--sampling-declaration") => Ok(Some(Self::SamplingDeclaration)), Some(value) if value.starts_with('-') => { Err(ParseError::new(format!("unknown option: {value}"))) } @@ -170,6 +174,7 @@ impl Flag { Self::Manifest => "--manifest", Self::Region => "--region", Self::ExpectedRatio => "--expected-ratio", + Self::SamplingDeclaration => "--sampling-declaration", } } } @@ -311,11 +316,18 @@ fn parse_batch(mut args: VecDeque) -> Result fn parse_inspect(mut args: VecDeque) -> Result { let mut input = None; let mut json = false; + let mut sampling_declaration = None; while let Some(token) = args.pop_front() { match Flag::parse(&token)? { Some(Flag::Help) => return Ok(ParseOutcome::Help), Some(Flag::Json) if !json => json = true, Some(Flag::Json) => return Err(ParseError::new("--json was provided more than once")), + Some(Flag::SamplingDeclaration) if sampling_declaration.is_none() => { + sampling_declaration = Some(PathBuf::from(take_value( + &mut args, + Flag::SamplingDeclaration, + )?)); + } Some(flag) => { return Err(ParseError::new(format!( "{} is not valid for inspect", @@ -329,6 +341,7 @@ fn parse_inspect(mut args: VecDeque) -> Result"))?, json, + sampling_declaration, })) } @@ -337,6 +350,7 @@ fn parse_process(mut args: VecDeque) -> Result return Ok(ParseOutcome::Help), @@ -353,6 +367,12 @@ fn parse_process(mut args: VecDeque) -> Result return Err(ParseError::new("--json is not valid for process")), + Some(Flag::SamplingDeclaration) if sampling_declaration.is_none() => { + sampling_declaration = Some(PathBuf::from(take_value( + &mut args, + Flag::SamplingDeclaration, + )?)); + } Some(flag) => { return Err(ParseError::new(format!( "{} was provided more than once", @@ -374,6 +394,7 @@ fn parse_process(mut args: VecDeque) -> Result std::process::ExitCode { fn run(command: Command) -> Status { match command { - Command::Inspect { input, json } => { + Command::Inspect { + input, + json, + sampling_declaration, + } => { eprintln!("plotx-cli: loading {}", input.display()); - match workflow::load_dataset(&input) { - Ok(loaded) => { - emit_warnings(&loaded.inspection); + let inspection = if let Some(path) = sampling_declaration { + plotx_io::nmr_sampling::read_declaration(&path) + .and_then(|declaration| { + plotx_io::nmr_sampling::read( + &input, + declaration, + &mut nmr::ExecutionContext::default(), + ) + }) + .map_err(WorkflowError::from) + .and_then(|dataset| workflow::inspect_nmr_dataset(&dataset)) + } else { + workflow::inspect_file(&input) + }; + match inspection { + Ok(inspection) => { + emit_warnings(&inspection); let result = if json { - serde_json::to_string_pretty(&loaded.inspection) + serde_json::to_string_pretty(&inspection) } else { - Ok(text_report(&loaded.inspection)) + Ok(text_report(&inspection)) }; match result { Ok(output) => { @@ -452,13 +491,26 @@ fn run(command: Command) -> Status { scheme, output, format, + sampling_declaration, } => { eprintln!( "plotx-cli: processing {} with {}", input.display(), scheme.display() ); - match workflow::process_file(&input, &scheme, &output, format.0) { + let result = if let Some(path) = sampling_declaration { + plotx_io::nmr_sampling::read_declaration(&path) + .map_err(WorkflowError::from) + .and_then(|declaration| { + workflow::load_dataset_with_sampling(&input, declaration) + }) + .and_then(|loaded| { + workflow::process_loaded_dataset(loaded, &scheme, &output, format.0) + }) + } else { + workflow::process_file(&input, &scheme, &output, format.0) + }; + match result { Ok(result) => { emit_warnings(&result.inspection); let value = json!({ @@ -548,7 +600,7 @@ fn fail_automation(error: AutomationError) -> Status { fn fail(error: WorkflowError) -> Status { let status = match &error { - WorkflowError::Load(_) => Status::Input, + WorkflowError::Load(_) | WorkflowError::Nmr(_) => Status::Input, WorkflowError::Scheme(_) | WorkflowError::Processing(_) | WorkflowError::Integration(_) @@ -639,148 +691,5 @@ fn text_report(report: &InspectionReport) -> String { } #[cfg(test)] -mod tests { - use super::*; - - fn parse(values: &[&str]) -> Result { - parse_args(values.iter().map(OsString::from)) - } - - #[test] - fn inspect_parser_accepts_json_on_either_side_of_input() { - let expected = ParseOutcome::Command(Command::Inspect { - input: "sample.jdf".into(), - json: true, - }); - assert_eq!( - parse(&["plotx-cli", "inspect", "--json", "sample.jdf"]), - Ok(expected.clone()) - ); - assert_eq!( - parse(&["plotx-cli", "inspect", "sample.jdf", "--json"]), - Ok(expected) - ); - } - - #[test] - fn process_parser_infers_format_and_requires_named_paths() { - assert_eq!( - parse(&[ - "plotx-cli", - "process", - "sample.jdf", - "--scheme", - "routine.plotxproc", - "--output", - "figure.svg", - ]), - Ok(ParseOutcome::Command(Command::Process { - input: "sample.jdf".into(), - scheme: "routine.plotxproc".into(), - output: "figure.svg".into(), - format: OutputFormat(ExportFormat::Svg), - })) - ); - assert!(parse(&["plotx-cli", "process", "sample.jdf"]).is_err()); - } - - #[test] - fn craft_parser_accepts_multiple_and_negative_ppm_regions() { - assert_eq!( - parse(&[ - "plotx-cli", - "craft", - "acquisitions", - "--region", - "-0.5:0.2", - "--region", - "6.3:6.5", - "--expected-ratio", - "0.75", - "--output", - "result.json", - ]), - Ok(ParseOutcome::Command(Command::Craft { - input: "acquisitions".into(), - output: "result.json".into(), - regions: vec![ - plotx_processing::craft::CraftRegion::new( - plotx_processing::craft::CraftRegionId(0), - -0.5, - 0.2, - ), - plotx_processing::craft::CraftRegion::new( - plotx_processing::craft::CraftRegionId(1), - 6.3, - 6.5, - ), - ], - expected_ratios: vec![0.75], - })) - ); - } - - #[test] - fn batch_parser_requires_workflow_and_manifest_paths() { - assert_eq!( - parse(&[ - "plotx-cli", - "batch", - "--workflow", - "workflow.json", - "--manifest", - "run.json", - ]), - Ok(ParseOutcome::Command(Command::Batch { - workflow: "workflow.json".into(), - manifest: "run.json".into(), - })) - ); - assert!(parse(&["plotx-cli", "batch", "workflow.json"]).is_err()); - } - - #[test] - fn text_inspection_includes_mass_spectrometry_statistics() { - let report = InspectionReport { - schema: plotx_core::workflow::INSPECTION_SCHEMA, - format: "sciex-wiff".to_owned(), - provenance: plotx_core::workflow::ProvenanceReport { - selected_path: "sample.wiff".into(), - data_path: "sample.wiff".into(), - parameter_paths: Vec::new(), - companion_paths: vec!["sample.wiff.scan".into()], - }, - dimension: plotx_core::workflow::DimensionReport { - count: 3, - shape: vec![2, 42, 1], - }, - domain: "mass_spectrometry".to_owned(), - warnings: Vec::new(), - electrophysiology: None, - afm: None, - mass_spectrometry: Some(plotx_core::workflow::MassSpecReport { - instrument: Some("SCIEX TripleTOF 6600".to_owned()), - stream_count: 2, - ms_scan_count: 42, - chromatograms: vec!["total ion current chromatogram".to_owned()], - }), - xrd: None, - xps: None, - }; - - let output = text_report(&report); - - assert!(output.contains("format: sciex-wiff")); - assert!(output.contains("mass_spec.streams: 2")); - assert!(output.contains("mass_spec.scans: 42")); - assert!(output.contains("mass_spec.chromatograms: total ion current chromatogram")); - } - - #[test] - fn workflow_errors_map_to_stable_exit_categories() { - let status = fail(WorkflowError::FigureUnavailable("NMR 1D")); - assert_eq!(status, Status::Canvas); - assert_eq!(Status::Usage as u8, 2); - assert_eq!(Status::Export as u8, 6); - } -} +#[path = "main_tests.rs"] +mod tests; diff --git a/crates/cli/src/main_tests.rs b/crates/cli/src/main_tests.rs new file mode 100644 index 00000000..fee820a5 --- /dev/null +++ b/crates/cli/src/main_tests.rs @@ -0,0 +1,145 @@ +use super::*; + +fn parse(values: &[&str]) -> Result { + parse_args(values.iter().map(OsString::from)) +} + +#[test] +fn inspect_parser_accepts_json_on_either_side_of_input() { + let expected = ParseOutcome::Command(Command::Inspect { + input: "sample.jdf".into(), + json: true, + sampling_declaration: None, + }); + assert_eq!( + parse(&["plotx-cli", "inspect", "--json", "sample.jdf"]), + Ok(expected.clone()) + ); + assert_eq!( + parse(&["plotx-cli", "inspect", "sample.jdf", "--json"]), + Ok(expected) + ); +} + +#[test] +fn process_parser_infers_format_and_requires_named_paths() { + assert_eq!( + parse(&[ + "plotx-cli", + "process", + "sample.jdf", + "--scheme", + "routine.plotxproc", + "--output", + "figure.svg", + ]), + Ok(ParseOutcome::Command(Command::Process { + input: "sample.jdf".into(), + scheme: "routine.plotxproc".into(), + output: "figure.svg".into(), + format: OutputFormat(ExportFormat::Svg), + sampling_declaration: None, + })) + ); + assert!(parse(&["plotx-cli", "process", "sample.jdf"]).is_err()); +} + +#[test] +fn craft_parser_accepts_multiple_and_negative_ppm_regions() { + assert_eq!( + parse(&[ + "plotx-cli", + "craft", + "acquisitions", + "--region", + "-0.5:0.2", + "--region", + "6.3:6.5", + "--expected-ratio", + "0.75", + "--output", + "result.json", + ]), + Ok(ParseOutcome::Command(Command::Craft { + input: "acquisitions".into(), + output: "result.json".into(), + regions: vec![ + plotx_processing::craft::CraftRegion::new( + plotx_processing::craft::CraftRegionId(0), + -0.5, + 0.2, + ), + plotx_processing::craft::CraftRegion::new( + plotx_processing::craft::CraftRegionId(1), + 6.3, + 6.5, + ), + ], + expected_ratios: vec![0.75], + })) + ); +} + +#[test] +fn batch_parser_requires_workflow_and_manifest_paths() { + assert_eq!( + parse(&[ + "plotx-cli", + "batch", + "--workflow", + "workflow.json", + "--manifest", + "run.json", + ]), + Ok(ParseOutcome::Command(Command::Batch { + workflow: "workflow.json".into(), + manifest: "run.json".into(), + })) + ); + assert!(parse(&["plotx-cli", "batch", "workflow.json"]).is_err()); +} + +#[test] +fn text_inspection_includes_mass_spectrometry_statistics() { + let report = InspectionReport { + schema: plotx_core::workflow::INSPECTION_SCHEMA, + format: "sciex-wiff".to_owned(), + provenance: plotx_core::workflow::ProvenanceReport { + selected_path: "sample.wiff".into(), + data_path: "sample.wiff".into(), + parameter_paths: Vec::new(), + companion_paths: vec!["sample.wiff.scan".into()], + }, + dimension: plotx_core::workflow::DimensionReport { + count: 3, + shape: vec![2, 42, 1], + }, + domain: "mass_spectrometry".to_owned(), + warnings: Vec::new(), + electrophysiology: None, + afm: None, + mass_spectrometry: Some(plotx_core::workflow::MassSpecReport { + instrument: Some("SCIEX TripleTOF 6600".to_owned()), + stream_count: 2, + ms_scan_count: 42, + chromatograms: vec!["total ion current chromatogram".to_owned()], + }), + xrd: None, + xps: None, + }; + + let output = text_report(&report); + + assert!(output.contains("format: sciex-wiff")); + assert!(output.contains("mass_spec.streams: 2")); + assert!(output.contains("mass_spec.scans: 42")); + assert!(output.contains("mass_spec.chromatograms: total ion current chromatogram")); +} + +#[test] +fn workflow_errors_map_to_stable_exit_categories() { + let status = fail(WorkflowError::FigureUnavailable("NMR 1D")); + assert_eq!(status, Status::Canvas); + assert_eq!(Status::Usage as u8, 2); + assert_eq!(Status::Export as u8, 6); +} diff --git a/crates/cli/tests/batch_cli.rs b/crates/cli/tests/batch_cli.rs index 6e3c7d6a..a7cf7041 100644 --- a/crates/cli/tests/batch_cli.rs +++ b/crates/cli/tests/batch_cli.rs @@ -47,7 +47,7 @@ fn scheme(path: &Path) { "schema_version": 1, "dimension_count": 1, "pipelines": [{"steps": [ - {"kind": {"Phase": {"phase0": 0.0, "phase1": 0.0, "pivot_frac": 0.5, "auto": null}}, "enabled": true, "source": "User"} + {"kind": "Invert", "enabled": true, "source": "User"} ]}], "group_delay_correct": false }"#, @@ -114,7 +114,7 @@ fn batch_cli_exit_stdout_and_saved_manifest_form_one_contract() { assert_eq!(stdout["schema"], "plotx.run-manifest.v1"); assert_eq!(stdout["caller"], "workflow"); assert_eq!(stdout["nodes"].as_array().unwrap().len(), 3); - assert_eq!(stdout["errors"].as_array().unwrap().len(), 1); + assert_eq!(stdout["errors"].as_array().unwrap().len(), 1, "{stdout:#}"); assert_eq!( stdout["nodes"][0]["result"]["targets"][0]["outcome"], "succeeded" diff --git a/crates/cli/tests/nmr_craft.rs b/crates/cli/tests/nmr_craft.rs new file mode 100644 index 00000000..546a6cb8 --- /dev/null +++ b/crates/cli/tests/nmr_craft.rs @@ -0,0 +1,91 @@ +use std::{ + f64::consts::{PI, TAU}, + process::Command, +}; + +#[test] +fn craft_reads_with_nmr_and_preserves_the_independent_tone_frequencies() { + let dir = tempfile::tempdir().unwrap(); + let input = dir.path().join("acquisition"); + std::fs::create_dir(&input).unwrap(); + let parameters = "##TITLE=PlotX synthetic CRAFT acceptance\n##$TD=8192\n##$PARMODE=0\n##$AQ_mod=3\n##$BYTORDA=0\n##$DTYPA=0\n##$SW_h=2000\n##$SFO1=500.005\n##$BF1=500\n##$O1=5000\n##$NUC1=<1H>\n##$GRPDLY=0\n##END=\n"; + std::fs::write(input.join("acqus"), parameters).unwrap(); + let bytes = (0..4096) + .flat_map(|index| { + let time = index as f64 / 2000.0; + let value = [(-75.0, 800_000.0, 0.3, 1.5), (120.0, 400_000.0, -0.2, 2.0)] + .into_iter() + .fold( + nmr::Complex64::new(0., 0.), + |sum, (frequency, amplitude, phase, width)| { + sum + nmr::Complex64::from_polar( + amplitude * (-PI * width * time).exp(), + phase + TAU * frequency * time, + ) + }, + ); + [value.re, value.im] + .into_iter() + .flat_map(|v| (v.round() as i32).to_le_bytes()) + }) + .collect::>(); + std::fs::write(input.join("fid"), bytes).unwrap(); + let output = dir.path().join("result.json"); + let result = Command::new(env!("CARGO_BIN_EXE_plotx-cli")) + .arg("craft") + .arg(&input) + .arg("--output") + .arg(&output) + .output() + .unwrap(); + let report: serde_json::Value = + serde_json::from_slice(&std::fs::read(&output).unwrap()).unwrap(); + assert!( + result.status.success(), + "{report}\n{}", + String::from_utf8_lossy(&result.stderr) + ); + let item = &report["datasets"][0]; + assert_eq!(item["inspection"]["format"], "bruker-raw"); + assert_eq!(item["acquisition"]["observe_frequency_mhz"], 500.005); + assert_eq!( + item["chemical_shift_reference"]["reference_frequency_mhz"], + 500.0 + ); + let components = item["components"].as_array().unwrap(); + for component in components { + let frequency = component["frequency_hz"].as_f64().unwrap(); + let ppm = component["chemical_shift_ppm"].as_f64().unwrap(); + assert!((ppm - (10.0 + frequency / 500.0)).abs() < 1e-12); + } + for expected in [-75.0, 120.0] { + assert!( + components + .iter() + .any( + |component| (component["frequency_hz"].as_f64().unwrap() - expected).abs() + < 0.05 + ), + "{components:?}" + ); + } + + std::fs::write(input.join("acqus"), parameters.replace("##$GRPDLY=0\n", "")).unwrap(); + let result = Command::new(env!("CARGO_BIN_EXE_plotx-cli")) + .arg("craft") + .arg(&input) + .arg("--output") + .arg(&output) + .output() + .unwrap(); + assert!(!result.status.success()); + let report: serde_json::Value = + serde_json::from_slice(&std::fs::read(output).unwrap()).unwrap(); + assert_eq!(report["datasets"][0]["status"], "failed"); + assert!( + report["datasets"][0]["error"] + .as_str() + .unwrap() + .contains("delay evidence") + ); +} diff --git a/crates/cli/tests/nmr_inspect.rs b/crates/cli/tests/nmr_inspect.rs new file mode 100644 index 00000000..8ecf0c23 --- /dev/null +++ b/crates/cli/tests/nmr_inspect.rs @@ -0,0 +1,88 @@ +use std::{path::PathBuf, process::Command}; + +fn fixture(name: &str) -> PathBuf { + PathBuf::from(env!("CARGO_MANIFEST_DIR")) + .join("../io/tests/fixtures/nmr") + .join(name) +} + +fn inspect(name: &str) -> std::process::Output { + Command::new(env!("CARGO_BIN_EXE_plotx-cli")) + .arg("inspect") + .arg(fixture(name)) + .arg("--json") + .output() + .unwrap() +} + +#[test] +fn inspection_uses_raw_preference_and_does_not_run_the_default_recipe() { + let output = inspect("bruker-1d"); + assert!( + output.status.success(), + "{}", + String::from_utf8_lossy(&output.stderr) + ); + let report: serde_json::Value = serde_json::from_slice(&output.stdout).unwrap(); + assert_eq!(report["domain"], "time"); + assert_eq!(report["format"], "bruker-raw"); + assert_eq!(report["dimension"]["shape"], serde_json::json!([2])); +} + +#[test] +fn selected_processed_input_and_sparse_logical_shape_are_preserved() { + for (name, format, shape) in [ + ("bruker-1d/pdata/1/1r", "bruker-processed-1d", vec![4]), + ("bruker-nus", "bruker-raw", vec![4, 2]), + ] { + let output = inspect(name); + assert!( + output.status.success(), + "{}", + String::from_utf8_lossy(&output.stderr) + ); + let report: serde_json::Value = serde_json::from_slice(&output.stdout).unwrap(); + assert_eq!(report["format"], format); + assert_eq!(report["dimension"]["shape"], serde_json::json!(shape)); + } +} + +#[test] +fn jeol_without_delay_evidence_is_inspectable_without_blanket_alerts() { + let output = inspect("jeol-complex.jdf"); + assert!( + output.status.success(), + "{}", + String::from_utf8_lossy(&output.stderr) + ); + let report: serde_json::Value = serde_json::from_slice(&output.stdout).unwrap(); + assert_eq!(report["domain"], "time"); + assert!( + !report["warnings"] + .as_array() + .unwrap() + .iter() + .any(|warning| warning["code"] == "experimental-nmr-semantics") + ); +} + +#[test] +fn ppm_spectrum_is_inspectable_with_its_declared_shape() { + let output = inspect("jcamp-ppm.dx"); + assert!( + output.status.success(), + "{}", + String::from_utf8_lossy(&output.stderr) + ); + let report: serde_json::Value = serde_json::from_slice(&output.stdout).unwrap(); + assert_eq!(report["domain"], "frequency"); + assert_eq!(report["dimension"]["shape"], serde_json::json!([4])); +} + +#[test] +fn invalid_vendor_input_does_not_fall_back_to_the_previous_reader() { + let output = inspect("varian-short-header.fid"); + assert_eq!(output.status.code(), Some(3)); + assert!(output.stdout.is_empty()); + assert!(String::from_utf8_lossy(&output.stderr).contains("header must contain 11 fields")); +} diff --git a/crates/cli/tests/nmr_sampling.rs b/crates/cli/tests/nmr_sampling.rs new file mode 100644 index 00000000..feb08273 --- /dev/null +++ b/crates/cli/tests/nmr_sampling.rs @@ -0,0 +1,115 @@ +use serde_json::{Value, json}; +use std::{ + path::{Path, PathBuf}, + process::Command, +}; + +fn fixture(name: &str) -> PathBuf { + PathBuf::from(env!("CARGO_MANIFEST_DIR")) + .join("../io/tests/fixtures/nmr") + .join(name) +} +fn declaration() -> Value { + json!({"assertion_id": "cli-table", "source": "synthetic CLI input", "grid_shape": [4], "coordinates": [[4], [2]], "index_base": "one", "component_counts": [2]}) +} +fn raw(dir: &Path) { + for name in ["ser", "acqus", "acqu2s"] { + std::fs::copy(fixture(&format!("bruker-nus/{name}")), dir.join(name)).unwrap(); + } +} + +#[test] +fn inspect_and_process_use_explicit_declaration_and_reject_conflicts() { + let dir = tempfile::tempdir().unwrap(); + raw(dir.path()); + let table = dir.path().join("sampling.json"); + std::fs::write(&table, serde_json::to_vec(&declaration()).unwrap()).unwrap(); + let inspect = Command::new(env!("CARGO_BIN_EXE_plotx-cli")) + .arg("inspect") + .arg(dir.path()) + .arg("--sampling-declaration") + .arg(&table) + .arg("--json") + .output() + .unwrap(); + assert!( + inspect.status.success(), + "{}", + String::from_utf8_lossy(&inspect.stderr) + ); + let value: Value = serde_json::from_slice(&inspect.stdout).unwrap(); + assert_eq!(value["dimension"]["shape"], json!([4, 2])); + let scheme = dir.path().join("empty.plotxproc"); + std::fs::write(&scheme, r#"{"schema_version":1,"dimension_count":2,"pipelines":[{"steps":[]},{"steps":[]}],"group_delay_correct":false}"#).unwrap(); + let svg = dir.path().join("spectrum.svg"); + let process = Command::new(env!("CARGO_BIN_EXE_plotx-cli")) + .arg("process") + .arg(dir.path()) + .arg("--sampling-declaration") + .arg(&table) + .arg("--scheme") + .arg(&scheme) + .arg("--output") + .arg(&svg) + .output() + .unwrap(); + assert!( + process.status.success(), + "{}", + String::from_utf8_lossy(&process.stderr) + ); + assert!(svg.exists()); + let mut bad = declaration(); + bad["component_counts"] = json!([1]); + std::fs::write(&table, serde_json::to_vec(&bad).unwrap()).unwrap(); + let rejected = Command::new(env!("CARGO_BIN_EXE_plotx-cli")) + .arg("inspect") + .arg(dir.path()) + .arg("--sampling-declaration") + .arg(&table) + .arg("--json") + .output() + .unwrap(); + assert_eq!(rejected.status.code(), Some(3)); + assert!(rejected.stdout.is_empty()); + assert!(!dir.path().join("nuslist").exists()); +} + +#[test] +fn batch_import_checks_the_same_declaration() { + let dir = tempfile::tempdir().unwrap(); + raw(dir.path()); + let workflow = dir.path().join("workflow.json"); + let manifest = dir.path().join("manifest.json"); + let definition = json!({ + "schema": "plotx.workflow.v1", "inputs": {}, + "nodes": [{"id":"import", "tool_id":"data.import", + "parameters":{"paths":[dir.path()], "sampling_declaration":declaration()}, + "targets":{"kind":"explicit", "ids":[]}}], + "failure_policy":"continue_compatible" + }); + std::fs::write(&workflow, serde_json::to_vec(&definition).unwrap()).unwrap(); + let result = Command::new(env!("CARGO_BIN_EXE_plotx-cli")) + .arg("batch") + .arg("--workflow") + .arg(&workflow) + .arg("--manifest") + .arg(&manifest) + .output() + .unwrap(); + assert!( + result.status.success(), + "{}\n{}", + String::from_utf8_lossy(&result.stderr), + String::from_utf8_lossy(&result.stdout) + ); + let value: Value = serde_json::from_slice(&result.stdout).unwrap(); + assert_eq!( + value["nodes"][0]["result"]["targets"][0]["outcome"], + "succeeded" + ); + assert_eq!( + value, + serde_json::from_slice::(&std::fs::read(manifest).unwrap()).unwrap() + ); +} diff --git a/crates/core/Cargo.toml b/crates/core/Cargo.toml index f732e8c3..e047dd4d 100644 --- a/crates/core/Cargo.toml +++ b/crates/core/Cargo.toml @@ -15,6 +15,7 @@ default = [] datafusion = ["dep:plotx-datafusion"] [dependencies] +nmr.workspace = true plotx-data.workspace = true plotx-datafusion = { workspace = true, optional = true } plotx-io.workspace = true diff --git a/crates/core/src/actions/app_impl/processing.rs b/crates/core/src/actions/app_impl/processing.rs index 753ba5af..d6b5eaf7 100644 --- a/crates/core/src/actions/app_impl/processing.rs +++ b/crates/core/src/actions/app_impl/processing.rs @@ -44,11 +44,14 @@ impl PlotxApp { DatasetProcessingState::Nmr2D { params, preset, + nus_request, group_delay_correct, }, ) => { n.params = params.clone(); n.preset = *preset; + n.base_stale |= n.nus_request != *nus_request; + n.nus_request = *nus_request; n.group_delay_correct = *group_delay_correct; } (Dataset::Xrd(data), DatasetProcessingState::Xrd(processing)) => { @@ -94,13 +97,17 @@ impl PlotxApp { DatasetProcessingState::Nmr2D { params, preset, + nus_request, group_delay_correct, }, ) = (self.doc.datasets.get_mut(dataset), state) { - let force_full = current.group_delay_correct != *group_delay_correct; + let force_full = current.group_delay_correct != *group_delay_correct + || current.nus_request != *nus_request; + current.base_stale |= force_full; current.params = params.clone(); current.preset = *preset; + current.nus_request = *nus_request; current.group_delay_correct = *group_delay_correct; self.schedule_2d_processing(dataset, force_full); return Ok(()); @@ -393,18 +400,11 @@ pub(super) fn validate_processing_state( ) -> Result<(), String> { match (dataset, state) { (Dataset::Nmr(dataset), DatasetProcessingState::Nmr { pipeline, .. }) => pipeline - .output_domain(dataset.data.domain) + .output_domain(dataset.input_domain()) .map(|_| ()) .map_err(|error| format!("Cannot apply invalid direct processing pipeline: {error}")), (Dataset::Nmr2D(dataset), DatasetProcessingState::Nmr2D { params, .. }) => { - params - .f2 - .output_domain(dataset.data.domain) - .map_err(|error| format!("Cannot apply invalid F2 processing pipeline: {error}"))?; - params - .f1 - .output_domain(dataset.data.domain) - .map_err(|error| format!("Cannot apply invalid F1 processing pipeline: {error}"))?; + plotx_processing::nmr_execution::validate_2d_domains(&dataset.data, params)?; Ok(()) } (Dataset::Xrd(_), DatasetProcessingState::Xrd(processing)) => { diff --git a/crates/core/src/actions/mod.rs b/crates/core/src/actions/mod.rs index 14c04c84..4bfe77a2 100644 --- a/crates/core/src/actions/mod.rs +++ b/crates/core/src/actions/mod.rs @@ -41,6 +41,7 @@ pub enum DatasetProcessingState { params: Params2D, preset: Preset2D, group_delay_correct: bool, + nus_request: Option, }, /// A table has no reversible processing recipe; its curve fits are edited /// through their own actions. diff --git a/crates/core/src/actions/processing_state.rs b/crates/core/src/actions/processing_state.rs index 9e86eeed..1712c3f1 100644 --- a/crates/core/src/actions/processing_state.rs +++ b/crates/core/src/actions/processing_state.rs @@ -47,6 +47,7 @@ impl DatasetProcessingState { Dataset::Nmr2D(n) => Self::Nmr2D { params: n.params.clone(), preset: n.preset, + nus_request: n.nus_request, group_delay_correct: n.group_delay_correct, }, Dataset::Table(_) => Self::Table, @@ -102,7 +103,7 @@ impl DatasetProcessingState { group_delay_correct, }, ) => { - pipeline.output_domain(n.data.domain).map_err(|error| { + pipeline.output_domain(n.input_domain()).map_err(|error| { ProcessingStateError::InvalidPipeline { axis: "direct", details: error.to_string(), @@ -114,17 +115,26 @@ impl DatasetProcessingState { *group_delay_correct, n.group_delay_correct, ); - n.pipeline = pipeline.clone(); - n.repair_step_allocator(); - n.group_delay_correct = *group_delay_correct; + let mut next = (**n).clone(); + next.pipeline = pipeline.clone(); + next.repair_step_allocator(); + next.group_delay_correct = *group_delay_correct; + let result = if full { + next.retransform() + } else { + next.rebuild() + }; + result.map_err(|details| ProcessingStateError::InvalidPipeline { + axis: "direct", + details, + })?; + next.recompute_integrals(); + **n = next; let rebuild = if full { - n.retransform(); ProcessingRebuild::Retransformed } else { - n.rebuild(); ProcessingRebuild::Rebuilt }; - n.recompute_integrals(); Ok(rebuild) } ( @@ -132,34 +142,40 @@ impl DatasetProcessingState { Self::Nmr2D { params, preset, + nus_request, group_delay_correct, }, ) => { - params.f2.output_domain(n.data.domain).map_err(|error| { - ProcessingStateError::InvalidPipeline { - axis: "F2", - details: error.to_string(), - } - })?; - params.f1.output_domain(n.data.domain).map_err(|error| { - ProcessingStateError::InvalidPipeline { - axis: "F1", - details: error.to_string(), - } + plotx_processing::nmr_execution::validate_2d_domains(&n.data, params).map_err( + |details| ProcessingStateError::InvalidPipeline { + axis: "2D", + details, + }, + )?; + let full = plotx_processing::needs_retransform_2d(params, &n.params) + || *group_delay_correct != n.group_delay_correct + || *nus_request != n.nus_request; + let mut next = (**n).clone(); + next.params = params.clone(); + next.repair_step_allocator(); + next.preset = *preset; + next.nus_request = *nus_request; + next.group_delay_correct = *group_delay_correct; + let result = if full { + next.retransform() + } else { + next.rebuild() + }; + result.map_err(|details| ProcessingStateError::InvalidPipeline { + axis: "2D", + details, })?; - let full = plotx_processing::needs_retransform_2d(params, &n.params); - let full = full || *group_delay_correct != n.group_delay_correct; - n.params = params.clone(); - n.repair_step_allocator(); - n.preset = *preset; - n.group_delay_correct = *group_delay_correct; - if full { - n.retransform(); - Ok(ProcessingRebuild::Retransformed) + **n = next; + Ok(if full { + ProcessingRebuild::Retransformed } else { - n.rebuild(); - Ok(ProcessingRebuild::Rebuilt) - } + ProcessingRebuild::Rebuilt + }) } (Dataset::Table(_), Self::Table) => Ok(ProcessingRebuild::Unchanged), (Dataset::Electrophysiology(data), Self::Electrophysiology(processing)) => { diff --git a/crates/core/src/actions/tests/align.rs b/crates/core/src/actions/tests/align.rs index f5031abf..5b4ce836 100644 --- a/crates/core/src/actions/tests/align.rs +++ b/crates/core/src/actions/tests/align.rs @@ -32,11 +32,9 @@ fn app_with(peaks: &[f64]) -> PlotxApp { let mut app = PlotxApp::new(); app.doc.save_include_view_snapshots = false; for &p in peaks { - app.doc - .datasets - .push(Dataset::Nmr(Box::new(NmrDataset::load(synthetic_at( - p, "1H", - ))))); + app.doc.datasets.push(Dataset::Nmr(Box::new( + NmrDataset::load(synthetic_at(p, "1H")).unwrap(), + ))); } let all: Vec = (0..peaks.len()).collect(); app.focus_datasets(&all, Some(0)); @@ -110,16 +108,12 @@ fn window_without_peak_skips_every_spectrum() { #[test] fn other_nuclei_and_non_1d_datasets_are_skipped_with_reasons() { let mut app = app_with(&[2.0, 2.5]); - app.doc - .datasets - .push(Dataset::Nmr(Box::new(NmrDataset::load(synthetic_at( - 2.2, "13C", - ))))); - app.doc - .datasets - .push(Dataset::Nmr2D(Box::new(crate::state::Nmr2DDataset::load( - synthetic_2d(), - )))); + app.doc.datasets.push(Dataset::Nmr(Box::new( + NmrDataset::load(synthetic_at(2.2, "13C")).unwrap(), + ))); + app.doc.datasets.push(Dataset::Nmr2D(Box::new( + crate::nmr_test_support::load_2d(synthetic_2d()).unwrap(), + ))); app.focus_datasets(&[0, 1, 2, 3], Some(0)); let carbon_before = peak_ppm(&app, 2); diff --git a/crates/core/src/actions/tests/arithmetic.rs b/crates/core/src/actions/tests/arithmetic.rs index 23935839..bee4b6a2 100644 --- a/crates/core/src/actions/tests/arithmetic.rs +++ b/crates/core/src/actions/tests/arithmetic.rs @@ -4,9 +4,9 @@ use plotx_processing::arithmetic::SpectrumBinaryOp; fn two_spectrum_app() -> PlotxApp { let mut app = sample_app(); - app.doc - .datasets - .push(Dataset::Nmr(Box::new(NmrDataset::load(synthetic_1d())))); + app.doc.datasets.push(Dataset::Nmr(Box::new( + NmrDataset::load(synthetic_1d()).unwrap(), + ))); app } @@ -86,7 +86,7 @@ fn result_dataset_replays_exactly_after_retransform() { app.combine_spectra_datasets(0, 1, SpectrumBinaryOp::Subtract, 0.5); let mut ds = app.doc.datasets[2].as_nmr().unwrap().clone(); let shown = ds.spectrum().unwrap().clone(); - ds.retransform(); + ds.retransform().unwrap(); assert_eq!(ds.spectrum().unwrap().values.len(), shown.values.len()); for (a, b) in ds.spectrum().unwrap().values.iter().zip(&shown.values) { assert!((a - b).norm() < 1e-9); @@ -103,14 +103,18 @@ fn nucleus_mismatch_is_rejected_without_side_effects() { other.nucleus = "13C".to_owned(); app.doc .datasets - .push(Dataset::Nmr(Box::new(NmrDataset::load(other)))); + .push(Dataset::Nmr(Box::new(NmrDataset::load(other).unwrap()))); let canvases_before = app.doc.canvases.len(); app.combine_spectra_datasets(0, 1, SpectrumBinaryOp::Subtract, 1.0); assert_eq!(app.doc.datasets.len(), 2); assert_eq!(app.doc.canvases.len(), canvases_before); - assert!(app.session.status.contains("Nuclei differ")); + assert!( + app.session.status.contains("incompatible"), + "{}", + app.session.status + ); assert!(app.spectrum_arithmetic_compat(0, 1).is_err()); } @@ -135,17 +139,16 @@ fn single_point_operands_combine_without_panicking() { domain: plotx_io::Domain::Frequency, values: vec![num_complex::Complex64::new(re, 0.0)], nucleus: "1H".to_owned(), - observe_freq_mhz: 400.0, + observe_freq_mhz: Some(400.0), + reference_freq_mhz: Some(400.0), + unit: nmr::axis::AxisUnit::Ppm, position: None, position_domain: plotx_io::Domain::Frequency, }; for (i, s) in [point(1.0, 2.0), point(3.0, 5.0)].into_iter().enumerate() { - app.doc - .datasets - .push(Dataset::Nmr(Box::new(NmrDataset::from_slice( - s, - format!("p{i}"), - )))); + app.doc.datasets.push(Dataset::Nmr(Box::new( + NmrDataset::from_slice(s, format!("p{i}")).unwrap(), + ))); } app.combine_spectra_datasets(0, 1, SpectrumBinaryOp::Add, 1.0); diff --git a/crates/core/src/actions/tests/linefit.rs b/crates/core/src/actions/tests/linefit.rs index 6720f723..10eb1f43 100644 --- a/crates/core/src/actions/tests/linefit.rs +++ b/crates/core/src/actions/tests/linefit.rs @@ -26,11 +26,15 @@ fn two_lorentzian_dataset(name: &str) -> Dataset { domain: plotx_io::Domain::Frequency, values, nucleus: "1H".to_owned(), - observe_freq_mhz: 400.0, + observe_freq_mhz: Some(400.0), + reference_freq_mhz: Some(400.0), + unit: nmr::axis::AxisUnit::Ppm, position: None, position_domain: plotx_io::Domain::Frequency, }; - Dataset::Nmr(Box::new(NmrDataset::from_slice(slice, name.to_owned()))) + Dataset::Nmr(Box::new( + NmrDataset::from_slice(slice, name.to_owned()).unwrap(), + )) } fn two_lorentzian_app() -> PlotxApp { diff --git a/crates/core/src/actions/tests/mod.rs b/crates/core/src/actions/tests/mod.rs index f534d66c..83204eb3 100644 --- a/crates/core/src/actions/tests/mod.rs +++ b/crates/core/src/actions/tests/mod.rs @@ -54,9 +54,9 @@ fn synthetic_1d() -> NmrData { pub(super) fn sample_app() -> PlotxApp { let mut app = PlotxApp::new(); app.doc.save_include_view_snapshots = false; - app.doc - .datasets - .push(Dataset::Nmr(Box::new(NmrDataset::load(synthetic_1d())))); + app.doc.datasets.push(Dataset::Nmr(Box::new( + NmrDataset::load(synthetic_1d()).unwrap(), + ))); push_canvas(&mut app, 0, "sample canvas", [120.0, 80.0]); app.focus_single(0); app.session.active_canvas = Some(0); @@ -115,7 +115,7 @@ fn data_tool_target_requires_data_verb_and_selected_plot() { fn insert_dataset_new_canvas_does_not_select_object() { let mut app = PlotxApp::new(); app.doc.save_include_view_snapshots = false; - let dataset = Dataset::Nmr(Box::new(NmrDataset::load(synthetic_1d()))); + let dataset = Dataset::Nmr(Box::new(NmrDataset::load(synthetic_1d()).unwrap())); app.execute_action(Action::insert_dataset_with_default_canvas( &app, @@ -141,7 +141,7 @@ fn insert_dataset_existing_canvas_does_not_select_inserted_object() { app.doc.canvases[0].selected_object = None; let inserted_id = app.doc.canvases[0].next_object_id; let dataset_index = app.doc.datasets.len(); - let dataset = Dataset::Nmr(Box::new(NmrDataset::load(synthetic_1d()))); + let dataset = Dataset::Nmr(Box::new(NmrDataset::load(synthetic_1d()).unwrap())); app.execute_action(Action::InsertDatasetWithCanvas { dataset_index, @@ -761,7 +761,12 @@ pub(super) fn synthetic_2d() -> plotx_io::NmrData2D { group_delay: 0.0, }; NmrData2D { - data: vec![Complex64::new(0.0, 0.0); rows * cols], + data: (0..rows * cols) + .map(|i| { + Complex64::from_polar((-0.05 * (i % cols) as f64).exp(), 0.8 * (i % cols) as f64) + * (0.3 * (i / cols) as f64).cos() + }) + .collect(), rows, cols, domain: Domain::Time, diff --git a/crates/core/src/actions/tests/more.rs b/crates/core/src/actions/tests/more.rs index 21fc1463..5cfb2de7 100644 --- a/crates/core/src/actions/tests/more.rs +++ b/crates/core/src/actions/tests/more.rs @@ -5,9 +5,9 @@ use super::*; #[test] fn stacked_binding_builds_distinctly_coloured_series_with_legend() { let mut app = sample_app(); - app.doc - .datasets - .push(Dataset::Nmr(Box::new(NmrDataset::load(synthetic_1d())))); + app.doc.datasets.push(Dataset::Nmr(Box::new( + NmrDataset::load(synthetic_1d()).unwrap(), + ))); let object = app.doc.canvases[0].objects[0].id; let mut second = crate::state::SeriesBinding::from_dataset(&app.doc.datasets[1]).unwrap(); second.set_primary_color(plotx_figure::Color::rgb(0x8a, 0x1c, 0x1c)); @@ -133,14 +133,12 @@ fn set_chart_type_switches_table_to_categorical_bars_and_undoes() { #[test] fn stack_candidates_reject_incompatible_datasets() { let mut app = sample_app(); - app.doc - .datasets - .push(Dataset::Nmr(Box::new(NmrDataset::load(synthetic_1d())))); - app.doc - .datasets - .push(Dataset::Nmr2D(Box::new(crate::state::Nmr2DDataset::load( - synthetic_2d(), - )))); + app.doc.datasets.push(Dataset::Nmr(Box::new( + NmrDataset::load(synthetic_1d()).unwrap(), + ))); + app.doc.datasets.push(Dataset::Nmr2D(Box::new( + crate::nmr_test_support::load_2d(synthetic_2d()).unwrap(), + ))); let binding = crate::state::DataBinding::single(&app.doc.datasets[0]); let candidates = app.stack_candidates(&binding); @@ -162,11 +160,9 @@ fn axis_projections_attach_and_project_survive_undo() { // dataset 0 = 1D (from sample_app), dataset 1 = a true-2D contour on canvas 1. let mut app = sample_app(); - app.doc - .datasets - .push(Dataset::Nmr2D(Box::new(crate::state::Nmr2DDataset::load( - synthetic_2d(), - )))); + app.doc.datasets.push(Dataset::Nmr2D(Box::new( + crate::nmr_test_support::load_2d(synthetic_2d()).unwrap(), + ))); assert!(app.doc.datasets[1].as_nmr2d().unwrap().is_true_2d()); push_canvas(&mut app, 1, "2d", [120.0, 80.0]); let ci = 1; @@ -224,7 +220,7 @@ fn axis_projections_attach_and_project_survive_undo() { fn auto_phase_pivot_reports_the_peak_ppm() { use crate::state::PhaseAxis; - let dataset = Dataset::Nmr(Box::new(NmrDataset::load(synthetic_1d()))); + let dataset = Dataset::Nmr(Box::new(NmrDataset::load(synthetic_1d()).unwrap())); let pivot = dataset.pivot_ppm(PhaseAxis::Direct).unwrap(); assert!( (pivot - 2.0).abs() < 0.2, @@ -238,7 +234,7 @@ fn auto_phase_pivot_reports_the_peak_ppm() { fn auto_phase_pivot_reports_the_peak_ppm_2d() { use crate::state::PhaseAxis; - let dataset = crate::state::Nmr2DDataset::load(synthetic_2d()); + let dataset = crate::state::Nmr2DDataset::load(synthetic_2d()).unwrap(); let s = match &dataset.base { plotx_processing::Processed2D::Ft(s) => s, plotx_processing::Processed2D::Stack(_) => unreachable!("synthetic_2d is true-2D"), @@ -273,11 +269,9 @@ fn manual_2d_phase_inherits_the_automatic_solution() { use crate::state::PhaseAxis; let mut app = PlotxApp::new(); - app.doc - .datasets - .push(Dataset::Nmr2D(Box::new(crate::state::Nmr2DDataset::load( - synthetic_2d(), - )))); + app.doc.datasets.push(Dataset::Nmr2D(Box::new( + crate::state::Nmr2DDataset::load(synthetic_2d()).unwrap(), + ))); let expected = app.doc.datasets[0] .automatic_phase_params(PhaseAxis::F2) .unwrap(); diff --git a/crates/core/src/actions/tests/multiplet.rs b/crates/core/src/actions/tests/multiplet.rs index a3e3fdb7..4aeb1a88 100644 --- a/crates/core/src/actions/tests/multiplet.rs +++ b/crates/core/src/actions/tests/multiplet.rs @@ -11,11 +11,13 @@ fn doublet_marked_app() -> PlotxApp { domain: plotx_io::Domain::Frequency, values, nucleus: "1H".to_owned(), - observe_freq_mhz: 400.0, + observe_freq_mhz: Some(400.0), + reference_freq_mhz: Some(400.0), + unit: nmr::axis::AxisUnit::Ppm, position: None, position_domain: plotx_io::Domain::Frequency, }; - let mut nmr = NmrDataset::from_slice(slice, "doublet".to_owned()); + let mut nmr = NmrDataset::from_slice(slice, "doublet".to_owned()).unwrap(); for (id, x) in [(0u64, 2.0), (1u64, 2.0 + 7.0 / 400.0)] { nmr.peaks.marks.push(PeakMark { id, diff --git a/crates/core/src/actions/tests/scheme_apply.rs b/crates/core/src/actions/tests/scheme_apply.rs index 33a31ec2..d86b6a21 100644 --- a/crates/core/src/actions/tests/scheme_apply.rs +++ b/crates/core/src/actions/tests/scheme_apply.rs @@ -25,9 +25,9 @@ fn group_delay(app: &PlotxApp, di: usize) -> bool { fn batch_template_apply_filters_incompatible_targets_and_undoes_as_one_step() { let mut app = PlotxApp::new(); for _ in 0..2 { - app.doc - .datasets - .push(Dataset::Nmr(Box::new(NmrDataset::load(synthetic_1d())))); + app.doc.datasets.push(Dataset::Nmr(Box::new( + NmrDataset::load(synthetic_1d()).unwrap(), + ))); } if let Dataset::Nmr(n) = &mut app.doc.datasets[0] { n.group_delay_correct = false; @@ -93,9 +93,9 @@ fn a_hand_written_scheme_without_step_ids_loads_and_applies() { serde_json::from_str(json).expect("a recipe may omit step identities"); let mut app = PlotxApp::new(); - app.doc - .datasets - .push(Dataset::Nmr(Box::new(NmrDataset::load(synthetic_1d())))); + app.doc.datasets.push(Dataset::Nmr(Box::new( + NmrDataset::load(synthetic_1d()).unwrap(), + ))); let plan = plan_scheme_application(&scheme, &app.doc.datasets, &[0]); assert_eq!(plan.compatible_count(), 1); @@ -119,9 +119,9 @@ fn a_hand_written_scheme_without_step_ids_loads_and_applies() { #[test] fn a_saved_scheme_omits_step_identities() { let mut app = PlotxApp::new(); - app.doc - .datasets - .push(Dataset::Nmr(Box::new(NmrDataset::load(synthetic_1d())))); + app.doc.datasets.push(Dataset::Nmr(Box::new( + NmrDataset::load(synthetic_1d()).unwrap(), + ))); let path = temp_scheme("no-step-ids"); save_scheme(&path, &app.doc.datasets[0]).unwrap(); let written = std::fs::read_to_string(&path).unwrap(); diff --git a/crates/core/src/actions/tests/stable_identity.rs b/crates/core/src/actions/tests/stable_identity.rs index be96a4b9..e79885f9 100644 --- a/crates/core/src/actions/tests/stable_identity.rs +++ b/crates/core/src/actions/tests/stable_identity.rs @@ -9,7 +9,7 @@ use plotx_processing::{ProcessingStep, StepKind, StepSource}; #[test] fn dataset_delete_undo_restores_identity_and_persistent_references() { let mut app = sample_app(); - let mut inserted = Dataset::Nmr(Box::new(NmrDataset::load(synthetic_1d()))); + let mut inserted = Dataset::Nmr(Box::new(NmrDataset::load(synthetic_1d()).unwrap())); let inserted_id = DatasetId::new(); inserted.set_resource_id(inserted_id); let action = Action::insert_dataset_with_default_canvas( @@ -71,7 +71,7 @@ fn canvas_dataset_ids_follow_first_appearance_and_page_indices_follow_document_o ]; app.doc.datasets[0].set_resource_id(ids[0]); for id in &ids[1..] { - let mut dataset = Dataset::Nmr(Box::new(NmrDataset::load(synthetic_1d()))); + let mut dataset = Dataset::Nmr(Box::new(NmrDataset::load(synthetic_1d()).unwrap())); dataset.set_resource_id(*id); app.doc.datasets.push(dataset); } @@ -121,7 +121,7 @@ fn syncing_integral_curves_ignores_a_stale_dataset_index() { #[test] fn series_reorder_preserves_ids_and_only_changes_order() { let mut app = sample_app(); - let second = Dataset::Nmr(Box::new(NmrDataset::load(synthetic_1d()))); + let second = Dataset::Nmr(Box::new(NmrDataset::load(synthetic_1d()).unwrap())); let second_id = second.resource_id(); app.doc.datasets.push(second); let plot = app.doc.canvases[0].objects[0].plot_mut().unwrap(); @@ -144,7 +144,7 @@ fn series_reorder_preserves_ids_and_only_changes_order() { #[test] fn step_and_series_allocators_do_not_rollback_with_undo() { let mut app = sample_app(); - let second = Dataset::Nmr(Box::new(NmrDataset::load(synthetic_1d()))); + let second = Dataset::Nmr(Box::new(NmrDataset::load(synthetic_1d()).unwrap())); let second_id = second.resource_id(); app.doc.datasets.push(second); @@ -254,9 +254,9 @@ fn step_and_series_allocators_do_not_rollback_with_undo() { #[test] fn an_expanded_step_does_not_leak_onto_another_dataset_with_the_same_id() { let mut app = sample_app(); - app.doc - .datasets - .push(Dataset::Nmr(Box::new(NmrDataset::load(synthetic_1d())))); + app.doc.datasets.push(Dataset::Nmr(Box::new( + NmrDataset::load(synthetic_1d()).unwrap(), + ))); let phase_id = |app: &crate::state::PlotxApp, index: usize| { app.doc.datasets[index] diff --git a/crates/core/src/actions/tests/stack.rs b/crates/core/src/actions/tests/stack.rs index e15f8736..409cd806 100644 --- a/crates/core/src/actions/tests/stack.rs +++ b/crates/core/src/actions/tests/stack.rs @@ -7,9 +7,9 @@ fn stacked_figure_is_domain_generic_with_offset_scale_and_hide() { // NMR 1D and Table domains exercise the same generic stacking path. let mut nmr = sample_app(); - nmr.doc - .datasets - .push(Dataset::Nmr(Box::new(NmrDataset::load(synthetic_1d())))); + nmr.doc.datasets.push(Dataset::Nmr(Box::new( + NmrDataset::load(synthetic_1d()).unwrap(), + ))); let (mut table, _) = table_app_with_sigma(vec![0.1, 0.1, 0.1]); let second = second_table_with_sigma(vec![0.2, 0.2, 0.2]); table.doc.datasets.push(Dataset::Table(Box::new(second))); @@ -83,16 +83,12 @@ fn field_overlay_stacks_two_2d_contours_in_distinct_colors() { 0.0, ); } - app.doc - .datasets - .push(Dataset::Nmr2D(Box::new(crate::state::Nmr2DDataset::load( - signed_grid.clone(), - )))); - app.doc - .datasets - .push(Dataset::Nmr2D(Box::new(crate::state::Nmr2DDataset::load( - signed_grid, - )))); + app.doc.datasets.push(Dataset::Nmr2D(Box::new( + crate::nmr_test_support::load_2d(signed_grid.clone()).unwrap(), + ))); + app.doc.datasets.push(Dataset::Nmr2D(Box::new( + crate::nmr_test_support::load_2d(signed_grid).unwrap(), + ))); let (a, b) = (app.doc.datasets.len() - 2, app.doc.datasets.len() - 1); let mut binding = DataBinding { series: vec![ @@ -145,9 +141,9 @@ fn field_overlay_stacks_two_2d_contours_in_distinct_colors() { #[test] fn plain_then_ctrl_click_selects_two_datasets_for_stacking() { let mut app = sample_app(); - app.doc - .datasets - .push(Dataset::Nmr(Box::new(NmrDataset::load(synthetic_1d())))); + app.doc.datasets.push(Dataset::Nmr(Box::new( + NmrDataset::load(synthetic_1d()).unwrap(), + ))); // The first click must count toward the stack: plain-click A then Ctrl-click // B yields a two-item selection (no "Ctrl the first item" trap). @@ -183,9 +179,9 @@ fn plain_then_ctrl_click_selects_two_datasets_for_stacking() { #[test] fn ctrl_clicking_two_identical_1d_datasets_enables_stack() { let mut app = sample_app(); - app.doc - .datasets - .push(Dataset::Nmr(Box::new(NmrDataset::load(synthetic_1d())))); + app.doc.datasets.push(Dataset::Nmr(Box::new( + NmrDataset::load(synthetic_1d()).unwrap(), + ))); app.clear_selection(); app.toggle_selection(0, true); @@ -203,9 +199,9 @@ fn ctrl_clicking_two_identical_1d_datasets_enables_stack() { #[test] fn selecting_canvas_populates_data_selection_with_its_datasets() { let mut app = sample_app(); - app.doc - .datasets - .push(Dataset::Nmr(Box::new(NmrDataset::load(synthetic_1d())))); + app.doc.datasets.push(Dataset::Nmr(Box::new( + NmrDataset::load(synthetic_1d()).unwrap(), + ))); let object = app.doc.canvases[0].objects[0].id; let binding = crate::state::DataBinding { series: vec![ @@ -238,9 +234,9 @@ fn selecting_canvas_populates_data_selection_with_its_datasets() { fn plot_object_reports_every_bound_dataset_for_selection_mirroring() { use crate::state::{DataBinding, SeriesBinding}; let mut app = sample_app(); - app.doc - .datasets - .push(Dataset::Nmr(Box::new(NmrDataset::load(synthetic_1d())))); + app.doc.datasets.push(Dataset::Nmr(Box::new( + NmrDataset::load(synthetic_1d()).unwrap(), + ))); let object = app.doc.canvases[0].objects[0].id; let binding = DataBinding { series: vec![ @@ -309,9 +305,9 @@ fn shear_sign_flips_the_pseudo_3d_lean_direction() { fn multi_selecting_pages_in_the_workspace_populates_data_for_stacking() { use crate::state::FrameRef; let mut app = sample_app(); - app.doc - .datasets - .push(Dataset::Nmr(Box::new(NmrDataset::load(synthetic_1d())))); + app.doc.datasets.push(Dataset::Nmr(Box::new( + NmrDataset::load(synthetic_1d()).unwrap(), + ))); push_canvas(&mut app, 1, "second canvas", [120.0, 80.0]); app.session.ui.frame_selection = @@ -331,9 +327,9 @@ fn multi_selecting_pages_in_the_workspace_populates_data_for_stacking() { fn selecting_one_page_pulls_active_into_the_set_so_no_phantom_highlight() { use crate::state::FrameRef; let mut app = sample_app(); - app.doc - .datasets - .push(Dataset::Nmr(Box::new(NmrDataset::load(synthetic_1d())))); + app.doc.datasets.push(Dataset::Nmr(Box::new( + NmrDataset::load(synthetic_1d()).unwrap(), + ))); push_canvas(&mut app, 1, "second canvas", [120.0, 80.0]); // A stale active dataset (0) points outside the frame about to be selected. @@ -352,9 +348,9 @@ fn selecting_one_page_pulls_active_into_the_set_so_no_phantom_highlight() { #[test] fn every_selection_mutator_keeps_active_inside_the_set() { let mut app = sample_app(); - app.doc - .datasets - .push(Dataset::Nmr(Box::new(NmrDataset::load(synthetic_1d())))); + app.doc.datasets.push(Dataset::Nmr(Box::new( + NmrDataset::load(synthetic_1d()).unwrap(), + ))); let holds = |a: &PlotxApp| { a.active_dataset() diff --git a/crates/core/src/actions/tests/symmetry.rs b/crates/core/src/actions/tests/symmetry.rs index 6d81751f..b05519ad 100644 --- a/crates/core/src/actions/tests/symmetry.rs +++ b/crates/core/src/actions/tests/symmetry.rs @@ -1,5 +1,5 @@ use super::*; -use crate::state::{Nmr2DDataset, Peak2DOrigin, Peak2DPoint, Peak2DReview, Peak2DSet}; +use crate::state::{Peak2DOrigin, Peak2DPoint, Peak2DReview, Peak2DSet}; #[test] fn cross_peak_pair_is_one_undoable_edit() { @@ -8,9 +8,9 @@ fn cross_peak_pair_is_one_undoable_edit() { data.experiment = Some("cosy".to_owned()); let mut app = PlotxApp::new(); - app.doc - .datasets - .push(Dataset::Nmr2D(Box::new(Nmr2DDataset::load(data)))); + app.doc.datasets.push(Dataset::Nmr2D(Box::new( + crate::nmr_test_support::load_2d(data).unwrap(), + ))); let dataset_id = app.doc.datasets[0].resource_id(); let before = Peak2DSet::default(); let mut after = before.clone(); diff --git a/crates/core/src/automation/properties_tests_inbound_value.rs b/crates/core/src/automation/properties_tests_inbound_value.rs index 3d891019..6821ecd8 100644 --- a/crates/core/src/automation/properties_tests_inbound_value.rs +++ b/crates/core/src/automation/properties_tests_inbound_value.rs @@ -214,7 +214,7 @@ fn smoothing_app() -> (PlotxApp, String) { source: "automation smoothing".to_owned(), group_delay: 0.0, }; - let mut dataset = NmrDataset::load(data); + let mut dataset = NmrDataset::load(data).unwrap(); let id = dataset.allocate_step_id(); dataset .pipeline diff --git a/crates/core/src/automation/properties_tests_outbound.rs b/crates/core/src/automation/properties_tests_outbound.rs index 0bc46afa..e23eb9b6 100644 --- a/crates/core/src/automation/properties_tests_outbound.rs +++ b/crates/core/src/automation/properties_tests_outbound.rs @@ -202,9 +202,8 @@ fn document_property_tools_address_the_document_root() { #[test] fn dataset_property_tools_expand_processing_steps_and_report_non_apodization_skips() { let mut app = PlotxApp::new(); - app.doc - .datasets - .push(Dataset::Nmr(Box::new(NmrDataset::load(NmrData { + app.doc.datasets.push(Dataset::Nmr(Box::new( + NmrDataset::load(NmrData { points: (0..32) .map(|value| num_complex::Complex64::new(f64::from(value), 0.0)) .collect(), @@ -215,7 +214,9 @@ fn dataset_property_tools_expand_processing_steps_and_report_non_apodization_ski nucleus: "1H".to_owned(), source: "automation apodization".to_owned(), group_delay: 0.0, - })))); + }) + .unwrap(), + ))); let dataset = app.doc.datasets[0].resource_id().to_string(); let request = request( &app, @@ -268,9 +269,8 @@ fn dataset_property_tools_expand_processing_steps_and_report_non_apodization_ski #[test] fn inspect_reports_the_actionable_reason_for_a_disabled_phase_parameter() { let mut app = PlotxApp::new(); - app.doc - .datasets - .push(Dataset::Nmr(Box::new(NmrDataset::load(NmrData { + app.doc.datasets.push(Dataset::Nmr(Box::new( + NmrDataset::load(NmrData { points: (0..32) .map(|value| num_complex::Complex64::new(f64::from(value), 0.25)) .collect(), @@ -281,7 +281,9 @@ fn inspect_reports_the_actionable_reason_for_a_disabled_phase_parameter() { nucleus: "1H".to_owned(), source: "automation phase availability".to_owned(), group_delay: 0.0, - })))); + }) + .unwrap(), + ))); let dataset = app.doc.datasets[0].resource_id().to_string(); let inspect = request( &app, @@ -301,9 +303,8 @@ fn inspect_reports_the_actionable_reason_for_a_disabled_phase_parameter() { #[test] fn degree_schema_dto_keeps_display_log_and_unit_consistent() { let mut app = PlotxApp::new(); - app.doc - .datasets - .push(Dataset::Nmr(Box::new(NmrDataset::load(NmrData { + app.doc.datasets.push(Dataset::Nmr(Box::new( + NmrDataset::load(NmrData { points: (0..32) .map(|value| num_complex::Complex64::new(f64::from(value), 0.25)) .collect(), @@ -314,7 +315,9 @@ fn degree_schema_dto_keeps_display_log_and_unit_consistent() { nucleus: "1H".to_owned(), source: "automation phase display".to_owned(), group_delay: 0.0, - })))); + }) + .unwrap(), + ))); let dataset = app.doc.datasets[0].resource_id().to_string(); let inspect = request( &app, diff --git a/crates/core/src/automation/properties_tests_rejections.rs b/crates/core/src/automation/properties_tests_rejections.rs index 87aae9a3..864c503d 100644 --- a/crates/core/src/automation/properties_tests_rejections.rs +++ b/crates/core/src/automation/properties_tests_rejections.rs @@ -158,9 +158,8 @@ fn whole_encoding_reset_is_not_a_tool() { #[test] fn the_catalog_capability_follows_addressable_components_not_the_dataset_kind() { let (mut app, _) = contour_app(); - app.doc - .datasets - .push(Dataset::Nmr(Box::new(NmrDataset::load(NmrData { + app.doc.datasets.push(Dataset::Nmr(Box::new( + NmrDataset::load(NmrData { points: (0..8) .map(|value| num_complex::Complex64::new(f64::from(value), 0.0)) .collect(), @@ -171,7 +170,9 @@ fn the_catalog_capability_follows_addressable_components_not_the_dataset_kind() nucleus: "1H".to_owned(), source: "capability gate".to_owned(), group_delay: 0.0, - })))); + }) + .unwrap(), + ))); let catalog = CapabilityId::new(CAP_PROPERTY_CATALOG); let provider = ProjectResourceProvider::new(&app); let descriptors = provider.descriptors(); diff --git a/crates/core/src/automation/registry.rs b/crates/core/src/automation/registry.rs index ef7ddea6..c337d9a5 100644 --- a/crates/core/src/automation/registry.rs +++ b/crates/core/src/automation/registry.rs @@ -122,6 +122,8 @@ pub(super) struct SchemeParams { #[serde(deny_unknown_fields)] pub(super) struct ImportParams { pub paths: Vec, + #[serde(default)] + pub sampling_declaration: Option, } #[derive(Deserialize)] #[serde(deny_unknown_fields)] @@ -165,7 +167,7 @@ schema!(CompareParams, [], ["before" => "array"]); schema!(RenameParams, ["name" => "string"], []); schema!(ThemeParams, ["theme_id" => "string"], []); schema!(SchemeParams, ["path" => "string"], ["compatible_only" => "boolean"]); -schema!(ImportParams, ["paths" => "array"], []); +schema!(ImportParams, ["paths" => "array"], ["sampling_declaration" => "object"]); schema!(TransformParams, ["plan" => "object", "name" => "string"], ["memory_limit_bytes" => "integer"]); schema!(ExportParams, ["directory" => "string", "format" => "string"], ["dpi" => "integer", "overwrite" => "boolean"]); schema!(super::properties::PropertyKeyParams, ["key" => "string"], []); diff --git a/crates/core/src/automation/tool_executors.rs b/crates/core/src/automation/tool_executors.rs index 56bf660e..ad7e9219 100644 --- a/crates/core/src/automation/tool_executors.rs +++ b/crates/core/src/automation/tool_executors.rs @@ -249,7 +249,10 @@ pub(super) fn execute_import( parent_id: None, local_id: None, }); - let loaded = match crate::workflow::load_dataset(path) { + let loaded = match params.sampling_declaration.clone().map_or_else( + || crate::workflow::load_dataset(path), + |declaration| crate::workflow::load_dataset_with_sampling(path, declaration), + ) { Ok(loaded) => loaded, Err(error) => { item_results.push(TargetResult { diff --git a/crates/core/src/data_export.rs b/crates/core/src/data_export.rs index 221458d1..fe5dcc5f 100644 --- a/crates/core/src/data_export.rs +++ b/crates/core/src/data_export.rs @@ -8,6 +8,8 @@ use plotx_processing::{Processed2D, Spectrum2D, StackSpectrum, StepKind}; use std::io::{self, Write}; use std::sync::Arc; +#[path = "data_export/nmr.rs"] +mod nmr_export; mod service; pub use service::*; mod write; @@ -482,7 +484,10 @@ fn capture_processed(dataset: &Dataset) -> Result Dataset::Nmr(nmr) => { let (axis, axis_label) = match &nmr.processed { plotx_processing::Processed1D::Time(trace) => (trace.time_s.clone(), "time_s"), - plotx_processing::Processed1D::Frequency(spectrum) => (spectrum.ppm.clone(), "ppm"), + plotx_processing::Processed1D::Frequency(spectrum) => ( + spectrum.ppm.clone(), + plotx_processing::axis_unit_label(Some(spectrum.unit)), + ), }; Ok(SnapshotData::Nmr1D { axis, @@ -490,23 +495,7 @@ fn capture_processed(dataset: &Dataset) -> Result values: nmr.processed.values().to_vec(), }) } - Dataset::Nmr2D(nmr) => match &nmr.processed { - Processed2D::Ft(spectrum) => Ok(SnapshotData::True2D(Arc::clone(spectrum))), - Processed2D::Stack(spectrum) => { - let axis = nmr.data.pseudo_axis.as_ref(); - Ok(SnapshotData::Pseudo2D { - spectrum: Arc::clone(spectrum), - ruler_name: axis - .map(|axis| axis.name.clone()) - .filter(|name| !name.is_empty()) - .unwrap_or_else(|| "Ruler".into()), - ruler_unit: axis.map(|axis| axis.unit.clone()).unwrap_or_default(), - ruler: axis - .map(|axis| axis.values.clone()) - .unwrap_or_else(|| (0..spectrum.increments()).map(|i| i as f64).collect()), - }) - } - }, + Dataset::Nmr2D(nmr) => nmr_export::snapshot_series(nmr), Dataset::Electrophysiology(recording) => { let channel = recording .data diff --git a/crates/core/src/data_export/nmr.rs b/crates/core/src/data_export/nmr.rs new file mode 100644 index 00000000..dd3572ef --- /dev/null +++ b/crates/core/src/data_export/nmr.rs @@ -0,0 +1,37 @@ +use super::*; + +pub(super) fn snapshot_series( + nmr: &crate::state::Nmr2DDataset, +) -> Result { + match &nmr.processed { + Processed2D::Ft(spectrum) => Ok(SnapshotData::True2D(Arc::clone(spectrum))), + Processed2D::Stack(spectrum) => { + let axis = nmr.data.pseudo_axis.as_ref(); + if nmr.stack_field_key() == "nmr.observations" { + let nus = nmr + .data + .nus + .as_ref() + .ok_or(DataExportError::ContentUnavailable)?; + return Ok(SnapshotData::Pseudo2D { + spectrum: Arc::clone(spectrum), + ruler_name: "Acquired grid index (zero based)".into(), + ruler_unit: String::new(), + ruler: nus.schedule.iter().map(|index| *index as f64).collect(), + }); + } + + Ok(SnapshotData::Pseudo2D { + spectrum: Arc::clone(spectrum), + ruler_name: axis + .map(|axis| axis.name.clone()) + .filter(|name| !name.is_empty()) + .unwrap_or_else(|| "Ruler".into()), + ruler_unit: axis.map(|axis| axis.unit.clone()).unwrap_or_default(), + ruler: axis + .map(|axis| axis.values.clone()) + .unwrap_or_else(|| (0..spectrum.increments()).map(|i| i as f64).collect()), + }) + } + } +} diff --git a/crates/core/src/data_export/tests.rs b/crates/core/src/data_export/tests.rs index cb76a2cd..1c0e13bf 100644 --- a/crates/core/src/data_export/tests.rs +++ b/crates/core/src/data_export/tests.rs @@ -225,6 +225,7 @@ fn complete_table_interleaves_sigma_and_leaves_missing_values_empty() { #[test] fn true_2d_matrix_and_long_keep_row_major_axis_order() { let spectrum = Arc::new(Spectrum2D { + magnitude_plane: None, f2_domain: plotx_io::Domain::Frequency, f1_domain: plotx_io::Domain::Frequency, f2_ppm: vec![10.0, 20.0], @@ -239,11 +240,13 @@ fn true_2d_matrix_and_long_keep_row_major_axis_order() { f1_size: 2, direct: plotx_processing::AxisMeta { nucleus: "1H".into(), - observe_freq_mhz: 400.0, + observe_freq_mhz: Some(400.0), + unit: Some(nmr::axis::AxisUnit::Ppm), }, indirect: plotx_processing::AxisMeta { nucleus: "1H".into(), - observe_freq_mhz: 400.0, + observe_freq_mhz: Some(400.0), + unit: Some(nmr::axis::AxisUnit::Ppm), }, source: String::new(), }); @@ -320,7 +323,8 @@ fn pseudo_2d_long_uses_the_actual_ruler_name_and_unit() { traces: vec![vec![Complex64::new(1.0, 2.0), Complex64::new(3.0, 4.0)]], direct: plotx_processing::AxisMeta { nucleus: "1H".into(), - observe_freq_mhz: 400.0, + observe_freq_mhz: Some(400.0), + unit: Some(nmr::axis::AxisUnit::Ppm), }, source: String::new(), }); @@ -462,7 +466,7 @@ fn default_channel_tracks_the_enabled_magnitude_display_step() { source: "spectrum".into(), group_delay: 0.0, }; - let mut nmr = crate::state::NmrDataset::load(data); + let mut nmr = crate::nmr_test_support::load_1d(data).unwrap(); let dataset = Dataset::Nmr(Box::new(nmr.clone())); assert_eq!( DataExportAvailability::for_dataset(&dataset).default_channel, diff --git a/crates/core/src/data_export/write.rs b/crates/core/src/data_export/write.rs index c83b8f5c..07dac634 100644 --- a/crates/core/src/data_export/write.rs +++ b/crates/core/src/data_export/write.rs @@ -259,8 +259,8 @@ pub(super) fn write_true_2d( spectrum: &Spectrum2D, request: DataExportRequest, ) -> io::Result<()> { - let f1_label = domain_column("f1", spectrum.f1_domain); - let f2_label = domain_column("f2", spectrum.f2_domain); + let f1_label = axis_column("f1", spectrum.indirect.unit); + let f2_label = axis_column("f2", spectrum.direct.unit); if request.shape == TableShape::Long { writer.write_record(&[ Field::Text(&f1_label), @@ -273,7 +273,12 @@ pub(super) fn write_true_2d( .data .get(row * spectrum.f2_size + column) .copied() - .map(|value| request.channel.reduce(value)); + .and_then(|value| match request.channel { + IntensityChannel::Magnitude => { + spectrum.magnitude_at(row * spectrum.f2_size + column) + } + _ => Some(request.channel.reduce(value)), + }); writer.write_record(&[ Field::Number(*f1), Field::Number(*f2), @@ -284,8 +289,8 @@ pub(super) fn write_true_2d( return Ok(()); } let mut header = Vec::with_capacity(spectrum.f2_ppm.len() + 1); - let corner = if spectrum.f1_domain == plotx_io::Domain::Frequency - && spectrum.f2_domain == plotx_io::Domain::Frequency + let corner = if spectrum.indirect.unit == Some(nmr::axis::AxisUnit::Ppm) + && spectrum.direct.unit == Some(nmr::axis::AxisUnit::Ppm) { "F1/F2 (ppm)".to_owned() } else { @@ -304,7 +309,12 @@ pub(super) fn write_true_2d( .get(row * spectrum.f2_size + column) .copied() .map_or(Field::Empty, |value| { - Field::Number(request.channel.reduce(value)) + Field::Number(match request.channel { + IntensityChannel::Magnitude => spectrum + .magnitude_at(row * spectrum.f2_size + column) + .expect("view shape"), + _ => request.channel.reduce(value), + }) }), ); } @@ -324,7 +334,7 @@ pub(super) fn write_pseudo_2d( let ruler_header = with_unit(ruler_name, ruler_unit); let direct_label = match spectrum.direct_domain { plotx_io::Domain::Time => "direct_time_s".to_owned(), - plotx_io::Domain::Frequency => "ppm".to_owned(), + plotx_io::Domain::Frequency => spectrum.direct.unit_label().to_owned(), }; if request.shape == TableShape::Long { writer.write_record(&[ @@ -369,10 +379,10 @@ pub(super) fn write_pseudo_2d( Ok(()) } -fn domain_column(axis: &str, domain: plotx_io::Domain) -> String { - match domain { - plotx_io::Domain::Time => format!("{axis}_time_s"), - plotx_io::Domain::Frequency => format!("{axis}_ppm"), +fn axis_column(axis: &str, unit: Option) -> String { + match unit { + Some(nmr::axis::AxisUnit::Second) => format!("{axis}_time_s"), + _ => format!("{axis}_{}", plotx_processing::axis_unit_label(unit)), } } diff --git a/crates/core/src/figures.rs b/crates/core/src/figures.rs index 715bc418..6ba3619d 100644 --- a/crates/core/src/figures.rs +++ b/crates/core/src/figures.rs @@ -7,7 +7,7 @@ use plotx_figure::{ Annotation, Axis, AxisFrame, Color, Contour, ContourBasePolicy, ContourLevelSpec, ContourSpec, Figure, Series, }; -use plotx_io::NmrData; +use plotx_io::nmr_view::NmrSource; use plotx_processing::{Preset2D, Processed1D, Spectrum, Spectrum2D, StackSpectrum, TimeTrace}; use crate::state::{ @@ -15,22 +15,35 @@ use crate::state::{ scalar_grid_capabilities, }; -pub fn build_figure(data: &NmrData, spec: &Spectrum, peaks: &[ResolvedPeak]) -> Figure { +pub fn build_figure(data: &NmrSource, spec: &Spectrum, peaks: &[ResolvedPeak]) -> Figure { let (ppm_lo, ppm_hi) = spec.ppm_bounds(); let (i_lo, i_hi) = spec.intensity_bounds(); let range = (i_hi - i_lo).max(f64::MIN_POSITIVE); // Pad the intensity range, with extra headroom on top for peak labels. let y = Axis::new("Intensity (a.u.)", i_lo - 0.05 * range, i_hi + 0.08 * range); // NMR convention: chemical shift increases to the left. - let x = Axis::new(axis_label(&data.nucleus), ppm_lo, ppm_hi).reversed(true); + let x = Axis::new( + if spec.unit == nmr::axis::AxisUnit::Ppm { + axis_label(data.nucleus()) + } else { + "Frequency (Hz)".into() + }, + ppm_lo, + ppm_hi, + ) + .reversed(spec.unit == nmr::axis::AxisUnit::Ppm); - let fig = Figure::new(format!("{} spectrum — {}", data.nucleus, data.source), x, y) - .with_series(Series::line("real", spec.real_points()).colored(Color::TRACE)); + let fig = Figure::new( + format!("{} spectrum — {}", data.nucleus(), data.source()), + x, + y, + ) + .with_series(Series::line("real", spec.real_points()).colored(Color::TRACE)); apply_peak_labels(fig, peaks) } -pub fn build_time_figure(data: &NmrData, trace: &TimeTrace) -> Figure { +pub fn build_time_figure(data: &NmrSource, trace: &TimeTrace) -> Figure { let (time_lo, time_hi) = trace.time_bounds(); let mut intensity = trace.values.iter().map(|value| value.re); let first = intensity.next().unwrap_or(0.0); @@ -44,12 +57,12 @@ pub fn build_time_figure(data: &NmrData, trace: &TimeTrace) -> Figure { minimum - 0.05 * range, maximum + 0.05 * range, ); - Figure::new(format!("{} FID — {}", data.nucleus, data.source), x, y) + Figure::new(format!("{} FID — {}", data.nucleus(), data.source()), x, y) .with_series(Series::line("real", trace.real_points()).colored(Color::TRACE)) } pub fn build_processed_1d_figure( - data: &NmrData, + data: &NmrSource, processed: &Processed1D, peaks: &[ResolvedPeak], ) -> Figure { @@ -77,8 +90,8 @@ pub fn apply_peak_labels(mut fig: Figure, peaks: &[ResolvedPeak]) -> Figure { pub fn build_figure_2d(spec: &Spectrum2D, preset: Preset2D) -> Figure { let (f2_lo, f2_hi) = spec.f2_bounds(); let (f1_lo, f1_hi) = spec.f1_bounds(); - let x = Axis::new(axis_label(&spec.direct.nucleus), f2_lo, f2_hi).reversed(true); - let y = Axis::new(axis_label(&spec.indirect.nucleus), f1_lo, f1_hi).reversed(true); + let x = nmr_axis(&spec.direct, f2_lo, f2_hi); + let y = nmr_axis(&spec.indirect, f1_lo, f1_hi); let mut fig = Figure::new(format!("{} — {}", preset.label(), spec.source), x, y) .with_axis_frame(AxisFrame::Box); @@ -91,7 +104,9 @@ pub fn build_figure_2d(spec: &Spectrum2D, preset: Preset2D) -> Figure { pub(crate) fn equal_scale_for_nmr_2d(spec: &Spectrum2D) -> bool { if spec.f2_domain != plotx_io::Domain::Frequency || spec.f1_domain != plotx_io::Domain::Frequency + || spec.direct.nucleus.is_empty() || spec.direct.nucleus != spec.indirect.nucleus + || spec.direct.unit != spec.indirect.unit { return false; } @@ -192,9 +207,7 @@ pub fn build_stack_figure(stack: &StackSpectrum) -> Figure { let x = match stack.direct_domain { plotx_io::Domain::Time => Axis::new("Direct acquisition time (s)", lo, hi), - plotx_io::Domain::Frequency => { - Axis::new(axis_label(&stack.direct.nucleus), lo, hi).reversed(true) - } + plotx_io::Domain::Frequency => nmr_axis(&stack.direct, lo, hi), }; // The stack is phased to absorptive, so traces carry the signed real part: // short-τ relaxation increments dip below their baseline (inverted peaks). @@ -215,6 +228,17 @@ pub fn build_stack_figure(stack: &StackSpectrum) -> Figure { fig } +pub(crate) fn nmr_axis(meta: &plotx_processing::AxisMeta, lo: f64, hi: f64) -> Axis { + use nmr::axis::AxisUnit; + let label = match meta.unit { + Some(AxisUnit::Ppm) => axis_label(&meta.nucleus), + Some(AxisUnit::Hertz) => format!("{} frequency (Hz)", format_nucleus(&meta.nucleus)), + Some(AxisUnit::Second) => "Acquisition time (s)".into(), + _ => "Coordinate".into(), + }; + Axis::new(label, lo, hi).reversed(meta.unit == Some(AxisUnit::Ppm)) +} + pub(crate) fn axis_label(nucleus: &str) -> String { format!("{} chemical shift (ppm)", format_nucleus(nucleus)) } @@ -559,6 +583,7 @@ mod tests { let f1_ppm = vec![0.0, 1.0, 2.0, 3.0]; let (f2_size, f1_size) = (f2_ppm.len(), f1_ppm.len()); Spectrum2D { + magnitude_plane: None, f2_domain: plotx_io::Domain::Frequency, f1_domain: plotx_io::Domain::Frequency, data: vec![Complex64::new(1.0, 0.0); f1_size * f2_size], @@ -568,11 +593,13 @@ mod tests { f1_size, direct: AxisMeta { nucleus: "1H".to_owned(), - observe_freq_mhz: 400.0, + observe_freq_mhz: Some(400.0), + unit: Some(nmr::axis::AxisUnit::Ppm), }, indirect: AxisMeta { nucleus: "13C".to_owned(), - observe_freq_mhz: 100.0, + observe_freq_mhz: Some(100.0), + unit: Some(nmr::axis::AxisUnit::Ppm), }, source: "test".to_owned(), } diff --git a/crates/core/src/lib.rs b/crates/core/src/lib.rs index 12730087..363b8bba 100644 --- a/crates/core/src/lib.rs +++ b/crates/core/src/lib.rs @@ -260,8 +260,9 @@ mod tests { .into_iter() .map(|v| Complex64::new(v, 0.0)) .collect(), - hz_per_point: 1.0, - observe_freq_mhz: 400.0, + unit: nmr::axis::AxisUnit::Ppm, + hz_per_point: Some(1.0), + observe_freq_mhz: Some(400.0), nucleus: "1H".to_owned(), } } @@ -274,3 +275,6 @@ mod tests { assert!((integral.area - 9.0).abs() < 1e-9); } } + +#[cfg(test)] +mod nmr_test_support; diff --git a/crates/core/src/nmr_test_support.rs b/crates/core/src/nmr_test_support.rs new file mode 100644 index 00000000..3ba8986b --- /dev/null +++ b/crates/core/src/nmr_test_support.rs @@ -0,0 +1,46 @@ +//! Explicit manual recipes for presentation and application-state fixtures. +use crate::state::{Nmr2DDataset, NmrDataset}; +use plotx_processing::{AxisPipeline, Params2D, PhaseParams, StepKind}; +fn manual(pipeline: &mut AxisPipeline) { + for step in &mut pipeline.steps { + if let StepKind::Phase(ref mut phase) = step.kind { + *phase = PhaseParams::MANUAL_ZERO; + } + } +} +pub(crate) fn load_1d(input: plotx_io::NmrData) -> Result { + let mut pipeline = match input.domain { + plotx_io::Domain::Time => AxisPipeline::default_1d(), + plotx_io::Domain::Frequency => AxisPipeline::frequency_1d(), + }; + manual(&mut pipeline); + NmrDataset::load_with_pipeline(input, Some(pipeline), None) +} +pub(crate) fn load_2d(input: plotx_io::NmrData2D) -> Result { + let source = plotx_io::nmr_series::NmrSeriesSource::try_from(input) + .map_err(|error| error.to_string())?; + let preset = plotx_processing::recommend_preset(&source); + let mut params = if source.direct.domain == nmr::axis::AxisDomain::Time { + Params2D::default_for(preset) + } else { + Params2D::frequency_domain(preset) + }; + if source.indirect.domain == nmr::axis::AxisDomain::Parameter { + params.layout = plotx_processing::Layout2D::Stack; + params.f1.steps.clear(); + } + manual(&mut params.f2); + manual(&mut params.f1); + for (axis, pipeline) in [(1, &mut params.f2), (0, &mut params.f1)] { + if source.source_dataset().axes()[axis].domain == nmr::axis::AxisDomain::Frequency + && !source.source_dataset().has_imaginary(axis) + { + for step in &mut pipeline.steps { + if matches!(step.kind, StepKind::Phase(_)) { + step.enabled = false; + } + } + } + } + Nmr2DDataset::load_with_pipeline(source, Some(params), None, None, true) +} diff --git a/crates/core/src/project/acquisition_identity_tests.rs b/crates/core/src/project/acquisition_identity_tests.rs index a1a80b2b..05df7b6f 100644 --- a/crates/core/src/project/acquisition_identity_tests.rs +++ b/crates/core/src/project/acquisition_identity_tests.rs @@ -20,40 +20,20 @@ fn v1_rejects_dataset_objects_without_acquisition_identity() { } #[test] -fn v1_requires_an_explicit_nmr_origin_and_preserves_it_exactly() { - let mut app = tests::sample_app(); - let origin = plotx_io::NmrOrigin::Instrument { - instrument: plotx_io::NmrInstrumentOrigin { - format: plotx_io::NmrSourceFormat::BrukerRaw, - source_sha256: [42; 32], - portable: plotx_io::NmrPortableMetadata::default(), - parameters: plotx_io::NmrSourceParameters::Bruker { - acqus: "##$TD= 2048".to_owned(), - title: Some("Sample".to_owned()), - pulse_program: Some("zg30".to_owned()), - }, - }, - }; - app.doc.datasets[0].as_nmr_mut().unwrap().origin = origin.clone(); - let mut objects = dataset_to_objects(&app.doc.datasets[0], "data-1", "recipe-1").unwrap(); - assert_eq!(read_nmr_origin(&objects.data).unwrap(), origin); - - objects.data.extensions["plotx.nmr"] - .as_object_mut() - .unwrap() - .remove("origin"); - let error = read_nmr_origin(&objects.data).unwrap_err(); - assert!( - error - .to_string() - .contains("missing required plotx.nmr.origin") - ); - - app.doc.datasets[0].as_nmr_mut().unwrap().origin = origin.clone(); - let path = tests::temp_project("nmr-instrument-origin"); +fn v1_embeds_one_nmr_snapshot_without_duplicate_vendor_metadata() { + let app = tests::sample_app(); + let source = app.doc.datasets[0].as_nmr().unwrap().data.dataset(); + let objects = dataset_to_objects(&app.doc.datasets[0], "data-1", "recipe-1").unwrap(); + assert_eq!(objects.data.payload.storage, super::nmr_snapshot::STORAGE); + assert!(objects.data.dimensions.is_empty()); + assert!(objects.data.extensions.get("plotx.nmr").is_none()); + let path = tests::temp_project("nmr-snapshot"); save_project(&app, &path, false).unwrap(); - let loaded = load_project(&path).unwrap(); - let restored = loaded.doc.datasets[0].as_nmr().unwrap(); - assert_eq!(restored.origin, origin); + let restored = load_project(&path).unwrap(); + let restored = restored.doc.datasets[0].as_nmr().unwrap(); + assert_eq!( + restored.data.dataset().canonical_digests(), + source.canonical_digests() + ); std::fs::remove_file(path).unwrap(); } diff --git a/crates/core/src/project/cleanup_tests.rs b/crates/core/src/project/cleanup_tests.rs index f38ed724..ec2f4a16 100644 --- a/crates/core/src/project/cleanup_tests.rs +++ b/crates/core/src/project/cleanup_tests.rs @@ -5,7 +5,7 @@ use crate::state::Dataset; #[test] fn project_and_scheme_roundtrips_preserve_cleanup_steps() { let mut app = PlotxApp::new(); - let mut dataset = NmrDataset::load(synthetic_1d()); + let mut dataset = NmrDataset::load(synthetic_1d()).unwrap(); let cleanup = [ StepKind::Smooth(SmoothMethod::SavitzkyGolay { window: 11, @@ -26,7 +26,7 @@ fn project_and_scheme_roundtrips_preserve_cleanup_steps() { .steps .push(ProcessingStep::new(id, kind.clone(), StepSource::User)); } - dataset.retransform(); + dataset.retransform().unwrap(); let expected: Vec = cleanup.to_vec(); app.doc.datasets.push(Dataset::Nmr(Box::new(dataset))); @@ -49,7 +49,7 @@ fn project_and_scheme_roundtrips_preserve_cleanup_steps() { save_scheme(&scheme_path, &loaded.doc.datasets[0]).unwrap(); let scheme = load_scheme(&scheme_path).unwrap(); let _ = std::fs::remove_file(&scheme_path); - let target = Dataset::Nmr(Box::new(NmrDataset::load(synthetic_1d()))); + let target = Dataset::Nmr(Box::new(NmrDataset::load(synthetic_1d()).unwrap())); let crate::actions::DatasetProcessingState::Nmr { pipeline, .. } = apply_scheme(&scheme, &target).unwrap() else { @@ -64,7 +64,7 @@ fn project_and_scheme_roundtrips_preserve_cleanup_steps() { #[test] fn applying_a_scheme_reports_an_invalid_stored_smoothing_window() { - let target = Dataset::Nmr(Box::new(NmrDataset::load(synthetic_1d()))); + let target = Dataset::Nmr(Box::new(NmrDataset::load(synthetic_1d()).unwrap())); let mut pipeline = AxisPipeline::default_1d(); pipeline.steps.push(ProcessingStep::new( StepId::new(99), diff --git a/crates/core/src/project/codec.rs b/crates/core/src/project/codec.rs index 8742c11f..474266ba 100644 --- a/crates/core/src/project/codec.rs +++ b/crates/core/src/project/codec.rs @@ -240,19 +240,7 @@ pub fn write_dataset_blob( ) -> Result<()> { zip.start_file(path, options)?; match blob { - DatasetBlob::Complex(values) => { - const VALUES_PER_CHUNK: usize = 4096; - let mut buffer = Vec::with_capacity(VALUES_PER_CHUNK * 16); - for chunk in values.chunks(VALUES_PER_CHUNK) { - buffer.clear(); - for value in chunk { - buffer.extend_from_slice(&value.re.to_le_bytes()); - buffer.extend_from_slice(&value.im.to_le_bytes()); - } - zip.write_all(&buffer)?; - } - Ok(()) - } + DatasetBlob::Nmr(source) => super::nmr_snapshot::write(zip, source), DatasetBlob::Electrophysiology(recording) => { super::electrophysiology_convert::write_electrophysiology_blob(zip, recording) } @@ -318,105 +306,6 @@ pub fn validate_manifest(manifest: &Manifest) -> Result<()> { Ok(()) } -pub fn complex_to_bytes(values: &[Complex64]) -> Vec { - let mut out = Vec::with_capacity(values.len() * 16); - for c in values { - out.extend_from_slice(&c.re.to_le_bytes()); - out.extend_from_slice(&c.im.to_le_bytes()); - } - out -} - -pub fn complex_from_bytes(raw: &[u8]) -> Result> { - if !raw.len().is_multiple_of(16) { - return Err(ProjectError::Invalid(format!( - "complex blob length {} is not divisible by 16", - raw.len() - ))); - } - Ok(raw - .as_chunks::<16>() - .0 - .iter() - .map(|chunk| { - let mut re = [0u8; 8]; - let mut im = [0u8; 8]; - re.copy_from_slice(&chunk[..8]); - im.copy_from_slice(&chunk[8..]); - Complex64::new(f64::from_le_bytes(re), f64::from_le_bytes(im)) - }) - .collect()) -} - -pub fn complex_from_reader(reader: &mut EntryReader<'_, R>) -> Result> { - if !reader.remaining().is_multiple_of(16) { - return Err(reader.invalid(format!( - "complex blob length {} is not divisible by 16", - reader.remaining() - ))); - } - let count = usize::try_from(reader.remaining() / 16) - .map_err(|_| reader.invalid("complex element count exceeds usize"))?; - let mut values = Vec::new(); - values - .try_reserve_exact(count) - .map_err(|_| reader.invalid("could not reserve complex data"))?; - let mut bytes = [0_u8; 16]; - for _ in 0..count { - reader - .read_exact(&mut bytes) - .map_err(|error| reader.invalid(format!("complex blob is truncated: {error}")))?; - let re = f64::from_le_bytes(bytes[..8].try_into().expect("fixed eight-byte half")); - let im = f64::from_le_bytes(bytes[8..].try_into().expect("fixed eight-byte half")); - values.push(Complex64::new(re, im)); - } - Ok(values) -} - -pub fn required(value: Option, name: &str) -> Result { - value.ok_or_else(|| ProjectError::Invalid(format!("missing dimension field {name}"))) -} - -pub fn nmr_source(data: &DataObject) -> String { - nmr_ext_str(data, "source") - .map(str::to_owned) - .unwrap_or_else(|| data.id.clone()) -} - -pub fn read_nmr_origin(data: &DataObject) -> Result { - let value = data - .extensions - .get("plotx.nmr") - .and_then(|value| value.get("origin")) - .cloned() - .ok_or_else(|| { - ProjectError::Invalid(format!( - "NMR dataset {} is missing required plotx.nmr.origin", - data.id - )) - })?; - serde_json::from_value(value).map_err(|error| { - ProjectError::Invalid(format!( - "NMR dataset {} has invalid plotx.nmr.origin: {error}", - data.id - )) - }) -} - -pub fn nmr_ext_str<'a>(data: &'a DataObject, key: &str) -> Option<&'a str> { - data.extensions - .get("plotx.nmr") - .and_then(|v| v.get(key)) - .and_then(|v| v.as_str()) -} - -pub fn nmr_ext_bool(data: &DataObject, key: &str) -> Option { - data.extensions - .get("plotx.nmr") - .and_then(|v| v.get(key)) - .and_then(|v| v.as_bool()) -} - pub fn temporary_path(path: &Path) -> PathBuf { let mut tmp = path.to_owned(); let name = path @@ -428,20 +317,6 @@ pub fn temporary_path(path: &Path) -> PathBuf { tmp } -pub fn domain_to_str(v: Domain) -> &'static str { - match v { - Domain::Time => "time", - Domain::Frequency => "frequency", - } -} - -pub fn domain_from_str(v: &str) -> Domain { - match v { - "frequency" => Domain::Frequency, - _ => Domain::Time, - } -} - pub fn layout_to_str(v: Layout2D) -> &'static str { match v { Layout2D::Ft => "ft", @@ -482,112 +357,6 @@ pub fn preset_from_str(v: &str) -> Preset2D { } } -pub fn quad_to_str(v: QuadMode) -> &'static str { - match v { - QuadMode::Complex => "complex", - QuadMode::States => "states", - QuadMode::StatesTppi => "states_tppi", - QuadMode::EchoAntiecho => "echo_antiecho", - } -} - -pub fn quad_from_str(v: &str) -> QuadMode { - match v { - "states" => QuadMode::States, - "states_tppi" => QuadMode::StatesTppi, - "echo_antiecho" => QuadMode::EchoAntiecho, - _ => QuadMode::Complex, - } -} - -pub fn pseudo_kind_to_str(v: PseudoKind) -> &'static str { - match v { - PseudoKind::Gradient => "gradient", - PseudoKind::Delay => "delay", - PseudoKind::Generic => "generic", - } -} - -pub fn pseudo_kind_from_str(v: &str) -> PseudoKind { - match v { - "gradient" => PseudoKind::Gradient, - "delay" => PseudoKind::Delay, - _ => PseudoKind::Generic, - } -} - -pub fn axis_source_to_str(v: AxisSource) -> &'static str { - match v { - AxisSource::EmbeddedList => "embedded_list", - AxisSource::EmbeddedRamp => "embedded_ramp", - AxisSource::LinearHeader => "linear_header", - AxisSource::Manual => "manual", - } -} - -pub fn axis_source_from_str(v: &str) -> AxisSource { - match v { - "embedded_list" => AxisSource::EmbeddedList, - "embedded_ramp" => AxisSource::EmbeddedRamp, - "manual" => AxisSource::Manual, - _ => AxisSource::LinearHeader, - } -} - -pub fn pseudo_axis_to_dto(axis: &PseudoAxis) -> PseudoAxisDto { - PseudoAxisDto { - name: axis.name.clone(), - kind: pseudo_kind_to_str(axis.kind).to_owned(), - values: axis.values.clone(), - unit: axis.unit.clone(), - source: axis_source_to_str(axis.source).to_owned(), - } -} - -pub fn pseudo_axis_from_dto(dto: PseudoAxisDto) -> PseudoAxis { - PseudoAxis { - name: dto.name, - kind: pseudo_kind_from_str(&dto.kind), - values: dto.values, - unit: dto.unit, - source: axis_source_from_str(&dto.source), - } -} - -pub fn diffusion_to_dto(meta: &DiffusionMeta) -> DiffusionMetaDto { - DiffusionMetaDto { - gamma: meta.gamma, - delta: meta.delta, - big_delta: meta.big_delta, - tau: meta.tau, - shape_factor: meta.shape_factor, - } -} - -pub fn diffusion_from_dto(dto: DiffusionMetaDto) -> DiffusionMeta { - DiffusionMeta { - gamma: dto.gamma, - delta: dto.delta, - big_delta: dto.big_delta, - tau: dto.tau, - shape_factor: dto.shape_factor, - } -} - -pub fn read_pseudo_axis(data: &DataObject) -> Option { - let value = data.extensions.get("plotx.nmr")?.get("pseudo_axis")?; - serde_json::from_value::(value.clone()) - .ok() - .map(pseudo_axis_from_dto) -} - -pub fn read_diffusion(data: &DataObject) -> Option { - let value = data.extensions.get("plotx.nmr")?.get("diffusion")?; - serde_json::from_value::(value.clone()) - .ok() - .map(diffusion_from_dto) -} - pub fn primary_view_to_str(v: PrimaryView) -> &'static str { match v { PrimaryView::Canvas => "canvas", diff --git a/crates/core/src/project/codec_tests.rs b/crates/core/src/project/codec_tests.rs index 90c9420e..5f03caaf 100644 --- a/crates/core/src/project/codec_tests.rs +++ b/crates/core/src/project/codec_tests.rs @@ -141,7 +141,7 @@ fn crc_failure_surfaces_when_success_path_consumes_eof() { } #[test] -fn complex_decoder_uses_constant_sized_read_requests() { +fn native_snapshot_decoder_keeps_streamed_reads_bounded() { struct Probe { inner: Cursor>, largest_request: usize, @@ -152,13 +152,33 @@ fn complex_decoder_uses_constant_sized_read_requests() { self.inner.read(buffer) } } + let input = plotx_io::nmr_view::NmrSource::try_from(NmrData { + points: vec![Complex64::new(0.0, 0.0); 100_000], + domain: Domain::Frequency, + spectral_width_hz: 1000.0, + observe_freq_mhz: 100.0, + carrier_ppm: 0.0, + nucleus: "1H".into(), + source: "streamed snapshot".into(), + group_delay: 0.0, + }) + .unwrap(); + let limits = nmr::snapshot::SnapshotLimits::default(); + let mut bytes = Vec::new(); + let mut context = nmr::ExecutionContext::default(); + plotx_io::nmr_bridge::snapshot::write(input.dataset(), &mut bytes, limits, &mut context) + .unwrap(); + let length = bytes.len() as u64; let probe = Probe { - inner: Cursor::new(vec![0_u8; 16 * 100_000]), + inner: Cursor::new(bytes), largest_request: 0, }; - let mut reader = EntryReader::new(probe, "nmr.bin", "NMR", 1_600_000, 1_600_000).unwrap(); - let values = complex_from_reader(&mut reader).unwrap(); - assert_eq!(values.len(), 100_000); - assert!(reader.inner.largest_request <= 16); + let mut reader = EntryReader::new(probe, "nmr.bin", "NMR snapshot", length, length).unwrap(); + let output = plotx_io::nmr_bridge::snapshot::read(&mut reader, limits, &mut context).unwrap(); + assert_eq!( + output.canonical_digests(), + input.dataset().canonical_digests() + ); + assert!(reader.inner.largest_request <= 64 * 1024); reader.finish().unwrap(); } diff --git a/crates/core/src/project/convert.rs b/crates/core/src/project/convert.rs index 1a1787d2..6cb12124 100644 --- a/crates/core/src/project/convert.rs +++ b/crates/core/src/project/convert.rs @@ -8,7 +8,7 @@ use crate::{DosyMethod, PseudoDisplay}; use plotx_processing::Processed2D; pub enum DatasetBlob<'a> { - Complex(&'a [Complex64]), + Nmr(&'a nmr::Dataset), Electrophysiology(&'a crate::state::ElectrophysiologyDataset), Afm(&'a plotx_io::AfmData), MassSpec(&'a crate::state::MassSpecDataset), @@ -47,18 +47,14 @@ pub fn dataset_to_objects<'a>( role: "data".to_owned(), classification: nmr_acquisition_classification(), label: n.name.clone(), - dimensions: vec![dimension_from_1d(&n.data)], + dimensions: Vec::new(), payload: Payload { - storage: STORAGE_COMPLEX_F64_LE.to_owned(), + storage: super::nmr_snapshot::STORAGE.to_owned(), blob: format!("objects/{data_id}/data.bin"), - shape: vec![n.data.points.len()], - domain: domain_to_str(n.data.domain).to_owned(), + shape: vec![n.data.len()], + domain: "nmr".to_owned(), }, extensions: serde_json::json!({ - "plotx.nmr": { - "source": &n.data.source, - "origin": &n.origin - }, "plotx.fields": &n.field_catalog }), }; @@ -86,7 +82,7 @@ pub fn dataset_to_objects<'a>( } }), }; - DatasetObjects::primary(data, DatasetBlob::Complex(&n.data.points), recipe) + DatasetObjects::primary(data, DatasetBlob::Nmr(n.data.dataset()), recipe) } Dataset::Nmr2D(n) => { let data = DataObject { @@ -94,26 +90,20 @@ pub fn dataset_to_objects<'a>( role: "data".to_owned(), classification: nmr_acquisition_classification(), label: n.name.clone(), - dimensions: vec![ - dimension_from_dim("f1", "indirect", 0, n.data.rows, &n.data.indirect), - dimension_from_dim("f2", "direct", 1, n.data.cols, &n.data.direct), - ], + dimensions: Vec::new(), payload: Payload { - storage: STORAGE_COMPLEX_F64_LE.to_owned(), + storage: super::nmr_snapshot::STORAGE.to_owned(), blob: format!("objects/{data_id}/data.bin"), - shape: vec![n.data.rows, n.data.cols], - domain: domain_to_str(n.data.domain).to_owned(), + shape: n + .data + .source_dataset() + .axes() + .iter() + .map(|axis| axis.points) + .collect(), + domain: "nmr".to_owned(), }, extensions: serde_json::json!({ - "plotx.nmr": { - "source": &n.data.source, - "origin": &n.origin, - "quad": quad_to_str(n.data.quad), - "indirect_conjugate": n.data.indirect_conjugate, - "experiment_hint": &n.data.experiment, - "pseudo_axis": n.data.pseudo_axis.as_ref().map(pseudo_axis_to_dto), - "diffusion": n.data.diffusion.as_ref().map(diffusion_to_dto), - }, "plotx.fields": &n.field_catalog }), }; @@ -177,7 +167,7 @@ pub fn dataset_to_objects<'a>( }; DatasetObjects { data, - blob: DatasetBlob::Complex(&n.data.data), + blob: DatasetBlob::Nmr(n.data.source_dataset().dataset()), recipe, extra_blobs, } @@ -320,6 +310,20 @@ pub fn dataset_to_objects<'a>( Dataset::Xps(xps) => super::xps_convert::to_objects(xps, data_id, recipe_id), }; write_acquisition_identity(&mut objects.data, dataset.acquisition_identity())?; + let execution = match dataset { + Dataset::Nmr(n) => Some(super::nmr_snapshot::execution_evidence( + &n.native_processed, + &n.phase_reports, + )?), + Dataset::Nmr2D(n) => Some(super::nmr_snapshot::execution_evidence( + &n.native_processed, + &n.phase_reports, + )?), + _ => None, + }; + if let Some(execution) = execution { + objects.data.extensions["plotx.nmr_execution"] = execution; + } Ok(objects) } @@ -489,144 +493,29 @@ pub fn object_to_dataset( data.classification.domain, data.classification.technique, data.classification.object ))); } - if data.payload.storage != STORAGE_COMPLEX_F64_LE { - return Err(ProjectError::Unsupported(format!( - "payload storage {}", - data.payload.storage - ))); + if recipe.parameters.dimension_count == 1 { + return super::nmr_snapshot::read_1d(zip, data, recipe); } - let expected_values = match data.dimensions.len() { - 1 => data - .payload - .shape - .first() - .copied() - .unwrap_or(data.dimensions[0].size), - 2 => data - .payload - .shape - .first() - .copied() - .zip(data.payload.shape.get(1).copied()) - .ok_or_else(|| ProjectError::Invalid("2D payload shape is incomplete".to_owned()))? - .0 - .checked_mul(data.payload.shape[1]) - .ok_or_else(|| ProjectError::Invalid("2D NMR shape overflows usize".to_owned()))?, - n => { - return Err(ProjectError::Unsupported(format!( - "NMR acquisitions with {n} dimensions" - ))); - } - }; - let expected_bytes = expected_values.checked_mul(16).ok_or_else(|| { - ProjectError::Invalid("NMR payload byte length overflows usize".to_owned()) - })?; - let values = read_entry( - zip, - &data.payload.blob, - "NMR complex-f64 payload", - ProjectLoadLimits::default().max_entry_bytes, - |reader| { - if reader.remaining() != expected_bytes as u64 { - return Err(reader.invalid(format!( - "complex payload has {} bytes but shape requires {expected_bytes}", - reader.remaining() - ))); - } - complex_from_reader(reader) - }, - )?; - match data.dimensions.len() { - 1 => { - let dim = data.dimensions.first().unwrap(); - let expected = data.payload.shape.first().copied().unwrap_or(dim.size); - if values.len() != expected { - return Err(ProjectError::Invalid(format!( - "1D data length {} does not match shape {expected}", - values.len() - ))); - } - let mut dataset = NmrDataset::load_with_origin( - NmrData { - points: values, - domain: domain_from_str(&data.payload.domain), - spectral_width_hz: required(dim.spectral_width_hz, "spectral_width_hz")?, - observe_freq_mhz: required(dim.observe_freq_mhz, "observe_freq_mhz")?, - carrier_ppm: required(dim.carrier_ppm, "carrier_ppm")?, - nucleus: dim.nucleus.clone().unwrap_or_else(|| "X".to_owned()), - source: nmr_source(data), - group_delay: dim.group_delay.unwrap_or(0.0), - }, - read_nmr_origin(data)?, - ); - dataset.acquisition_identity = read_acquisition_identity(data)?; - dataset.field_catalog = read_field_catalog(data)?; - apply_1d_recipe(&mut dataset, recipe)?; - dataset.name = data.label.clone(); - dataset.retransform(); - let dataset = Dataset::Nmr(Box::new(dataset)); - dataset - .validate_field_catalog() - .map_err(ProjectError::Invalid)?; - Ok(dataset) - } + let source = super::nmr_snapshot::read(zip, data)?; + match source.axes().len() { 2 => { - let rows = *data - .payload - .shape - .first() - .ok_or_else(|| ProjectError::Invalid("2D payload missing rows".to_owned()))?; - let cols = *data - .payload - .shape - .get(1) - .ok_or_else(|| ProjectError::Invalid("2D payload missing cols".to_owned()))?; - let expected_len = rows - .checked_mul(cols) - .ok_or_else(|| ProjectError::Invalid("2D NMR shape overflows usize".to_owned()))?; - if values.len() != expected_len { - return Err(ProjectError::Invalid(format!( - "2D data length {} does not match shape {}x{}", - values.len(), - rows, - cols - ))); - } - let direct = data - .dimensions - .iter() - .find(|d| d.role == "direct") - .or_else(|| data.dimensions.iter().find(|d| d.storage_axis == 1)) - .ok_or_else(|| { - ProjectError::Invalid("2D data missing direct dimension".to_owned()) - })?; - let indirect = data - .dimensions - .iter() - .find(|d| d.role == "indirect") - .or_else(|| data.dimensions.iter().find(|d| d.storage_axis == 0)) - .ok_or_else(|| { - ProjectError::Invalid("2D data missing indirect dimension".to_owned()) - })?; - let mut dataset = Nmr2DDataset::load_with_origin_and_equal_scale_preference( - NmrData2D { - data: values, - rows, - cols, - domain: domain_from_str(&data.payload.domain), - direct: dim_from_dimension(direct)?, - indirect: dim_from_dimension(indirect)?, - quad: quad_from_str(nmr_ext_str(data, "quad").unwrap_or("complex")), - indirect_conjugate: nmr_ext_bool(data, "indirect_conjugate").unwrap_or(false), - experiment: nmr_ext_str(data, "experiment_hint").map(str::to_owned), - pseudo_axis: read_pseudo_axis(data), - diffusion: read_diffusion(data), - nus: None, - source: nmr_source(data), + let initial = Params2D { + layout: if source.axes()[0].domain == nmr::axis::AxisDomain::Parameter + || source + .dataset() + .as_raw() + .is_some_and(|raw| raw.data().is_sparse()) + { + Layout2D::Stack + } else { + Layout2D::Ft }, - read_nmr_origin(data)?, - true, - ); + f2: AxisPipeline { steps: Vec::new() }, + f1: AxisPipeline { steps: Vec::new() }, + }; + let mut dataset = + Nmr2DDataset::load_with_pipeline(source, Some(initial), Some(false), None, true) + .map_err(ProjectError::Invalid)?; dataset.acquisition_identity = read_acquisition_identity(data)?; dataset.field_catalog = read_field_catalog(data)?; apply_2d_recipe(&mut dataset, recipe)?; @@ -634,7 +523,7 @@ pub fn object_to_dataset( read_integrals_2d(&mut dataset, recipe)?; read_peaks_2d(&mut dataset, recipe)?; dataset.name = data.label.clone(); - dataset.retransform(); + dataset.retransform().map_err(ProjectError::Invalid)?; // `retransform` deliberately invalidates every analysis map. Restore // auxiliary DOSY state only after it, or a load will silently erase // the stored result and serve the stack fallback instead. diff --git a/crates/core/src/project/convert_dimensions.rs b/crates/core/src/project/convert_dimensions.rs deleted file mode 100644 index 25a1bfed..00000000 --- a/crates/core/src/project/convert_dimensions.rs +++ /dev/null @@ -1,51 +0,0 @@ -use super::*; - -pub fn dimension_from_1d(data: &NmrData) -> Dimension { - Dimension { - id: "f2".to_owned(), - role: "direct".to_owned(), - size: data.points.len(), - storage_axis: 0, - quantity: "time_or_frequency".to_owned(), - display_quantity: Some("chemical_shift".to_owned()), - unit: Some("ppm".to_owned()), - nucleus: Some(data.nucleus.clone()), - spectral_width_hz: Some(data.spectral_width_hz), - observe_freq_mhz: Some(data.observe_freq_mhz), - carrier_ppm: Some(data.carrier_ppm), - group_delay: Some(data.group_delay), - } -} - -pub fn dimension_from_dim( - id: &str, - role: &str, - storage_axis: usize, - size: usize, - dim: &Dim, -) -> Dimension { - Dimension { - id: id.to_owned(), - role: role.to_owned(), - size, - storage_axis, - quantity: "time_or_frequency".to_owned(), - display_quantity: Some("chemical_shift".to_owned()), - unit: Some("ppm".to_owned()), - nucleus: Some(dim.nucleus.clone()), - spectral_width_hz: Some(dim.spectral_width_hz), - observe_freq_mhz: Some(dim.observe_freq_mhz), - carrier_ppm: Some(dim.carrier_ppm), - group_delay: Some(dim.group_delay), - } -} - -pub fn dim_from_dimension(dim: &Dimension) -> Result { - Ok(Dim { - spectral_width_hz: required(dim.spectral_width_hz, "spectral_width_hz")?, - observe_freq_mhz: required(dim.observe_freq_mhz, "observe_freq_mhz")?, - carrier_ppm: required(dim.carrier_ppm, "carrier_ppm")?, - nucleus: dim.nucleus.clone().unwrap_or_else(|| "X".to_owned()), - group_delay: dim.group_delay.unwrap_or(0.0), - }) -} diff --git a/crates/core/src/project/convert_recipes.rs b/crates/core/src/project/convert_recipes.rs index 4c1030a5..bc0ba532 100644 --- a/crates/core/src/project/convert_recipes.rs +++ b/crates/core/src/project/convert_recipes.rs @@ -19,7 +19,7 @@ pub fn apply_1d_recipe(dataset: &mut NmrDataset, recipe: &RecipeObject) -> Resul .unwrap_or(0); dataset.repair_step_allocator(); dataset.group_delay_correct = p.group_delay_correct; - dataset.has_imaginary = true; + dataset.has_imaginary = dataset.data.has_imaginary(0); let analysis = recipe.extensions.get("plotx.analysis").ok_or_else(|| { ProjectError::Invalid("1D NMR recipe is missing plotx.analysis".to_owned()) })?; @@ -88,6 +88,10 @@ pub(super) fn nmr2d_recipe_extensions( dosy: Option, ) -> serde_json::Value { let mut extensions = serde_json::Map::new(); + extensions.insert( + "plotx.nus_request".into(), + serde_json::json!(dataset.nus_request), + ); extensions.insert( "plotx.step_allocator".to_owned(), serde_json::json!({ "next_id": dataset.next_step_id }), @@ -123,6 +127,13 @@ pub(super) fn nmr2d_recipe_extensions( } pub fn apply_2d_recipe(dataset: &mut Nmr2DDataset, recipe: &RecipeObject) -> Result<()> { + dataset.nus_request = recipe + .extensions + .get("plotx.nus_request") + .map(|value| serde_json::from_value(value.clone())) + .transpose() + .map_err(|error| ProjectError::Invalid(format!("Invalid NUS request: {error}")))? + .flatten(); let p = &recipe.parameters; let preset = p .preset @@ -141,14 +152,8 @@ pub fn apply_2d_recipe(dataset: &mut Nmr2DDataset, recipe: &RecipeObject) -> Res .as_deref() .map(layout_from_str) .unwrap_or_else(|| preset.layout()); - params - .f2 - .output_domain(dataset.data.domain) - .map_err(|error| ProjectError::Invalid(format!("invalid F2 pipeline: {error}")))?; - params - .f1 - .output_domain(dataset.data.domain) - .map_err(|error| ProjectError::Invalid(format!("invalid F1 pipeline: {error}")))?; + plotx_processing::nmr_execution::validate_2d_domains(&dataset.data, ¶ms) + .map_err(ProjectError::Invalid)?; dataset.preset = preset; dataset.params = params; @@ -160,6 +165,6 @@ pub fn apply_2d_recipe(dataset: &mut Nmr2DDataset, recipe: &RecipeObject) -> Res .unwrap_or(0); dataset.repair_step_allocator(); dataset.group_delay_correct = p.group_delay_correct; - dataset.has_imaginary = true; + dataset.has_imaginary = dataset.data.source_dataset().has_imaginary(1); Ok(()) } diff --git a/crates/core/src/project/craft_tests.rs b/crates/core/src/project/craft_tests.rs index 57bdbadd..71a3e9df 100644 --- a/crates/core/src/project/craft_tests.rs +++ b/crates/core/src/project/craft_tests.rs @@ -15,10 +15,10 @@ use plotx_processing::craft::{ fn sample_run(data: &NmrData) -> StoredCraftRun { StoredCraftRun::from_result( CraftRunId(4), - data, + &data.clone().try_into().unwrap(), resolve_craft_invocation( data, - CraftReference::new(data.carrier_ppm, 0.25), + CraftReference::new(data.carrier_ppm, data.observe_freq_mhz, 0.25), &CraftParamOverrides::from_params(CraftParams::conventional()), None, ), @@ -143,7 +143,7 @@ fn sample_run(data: &NmrData) -> StoredCraftRun { #[test] fn craft_runs_survive_project_roundtrip_and_reseed_ids() { let data = synthetic_1d(); - let mut dataset = NmrDataset::load(data.clone()); + let mut dataset = NmrDataset::load(data.clone()).unwrap(); dataset.craft_runs.push(sample_run(&data)); dataset.reconcile_craft_fields(); dataset.next_craft_run_id = 5; @@ -166,7 +166,7 @@ fn craft_runs_survive_project_roundtrip_and_reseed_ids() { #[test] fn recipe_without_craft_runs_is_rejected() { let data = synthetic_1d(); - let mut dataset = NmrDataset::load(data.clone()); + let mut dataset = NmrDataset::load(data.clone()).unwrap(); dataset.craft_runs.push(sample_run(&data)); dataset.next_craft_run_id = 5; let recipe = RecipeObject { @@ -210,7 +210,7 @@ fn unavailable_craft_diagnostics_survive_project_roundtrip() { run.components[0].phase_std_rad = None; run.diagnostics.maximum_condition_number = None; run.diagnostics.modeling_windows[0].training_bic = None; - let mut dataset = NmrDataset::load(data); + let mut dataset = NmrDataset::load(data).unwrap(); dataset.craft_runs.push(run.clone()); dataset.reconcile_craft_fields(); let mut app = crate::state::PlotxApp::new(); @@ -250,9 +250,9 @@ fn only_stable_complete_runs_create_quantitative_reports() { #[test] fn report_status_tracks_stability_and_source_availability() { let data = synthetic_1d(); - let mut dataset = NmrDataset::load(data.clone()); + let mut dataset = NmrDataset::load(data.clone()).unwrap(); let mut run = sample_run(&data); - run.provenance.invocation.reference = dataset.craft_reference(); + run.provenance.invocation.reference = dataset.craft_reference().unwrap(); let definition = CraftReportDefinition::default(); let snapshot = run.amplitude_report(definition.clone()).unwrap(); let source = ReportSource { @@ -269,7 +269,7 @@ fn report_status_tracks_stability_and_source_availability() { source, definition: serde_json::to_value(definition).unwrap(), snapshot: serde_json::to_value(snapshot).unwrap(), - source_fingerprint: crate::state::craft_input_sha256(&data), + source_fingerprint: crate::state::craft_input_sha256(&data.clone().try_into().unwrap()), schema_version: 1, }); @@ -303,7 +303,7 @@ fn stability_snapshot_survives_project_roundtrip() { maximum: 0.502, relative_dispersion: 0.008, }); - let mut dataset = NmrDataset::load(data); + let mut dataset = NmrDataset::load(data).unwrap(); dataset.craft_runs.push(run.clone()); dataset.reconcile_craft_fields(); let mut app = crate::state::PlotxApp::new(); @@ -326,7 +326,7 @@ fn stability_snapshot_survives_project_roundtrip() { #[test] fn craft_component_table_link_and_board_visibility_survive_roundtrip() { let data = synthetic_1d(); - let mut dataset = NmrDataset::load(data.clone()); + let mut dataset = NmrDataset::load(data.clone()).unwrap(); dataset.craft_runs.push(sample_run(&data)); dataset.reconcile_craft_fields(); let mut app = crate::state::PlotxApp::new(); @@ -357,7 +357,7 @@ fn craft_component_table_link_and_board_visibility_survive_roundtrip() { #[test] fn craft_result_canvas_round_trips_binding_fields_and_linked_x_axis() { let data = synthetic_1d(); - let mut dataset = NmrDataset::load(data.clone()); + let mut dataset = NmrDataset::load(data.clone()).unwrap(); let dataset_id = dataset.resource_id; dataset.store_craft_run(sample_run(&data)); let mut app = crate::state::PlotxApp::new(); @@ -471,7 +471,7 @@ fn craft_group_field_uses_requested_reconstruction_duration() { run.provenance.invocation.derived_plan.reconstruction_points, requested_points ); - let mut dataset = NmrDataset::load(data); + let mut dataset = NmrDataset::load(data).unwrap(); dataset.store_craft_run(run); let dataset = Dataset::Nmr(Box::new(dataset)); let group_field = dataset diff --git a/crates/core/src/project/dto.rs b/crates/core/src/project/dto.rs index 11db4d88..42610435 100644 --- a/crates/core/src/project/dto.rs +++ b/crates/core/src/project/dto.rs @@ -247,28 +247,6 @@ pub enum StepSourceDto { Imported, } -/// Serialized in the 2D data extension so a save/load round-trip keeps a dataset -/// a `PseudoNmr` rather than degrading it to a plain 2D. -#[derive(Serialize, Deserialize, Clone)] -pub struct PseudoAxisDto { - pub name: String, - pub kind: String, - pub values: Vec, - pub unit: String, - pub source: String, -} - -/// Serialized alongside the pseudo axis so the Stejskal–Tanner b-factor survives -/// a round-trip. -#[derive(Serialize, Deserialize, Clone, Copy)] -pub struct DiffusionMetaDto { - pub gamma: f64, - pub delta: f64, - pub big_delta: f64, - pub tau: f64, - pub shape_factor: f64, -} - #[derive(Serialize, Deserialize)] #[serde(deny_unknown_fields)] pub struct ViewObject { diff --git a/crates/core/src/project/field_catalog.rs b/crates/core/src/project/field_catalog.rs index 307d7cf1..d74abac8 100644 --- a/crates/core/src/project/field_catalog.rs +++ b/crates/core/src/project/field_catalog.rs @@ -81,7 +81,7 @@ mod tests { source: "field validation".to_owned(), group_delay: 0.0, }; - let dataset = Dataset::Nmr(Box::new(crate::state::NmrDataset::load(source))); + let dataset = Dataset::Nmr(Box::new(crate::nmr_test_support::load_1d(source).unwrap())); let field = dataset.default_field_id().unwrap(); let error = validate_series( &dataset, diff --git a/crates/core/src/project/field_encoding_tests.rs b/crates/core/src/project/field_encoding_tests.rs index c62127f8..264116a1 100644 --- a/crates/core/src/project/field_encoding_tests.rs +++ b/crates/core/src/project/field_encoding_tests.rs @@ -1,6 +1,6 @@ use super::tests::{first_plot, synthetic_true_2d, temp_project}; use super::*; -use crate::state::{AfmDataset, CanvasDocument, Dataset, Nmr2DDataset, ObjectFrame, PlotxApp}; +use crate::state::{AfmDataset, CanvasDocument, Dataset, ObjectFrame, PlotxApp}; use plotx_figure::{HeatmapSpec, SeriesEncoding}; use std::collections::BTreeSet; use std::sync::Arc; @@ -8,11 +8,9 @@ use std::sync::Arc; #[test] fn project_roundtrip_preserves_concrete_contour_series_encoding() { let mut app = PlotxApp::new(); - app.doc - .datasets - .push(Dataset::Nmr2D(Box::new(Nmr2DDataset::load( - synthetic_true_2d(), - )))); + app.doc.datasets.push(Dataset::Nmr2D(Box::new( + crate::nmr_test_support::load_2d(synthetic_true_2d()).unwrap(), + ))); let mut canvas = CanvasDocument::new("contour".to_owned(), [120.0, 80.0]); let [width, height] = canvas.size_pt(); let object = app.build_plot_object( @@ -40,7 +38,9 @@ fn project_roundtrip_preserves_concrete_contour_series_encoding() { #[test] fn nmr_two_dimensional_fields_expose_real_and_magnitude_capabilities() { - let dataset = Dataset::Nmr2D(Box::new(Nmr2DDataset::load(synthetic_true_2d()))); + let dataset = Dataset::Nmr2D(Box::new( + crate::nmr_test_support::load_2d(synthetic_true_2d()).unwrap(), + )); let fields = dataset.field_descriptors(); assert_eq!(fields.len(), 2); assert_eq!(fields[0].local_id, "nmr.real"); diff --git a/crates/core/src/project/lineage_tests.rs b/crates/core/src/project/lineage_tests.rs index ba064ab7..e916b07e 100644 --- a/crates/core/src/project/lineage_tests.rs +++ b/crates/core/src/project/lineage_tests.rs @@ -9,18 +9,18 @@ use crate::state::{ #[test] fn project_roundtrip_maps_multi_source_lineage_by_data_id() { let mut app = PlotxApp::new(); - app.doc - .datasets - .push(Dataset::Nmr(Box::new(NmrDataset::load(synthetic_1d())))); - app.doc - .datasets - .push(Dataset::Nmr(Box::new(NmrDataset::load(synthetic_1d())))); + app.doc.datasets.push(Dataset::Nmr(Box::new( + NmrDataset::load(synthetic_1d()).unwrap(), + ))); + app.doc.datasets.push(Dataset::Nmr(Box::new( + NmrDataset::load(synthetic_1d()).unwrap(), + ))); let sources = [ app.doc.datasets[1].resource_id(), app.doc.datasets[0].resource_id(), app.doc.datasets[1].resource_id(), ]; - let mut derived = Dataset::Nmr(Box::new(NmrDataset::load(synthetic_1d()))); + let mut derived = Dataset::Nmr(Box::new(NmrDataset::load(synthetic_1d()).unwrap())); derived.set_lineage(Some(DatasetLineage::new( DerivationKind::SpectrumArithmetic, sources, @@ -48,9 +48,9 @@ fn project_roundtrip_maps_multi_source_lineage_by_data_id() { #[test] fn region_provenance_without_lineage_stays_unlinked() { let mut app = PlotxApp::new(); - app.doc - .datasets - .push(Dataset::Nmr(Box::new(NmrDataset::load(synthetic_1d())))); + app.doc.datasets.push(Dataset::Nmr(Box::new( + NmrDataset::load(synthetic_1d()).unwrap(), + ))); let source_resource = app.doc.datasets[0].resource_id().to_string(); let source_field = app.doc.datasets[0].default_field_id().unwrap(); let mut table = materialized_float_series_table( @@ -94,8 +94,8 @@ fn region_provenance_without_lineage_stays_unlinked() { fn lineage_resolution_rejects_missing_self_and_cycles() { let datasets = || { vec![ - Dataset::Nmr(Box::new(NmrDataset::load(synthetic_1d()))), - Dataset::Nmr(Box::new(NmrDataset::load(synthetic_1d()))), + Dataset::Nmr(Box::new(NmrDataset::load(synthetic_1d()).unwrap())), + Dataset::Nmr(Box::new(NmrDataset::load(synthetic_1d()).unwrap())), ] }; let binding = |data: &str, sources: &[&str]| DatasetBinding { @@ -134,9 +134,9 @@ fn v1_dataset_binding_without_derivation_deserializes() { #[test] fn v1_table_roundtrip_preserves_units_missing_uncertainty_and_lineage() { let mut app = PlotxApp::new(); - app.doc - .datasets - .push(Dataset::Nmr(Box::new(NmrDataset::load(synthetic_1d())))); + app.doc.datasets.push(Dataset::Nmr(Box::new( + NmrDataset::load(synthetic_1d()).unwrap(), + ))); let mut table = materialized_float_series_table( ("Time".into(), "s".into(), vec![Some(0.0), Some(1.0)]), vec![FloatSeries { diff --git a/crates/core/src/project/linefit_tests.rs b/crates/core/src/project/linefit_tests.rs index b9ea3333..954129ac 100644 --- a/crates/core/src/project/linefit_tests.rs +++ b/crates/core/src/project/linefit_tests.rs @@ -114,7 +114,7 @@ fn table_statistics_survive_project_roundtrip() { #[test] fn recipe_without_line_fits_key_is_rejected() { - let mut dataset = NmrDataset::load(synthetic_1d()); + let mut dataset = NmrDataset::load(synthetic_1d()).unwrap(); dataset.line_fits.push(sample_line_fit()); dataset.next_line_fit_id = 8; let recipe = RecipeObject { diff --git a/crates/core/src/project/mod.rs b/crates/core/src/project/mod.rs index ffe0b3e1..de402b5b 100644 --- a/crates/core/src/project/mod.rs +++ b/crates/core/src/project/mod.rs @@ -5,8 +5,10 @@ use crate::state::{ ObjectFrame, ObjectId, PlotObject, PlotxApp, PrimaryView, SeriesBinding, ShapeKind, ShapeObject, StackMode, StackSpec, TextAlign, TextBox, Tool, }; +#[cfg(test)] use num_complex::Complex64; use plotx_figure::Color; +#[cfg(test)] use plotx_io::{ AxisSource, DiffusionMeta, Dim, Domain, NmrData, NmrData2D, PseudoAxis, PseudoKind, QuadMode, }; @@ -29,7 +31,6 @@ mod asset_codec; mod axis_overrides; mod codec; mod convert; -mod convert_dimensions; mod convert_recipes; mod convert_views; mod dosy_convert; @@ -39,6 +40,7 @@ mod field_catalog; mod integrals2d; mod lineage_convert; mod mass_spec_convert; +mod nmr_snapshot; mod peaks2d; mod persistence; mod pipeline_conv; @@ -50,7 +52,6 @@ mod xrd_convert; pub use codec::*; pub use convert::*; -pub use convert_dimensions::*; pub use convert_recipes::*; pub use convert_views::*; pub use dto::*; @@ -66,7 +67,6 @@ pub use typed_table::*; const FORMAT: &str = "plotx-project"; const SCHEMA_VERSION: u32 = 1; -const STORAGE_COMPLEX_F64_LE: &str = "complex_f64_le"; const STORAGE_TABLE_V1: &str = "plotx_table_envelope_v1"; const STORAGE_AFM_V1: &str = "plotx_afm_v1"; const STORAGE_DOSY_V1: &str = "plotx_dosy_v1"; @@ -679,6 +679,8 @@ mod linefit_tests; #[cfg(test)] mod multiplet_tests; #[cfg(test)] +mod nmr_snapshot_tests; +#[cfg(test)] mod panel_schema_tests; #[cfg(test)] mod pipeline_domain_tests; diff --git a/crates/core/src/project/multiplet_tests.rs b/crates/core/src/project/multiplet_tests.rs index b8026a87..33b921a8 100644 --- a/crates/core/src/project/multiplet_tests.rs +++ b/crates/core/src/project/multiplet_tests.rs @@ -26,7 +26,7 @@ pub(super) fn sample_multiplet() -> StoredMultiplet { #[test] fn multiplets_survive_project_roundtrip() { - let mut dataset = NmrDataset::load(synthetic_1d()); + let mut dataset = NmrDataset::load(synthetic_1d()).unwrap(); dataset.multiplets.push(sample_multiplet()); dataset.next_multiplet_id = 4; let mut app = crate::state::PlotxApp::new(); @@ -45,7 +45,7 @@ fn multiplets_survive_project_roundtrip() { #[test] fn recipe_without_multiplets_key_is_rejected() { - let mut dataset = NmrDataset::load(synthetic_1d()); + let mut dataset = NmrDataset::load(synthetic_1d()).unwrap(); dataset.multiplets.push(sample_multiplet()); dataset.next_multiplet_id = 4; let recipe = RecipeObject { diff --git a/crates/core/src/project/nmr_snapshot.rs b/crates/core/src/project/nmr_snapshot.rs new file mode 100644 index 00000000..9b88a923 --- /dev/null +++ b/crates/core/src/project/nmr_snapshot.rs @@ -0,0 +1,131 @@ +//! The NMR snapshot is the sole persisted scientific payload. + +use super::*; +use plotx_io::nmr_view::NmrSource; + +pub(super) const STORAGE: &str = "nmr_snapshot_v1"; + +/// Evidence of the last completed output, which may precede a queued recipe +/// edit. It is never used as a sample cache or replayed in place of the recipe. +pub(super) fn execution_evidence( + source: &NmrSource, + phases: &[plotx_processing::nmr_bridge::PhaseReport], +) -> Result { + let Some(processed) = source.dataset().as_processed() else { + return Ok(serde_json::Value::Null); + }; + let mut report = Vec::new(); + nmr::execution_report::write_json( + processed, + &[], + &mut report, + ProjectLoadLimits::default().max_metadata_bytes as usize, + ) + .map_err(|error| ProjectError::Invalid(format!("NMR execution evidence: {error}")))?; + let report: serde_json::Value = serde_json::from_slice(&report)?; + let digest = |value: nmr::provenance::CanonicalDatasetDigests| { + value + .dataset() + .as_bytes() + .iter() + .map(|byte| format!("{byte:02x}")) + .collect::() + }; + let phases: Vec<_> = phases.iter().map(|phase| serde_json::json!({ + "step_id": phase.step.get(), "axis": phase.axis, "points": phase.points, + "display_pivot": phase.display_pivot, "method": format!("{:?}", phase.method), + "algorithm": phase.method.algorithm_version(), "input": digest(phase.input), + "correction": { "p0_degrees": phase.correction.p0_degrees(), "p1_degrees": phase.correction.p1_degrees(), + "pivot_fraction": phase.correction.pivot_fraction(), "convention": "exp(+i phase), i/N" }, + "objective": phase.objective, "evaluations": phase.evaluations, + "representative": phase.representative.as_ref().map(|trace| serde_json::json!({ + "policy": "strongest-cartesian-component.v1", "removed_axis": trace.removed_axis, + "logical_index": trace.index, "component": trace.component, "input": digest(trace.input) + })) + })).collect(); + Ok(serde_json::json!({ "library": report, "automatic_phase": phases })) +} + +fn limits() -> nmr::snapshot::SnapshotLimits { + let project = ProjectLoadLimits::default(); + nmr::snapshot::SnapshotLimits { + max_bytes: project.max_entry_bytes, + max_sample_bytes: project.max_materialized_bytes as usize, + max_metadata_bytes: project.max_metadata_bytes as usize, + max_working_bytes: (project.max_materialized_bytes + project.max_metadata_bytes) as usize, + ..Default::default() + } +} + +pub(super) fn write(writer: &mut impl Write, source: &nmr::Dataset) -> Result<()> { + plotx_io::nmr_bridge::snapshot::write( + source, + writer, + limits(), + &mut nmr::ExecutionContext::default(), + ) + .map_err(|error| ProjectError::Invalid(format!("NMR snapshot: {error}"))) +} + +pub(super) fn read(zip: &mut ZipArchive, data: &DataObject) -> Result { + if data.payload.storage != STORAGE { + return Err(ProjectError::Unsupported(format!( + "NMR payload storage {}", + data.payload.storage + ))); + } + if !data.dimensions.is_empty() || data.payload.domain != "nmr" { + return Err(ProjectError::Invalid( + "NMR dimensions and calibration belong to the snapshot".into(), + )); + } + let source = read_entry( + zip, + &data.payload.blob, + "NMR snapshot", + limits().max_bytes, + |reader| { + plotx_io::nmr_bridge::snapshot::read( + reader, + limits(), + &mut nmr::ExecutionContext::default(), + ) + .map_err(|error| ProjectError::Invalid(format!("NMR snapshot: {error}"))) + }, + )?; + let shape = plotx_io::nmr_bridge::shape(&source) + .map_err(|error| ProjectError::Invalid(error.to_string()))?; + if shape != data.payload.shape { + return Err(ProjectError::Invalid( + "NMR snapshot shape differs from the object index".into(), + )); + } + let identity = read_acquisition_identity(data)?; + NmrSource::new(source) + .map(|source| source.with_display_label(identity.source_label)) + .map_err(|error| ProjectError::Invalid(error.to_string())) +} + +pub(super) fn read_1d( + zip: &mut ZipArchive, + data: &DataObject, + recipe: &RecipeObject, +) -> Result { + let source = read(zip, data)?; + let mut dataset = NmrDataset::load_with_pipeline( + source, + Some(AxisPipeline { steps: Vec::new() }), + Some(false), + ) + .map_err(ProjectError::Invalid)?; + dataset.acquisition_identity = read_acquisition_identity(data)?; + dataset.field_catalog = super::field_catalog::read(data)?; + apply_1d_recipe(&mut dataset, recipe)?; + dataset.name = data.label.clone(); + dataset.retransform().map_err(ProjectError::Invalid)?; + let dataset = Dataset::Nmr(Box::new(dataset)); + dataset + .validate_field_catalog() + .map_err(ProjectError::Invalid)?; + Ok(dataset) +} diff --git a/crates/core/src/project/nmr_snapshot_tests.rs b/crates/core/src/project/nmr_snapshot_tests.rs new file mode 100644 index 00000000..08bbd022 --- /dev/null +++ b/crates/core/src/project/nmr_snapshot_tests.rs @@ -0,0 +1,415 @@ +use super::*; +use crate::state::NmrImportDraft; +use plotx_io::nmr_view::NmrSource; +use std::sync::Arc; + +fn path(suffix: &str) -> PathBuf { + std::env::temp_dir().join(format!("plotx-nmr-{}-{suffix}", uuid::Uuid::new_v4())) +} + +#[test] +fn user_sampling_import_errors_are_visible_and_declarations_reopen_without_vendor_files() { + let dir = path("sampling-input"); + std::fs::create_dir(&dir).unwrap(); + let fixture = + PathBuf::from(env!("CARGO_MANIFEST_DIR")).join("../io/tests/fixtures/nmr/bruker-nus"); + for name in ["ser", "acqus", "acqu2s"] { + std::fs::copy(fixture.join(name), dir.join(name)).unwrap(); + } + let mut draft = NmrImportDraft::new(dir.clone()); + draft.grid = "4".into(); + draft.lanes = "2".into(); + draft.source = "user supplied synthetic table".into(); + draft.rows = "2\n2".into(); + assert!(draft.declaration().is_err()); + draft.one_based = Some(true); + let declaration = draft.declaration().unwrap(); + let mut invalid = declaration.clone(); + invalid.grid_shape = vec![5]; + let mut app = PlotxApp::new(); + assert!(!app.load_nmr_with_sampling(&dir, invalid)); + assert!(app.doc.datasets.is_empty()); + assert!(app.session.status.contains("Failed to load")); + assert!( + app.load_nmr_with_sampling(&dir, declaration.clone()), + "{}", + app.session.status + ); + let nmr = app.doc.datasets[0].as_nmr2d().unwrap(); + assert_eq!(nmr.data.nus.as_ref().unwrap().schedule, [1, 1]); + let field = nmr.field_catalog.id_for_key("nmr.observations").unwrap(); + let items = nmr + .field_catalog + .trace_collection(field) + .unwrap() + .items + .clone(); + assert_ne!(items[0].id, items[1].id); + let project = path("declared-nus.plotx"); + save_project(&app, &project, false).unwrap(); + for name in ["ser", "acqus", "acqu2s"] { + std::fs::remove_file(dir.join(name)).unwrap(); + } + std::fs::remove_dir(dir).unwrap(); + let restored = load_project(&project).unwrap(); + std::fs::remove_file(project).unwrap(); + let nmr = restored.doc.datasets[0].as_nmr2d().unwrap(); + let source = nmr.data.source_dataset().dataset().as_raw().unwrap(); + assert_eq!( + source.sampling_schedule().unwrap().declaration(), + Some(&declaration.into_native().unwrap()) + ); + assert_eq!( + nmr.field_catalog.trace_collection(field).unwrap().items, + items + ); +} + +fn rewrite(path: &Path, mut change: impl FnMut(&str, &mut Vec)) { + let mut archive = ZipArchive::new(File::open(path).unwrap()).unwrap(); + let entries: Vec<_> = (0..archive.len()) + .map(|index| { + let mut entry = archive.by_index(index).unwrap(); + let name = entry.name().to_owned(); + let mut bytes = Vec::new(); + entry.read_to_end(&mut bytes).unwrap(); + change(&name, &mut bytes); + (name, bytes) + }) + .collect(); + drop(archive); + let mut archive = zip::ZipWriter::new(File::create(path).unwrap()); + for (name, bytes) in entries { + write_bytes(&mut archive, SimpleFileOptions::default(), &name, &bytes).unwrap(); + } + archive.finish().unwrap(); +} + +#[test] +fn imported_hertz_spectrum_reopens_without_vendor_files_or_invented_calibration() { + let vendor = path("spectrum.dx"); + std::fs::write( + &vendor, + include_str!("../../../io/tests/fixtures/nmr/jcamp-hz.dx") + .lines() + .filter(|line| !line.starts_with("##.OBSERVE FREQUENCY")) + .collect::>() + .join("\n"), + ) + .unwrap(); + let loaded = plotx_io::load_path(&vendor).unwrap(); + let plotx_io::Acquisition::Nmr(source) = loaded.acquisition else { + panic!("NMR import"); + }; + let original = source.dataset().canonical_digests(); + let mut app = PlotxApp::new(); + app.doc + .datasets + .push(Dataset::Nmr(Box::new(NmrDataset::load(source).unwrap()))); + let project = path("offline.plotx"); + save_project(&app, &project, false).unwrap(); + std::fs::remove_file(vendor).unwrap(); + let restored = load_project(&project).unwrap(); + std::fs::remove_file(project).unwrap(); + let nmr = restored.doc.datasets[0].as_nmr().unwrap(); + assert_eq!(original, nmr.data.dataset().canonical_digests()); + assert_eq!(nmr.spectrum().unwrap().ppm, [4.0, 3.0, 2.0, 1.0]); + assert_eq!(nmr.spectrum().unwrap().unit, nmr::axis::AxisUnit::Hertz); + assert_eq!(nmr.data.axes()[0].observe_frequency_mhz(), None); + assert!(!nmr.data.has_imaginary(0)); + assert_eq!( + restored.doc.datasets[0].field_descriptors()[0].units, + ["Hz"] + ); + let figure = crate::figures::build_figure(&nmr.data, nmr.spectrum().unwrap(), &[]); + assert!(figure.x.label.contains("Hz")); + assert!( + restored + .analyze_multiplets(0, 1.0, 4.0) + .unwrap_err() + .contains("calibrated in ppm") + ); +} + +#[test] +fn project_rejects_corrupt_trailing_old_storage_and_conflicting_shape() { + let mut app = PlotxApp::new(); + app.doc.datasets.push(Dataset::Nmr(Box::new( + NmrDataset::load(super::tests::synthetic_1d()).unwrap(), + ))); + for damage in ["sample", "trailing", "storage", "shape"] { + let project = path("corrupt.plotx"); + save_project(&app, &project, false).unwrap(); + rewrite(&project, |name, bytes| { + if name.ends_with("/data.bin") { + if damage == "sample" { + let last = bytes.len() - 1; + bytes[last] ^= 1; + } + if damage == "trailing" { + bytes.push(0); + } + } else if name.ends_with("/object.json") { + let mut value: serde_json::Value = serde_json::from_slice(bytes).unwrap(); + if value.get("payload").is_some() { + assert_eq!(value["payload"]["storage"], "nmr_snapshot_v1"); + assert_eq!(value["dimensions"], serde_json::json!([])); + if damage == "storage" { + value["payload"]["storage"] = "complex_f64_le".into(); + } + if damage == "shape" { + value["payload"]["shape"] = serde_json::json!([3]); + } + *bytes = serde_json::to_vec(&value).unwrap(); + } + } + }); + let error = load_project(&project) + .err() + .expect("untrusted payload must fail") + .to_string(); + std::fs::remove_file(project).unwrap(); + assert!( + error.contains("NMR") || error.contains("trailing"), + "{damage}: {error}" + ); + } +} + +#[test] +fn column_derivation_and_snapshot_preserve_indirect_cartesian_components() { + use nmr::axis::{AxisCoordinates, AxisDomain, AxisRole, AxisUnit}; + use nmr::processed::{ + ComponentBasis, ProcessedAxis, ProcessedData, ProcessedDataset, ProcessedDescriptor, + ProcessedOrigin, ProcessedProvenance, + }; + let axes = [2, 3] + .into_iter() + .map(|points| { + ProcessedAxis::new( + AxisRole::Signal, + AxisDomain::Frequency, + Some(AxisUnit::Hertz), + points, + AxisCoordinates::Uniform { + start: 0.0, + step: 1.0, + }, + ComponentBasis::Cartesian, + ) + .unwrap() + }) + .collect(); + let descriptor = ProcessedDescriptor::new(axes).unwrap(); + let data = + ProcessedData::from_descriptor(&descriptor, (1..=24).map(f64::from).collect()).unwrap(); + let native = ProcessedDataset::new( + descriptor, + data, + ProcessedProvenance::new(ProcessedOrigin::Unknown, vec![]).unwrap(), + ) + .unwrap(); + let expected: Vec<_> = (0..2) + .map(|row| { + Complex64::new( + native.data().get(&[row, 1], &[0, 0]).unwrap(), + native.data().get(&[row, 1], &[1, 0]).unwrap(), + ) + }) + .collect(); + let source = NmrSource::new(Arc::new(native.into())).unwrap(); + let (column, view) = plotx_processing::slice::extract( + &source, + plotx_processing::SliceKind::Column, + plotx_processing::slice::Reduction::Slice(1), + ) + .unwrap(); + assert_eq!(view.values, expected); + let mut app = PlotxApp::new(); + app.doc.datasets.push(Dataset::Nmr(Box::new( + NmrDataset::load_with_pipeline(column, Some(AxisPipeline { steps: vec![] }), Some(false)) + .unwrap(), + ))); + let project = path("column.plotx"); + save_project(&app, &project, false).unwrap(); + let restored = load_project(&project).unwrap(); + std::fs::remove_file(project).unwrap(); + let nmr = restored.doc.datasets[0].as_nmr().unwrap(); + assert_eq!(nmr.data.trace().unwrap(), expected); + assert!(nmr.data.has_imaginary(0)); + assert!( + nmr.data + .dataset() + .as_processed() + .unwrap() + .provenance() + .history() + .is_some() + ); + assert_eq!(nmr.native_processed.reference_frequency_mhz(0), None); +} + +#[test] +fn nus_observation_identities_keep_duplicate_schedule_order_across_project_roundtrip() { + let mut data = super::tests::synthetic_dosy_2d(); + data.pseudo_axis = None; + data.diffusion = None; + data.experiment = None; + data.rows = 3; + data.data.truncate(3 * data.cols); + data.nus = Some(plotx_io::NusMeta { + grid: 8, + acquired: 3, + schedule: Some(vec![5, 1, 5]), + }); + let nmr = Nmr2DDataset::load(data).unwrap(); + let field = nmr.field_catalog.id_for_key("nmr.observations").unwrap(); + let items = nmr + .field_catalog + .trace_collection(field) + .unwrap() + .items + .clone(); + assert_eq!(items.len(), 3); + assert_ne!(items[0].id, items[2].id); + let mut app = PlotxApp::new(); + app.doc.datasets.push(Dataset::Nmr2D(Box::new(nmr))); + assert_eq!(app.doc.datasets[0].default_field_id(), Some(field)); + let project = path("nus.plotx"); + save_project(&app, &project, false).unwrap(); + let restored = load_project(&project).unwrap(); + std::fs::remove_file(project).unwrap(); + let nmr = restored.doc.datasets[0].as_nmr2d().unwrap(); + assert_eq!(nmr.data.nus.as_ref().unwrap().schedule, [5, 1, 5]); + assert_eq!( + nmr.field_catalog.trace_collection(field).unwrap().items, + items + ); + assert!(nmr.nus_request.is_none()); +} + +#[test] +fn completed_execution_evidence_records_actual_method_and_input() { + let nmr = NmrDataset::load(super::tests::synthetic_1d()).unwrap(); + let dataset = Dataset::Nmr(Box::new(nmr)); + let objects = dataset_to_objects(&dataset, "d0", "r0").unwrap(); + let evidence = &objects.data.extensions["plotx.nmr_execution"]; + assert!(evidence["library"].is_object()); + let phases = evidence["automatic_phase"].as_array().unwrap(); + assert_eq!(phases.len(), 1); + assert!(phases[0]["algorithm"].as_str().unwrap().contains("entropy")); + assert_eq!(phases[0]["input"].as_str().unwrap().len(), 64); + assert!(phases[0]["evaluations"].as_u64().unwrap() > 0); +} + +#[test] +fn nus_reconstruction_changes_live_bindings_and_reopens_with_grid_identities() { + use plotx_processing::{Layout2D, ProcessingStep, StepKind, StepSource}; + let mut data = super::tests::synthetic_dosy_2d(); + data.pseudo_axis = None; + data.diffusion = None; + data.experiment = None; + data.rows = 6; + data.cols = 8; + data.quad = plotx_io::QuadMode::States; + data.data = [5, 1, 6] + .into_iter() + .flat_map(|row| { + (0..2).flat_map(move |lane| { + (0..8).map(move |col| { + let phase = std::f64::consts::TAU * row as f64 / 8.0; + let amplitude = if lane == 0 { phase.cos() } else { phase.sin() }; + Complex64::from_polar(amplitude, std::f64::consts::TAU * col as f64 / 8.0) + }) + }) + }) + .collect(); + data.nus = Some(plotx_io::NusMeta { + grid: 8, + acquired: 3, + schedule: Some(vec![5, 1, 6]), + }); + let mut app = PlotxApp::new(); + app.doc + .datasets + .push(Dataset::Nmr2D(Box::new(Nmr2DDataset::load(data).unwrap()))); + let canvas = crate::workflow::build_default_canvas(&app.doc.datasets[0], "NUS"); + app.doc.canvases.push(canvas); + let observations = app.doc.canvases[0].objects[0] + .plot() + .unwrap() + .binding + .series[0] + .source; + for layout in [Layout2D::Stack, Layout2D::Ft, Layout2D::Stack] { + let nmr = app.doc.datasets[0].as_nmr2d_mut().unwrap(); + nmr.params.layout = layout; + nmr.params.f2.steps = vec![ProcessingStep::new( + nmr.allocate_step_id(), + StepKind::Fft, + StepSource::User, + )]; + nmr.params.f1.steps = if layout == Layout2D::Ft { + vec![ProcessingStep::new( + nmr.allocate_step_id(), + StepKind::Fft, + StepSource::User, + )] + } else { + vec![] + }; + nmr.nus_request = Some(plotx_processing::nmr_execution::NusRequest { + max_iterations: 1000, + noise_standard_deviation: Some(0.0), + }); + assert!(app.schedule_2d_processing(0, true)); + let deadline = std::time::Instant::now() + std::time::Duration::from_secs(10); + while app.compute_busy() && std::time::Instant::now() < deadline { + app.poll_compute(); + std::thread::sleep(std::time::Duration::from_millis(5)); + } + app.poll_compute(); + assert!(!app.compute_busy()); + assert!( + !app.session.status.contains("failed"), + "{}", + app.session.status + ); + let dataset = &app.doc.datasets[0]; + let field = dataset.default_field_id().unwrap(); + assert_ne!(field, observations.field); + let plot = app.doc.canvases[0].objects[0].plot().unwrap(); + let binding = app.display_binding(plot.display_owner, &plot.binding); + assert!(!binding.series.is_empty()); + assert!( + binding + .series + .iter() + .all(|series| series.source.field == field) + ); + if layout == Layout2D::Stack { + assert_eq!(binding.series.len(), 8); + assert!( + binding + .series + .iter() + .all(|series| series.source.item != observations.item) + ); + } + let project = path("reconstructed.plotx"); + save_project(&app, &project, false).unwrap(); + let restored = load_project(&project).unwrap(); + std::fs::remove_file(project).unwrap(); + assert_eq!( + restored.doc.canvases[0].objects[0].plot().unwrap().binding, + plot.binding + ); + let merged = app.merge_display_binding(plot.display_owner, &plot.binding, binding); + assert!( + merged + .series + .iter() + .any(|series| series.source == observations) + ); + } +} diff --git a/crates/core/src/project/pipeline_conv.rs b/crates/core/src/project/pipeline_conv.rs index 2b4cdb7f..86e74359 100644 --- a/crates/core/src/project/pipeline_conv.rs +++ b/crates/core/src/project/pipeline_conv.rs @@ -18,22 +18,37 @@ pub fn pipeline_from_dto(dto: &AxisPipelineDto) -> AxisPipeline { /// will receive. The error names the stored value and the data-derived bound so /// a malformed project or scheme never opens into a silently rewritten state. pub fn validate_1d_pipeline( - data: &plotx_io::NmrData, + data: &plotx_io::nmr_view::NmrSource, pipeline: &AxisPipeline, group_delay_correct: bool, ) -> std::result::Result<(), String> { - let output = pipeline - .output_domain(data.domain) - .map_err(|error| error.to_string())?; - if output == plotx_io::Domain::Time { - return Ok(()); - } - let mut spectrum = plotx_processing::transform_base(data, pipeline, group_delay_correct); - for step in pipeline - .steps - .iter() - .skip_while(|step| step.kind.at_or_before_fft()) - { + use plotx_processing::nmr_bridge::{DelayPolicy, RecipeRange}; + for (index, step) in pipeline.steps.iter().enumerate() { + if !matches!( + step.kind, + StepKind::Smooth(_) | StepKind::Normalize(_) | StepKind::Bin(_) + ) { + continue; + } + let prefix = AxisPipeline { + steps: pipeline.steps[..index].to_vec(), + }; + let output = plotx_processing::nmr_execution::execute_1d( + data, + &prefix, + if group_delay_correct { + DelayPolicy::AxisEvidence + } else { + DelayPolicy::Disabled + }, + RecipeRange::All, + &mut nmr::ExecutionContext::default(), + ) + .map_err(|e| e.to_string())?; + let spectrum = output + .view + .as_frequency() + .ok_or_else(|| "Cleanup requires frequency-domain data".to_owned())?; match step.kind { StepKind::Smooth(method) => { let capped = spectrum.values.len().min(201); @@ -72,7 +87,7 @@ pub fn validate_1d_pipeline( )); } StepKind::Bin(params) => { - let minimum = 1.5 * plotx_processing::cleanup::axis_step(&spectrum.ppm); + let minimum = 1.5 * spectrum.coordinate_spacing().unwrap_or(0.0); if !params.width.is_finite() || params.width <= minimum { return Err(format!( "stored bin width {} is out of range: it must be greater than {minimum} for this axis", @@ -82,11 +97,20 @@ pub fn validate_1d_pipeline( } _ => {} } - if step.enabled { - plotx_processing::apply_freq_step(&mut spectrum, &step.kind); - } } - Ok(()) + plotx_processing::nmr_execution::execute_1d( + data, + pipeline, + if group_delay_correct { + DelayPolicy::AxisEvidence + } else { + DelayPolicy::Disabled + }, + RecipeRange::All, + &mut nmr::ExecutionContext::default(), + ) + .map(|_| ()) + .map_err(|error| error.to_string()) } /// Drop step identities from a pipeline destined for a detached recipe diff --git a/crates/core/src/project/pipeline_domain_tests.rs b/crates/core/src/project/pipeline_domain_tests.rs index 30d8b017..994f4acd 100644 --- a/crates/core/src/project/pipeline_domain_tests.rs +++ b/crates/core/src/project/pipeline_domain_tests.rs @@ -15,9 +15,9 @@ fn invalid_stack_pipelines() -> Vec { #[test] fn stack_scheme_rejects_an_invalid_dormant_f1_pipeline() { - let target = Dataset::Nmr2D(Box::new(Nmr2DDataset::load( - super::tests::synthetic_dosy_2d(), - ))); + let target = Dataset::Nmr2D(Box::new( + Nmr2DDataset::load(super::tests::synthetic_dosy_2d()).unwrap(), + )); let scheme = ProcessingScheme { schema_version: 1, dimension_count: 2, @@ -31,7 +31,7 @@ fn stack_scheme_rejects_an_invalid_dormant_f1_pipeline() { #[test] fn project_recipe_rejects_an_invalid_2d_pipeline_before_retransform() { - let mut dataset = Nmr2DDataset::load(super::tests::synthetic_dosy_2d()); + let mut dataset = Nmr2DDataset::load(super::tests::synthetic_dosy_2d()).unwrap(); let recipe = RecipeObject { id: "recipe_000000".to_owned(), role: "recipe".to_owned(), diff --git a/crates/core/src/project/pseudo_tests.rs b/crates/core/src/project/pseudo_tests.rs index 86b64068..3bc61851 100644 --- a/crates/core/src/project/pseudo_tests.rs +++ b/crates/core/src/project/pseudo_tests.rs @@ -150,7 +150,7 @@ fn rewrite_project(path: &Path, mut edit: impl FnMut(&str, &mut Vec) -> bool fn pseudo_project_with_view(name: &str) -> PathBuf { let mut app = PlotxApp::new(); - let dataset = Dataset::Nmr2D(Box::new(Nmr2DDataset::load(synthetic_dosy_2d()))); + let dataset = Dataset::Nmr2D(Box::new(Nmr2DDataset::load(synthetic_dosy_2d()).unwrap())); let canvas = crate::workflow::build_default_canvas(&dataset, "strict-pseudo"); app.doc.datasets.push(dataset); app.doc.canvases.push(canvas); @@ -217,7 +217,7 @@ fn assert_f64_bits_equal(actual: &[f64], expected: &[f64]) { #[test] fn project_load_ignores_stored_pseudo_fit_curve() { let mut app = PlotxApp::new(); - let ds = Nmr2DDataset::load(synthetic_dosy_2d()); + let ds = Nmr2DDataset::load(synthetic_dosy_2d()).unwrap(); app.doc.datasets.push(Dataset::Nmr2D(Box::new(ds))); let path = temp_project("pseudo-fit-curve"); @@ -240,7 +240,7 @@ fn project_load_ignores_stored_pseudo_fit_curve() { #[test] fn project_round_trip_restores_both_real_dosy_maps_after_retransform() { let mut app = PlotxApp::new(); - let mut ds = Nmr2DDataset::load(synthetic_dosy_2d()); + let mut ds = Nmr2DDataset::load(synthetic_dosy_2d()).unwrap(); assert!(ds.build_dosy_map(), "the real per-column fit must populate"); let original_dosy = ds.dosy_map.clone().unwrap(); let params = crate::IltParams { @@ -365,7 +365,7 @@ fn project_round_trip_restores_both_real_dosy_maps_after_retransform() { #[test] fn mismatched_fingerprint_keeps_the_stored_map_and_reports_both_fingerprints() { let mut app = PlotxApp::new(); - let mut ds = Nmr2DDataset::load(synthetic_dosy_2d()); + let mut ds = Nmr2DDataset::load(synthetic_dosy_2d()).unwrap(); assert!(ds.build_dosy_map()); let original = ds.dosy_map.clone().unwrap(); app.doc.datasets.push(Dataset::Nmr2D(Box::new(ds))); @@ -374,8 +374,14 @@ fn mismatched_fingerprint_keeps_the_stored_map_and_reports_both_fingerprints() { let _ = std::fs::remove_file(&path); save_project(&app, &path, false).unwrap(); rewrite_project(&path, |name, bytes| { - if name.ends_with("/data.bin") { - bytes[0] ^= 1; + if name.ends_with(".json") { + let mut value: serde_json::Value = serde_json::from_slice(bytes).unwrap(); + if let Some(fingerprint) = + value.pointer_mut("/extensions/plotx.dosy/provenance/diffusion/data_fingerprint") + { + *fingerprint = serde_json::Value::String("0".repeat(64)); + *bytes = serde_json::to_vec(&value).unwrap(); + } } true }); @@ -424,7 +430,7 @@ fn mismatched_fingerprint_keeps_the_stored_map_and_reports_both_fingerprints() { #[test] fn missing_selected_blob_explains_the_stack_fallback() { let mut app = PlotxApp::new(); - let mut ds = Nmr2DDataset::load(synthetic_dosy_2d()); + let mut ds = Nmr2DDataset::load(synthetic_dosy_2d()).unwrap(); assert!(ds.build_dosy_map()); app.doc.datasets.push(Dataset::Nmr2D(Box::new(ds))); @@ -491,7 +497,7 @@ fn project_json_numbers_survive_a_round_trip_bit_for_bit() { #[test] fn a_snapshot_is_not_replayed_when_the_stored_map_could_not_be_restored() { let mut app = PlotxApp::new(); - let mut ds = Nmr2DDataset::load(synthetic_dosy_2d()); + let mut ds = Nmr2DDataset::load(synthetic_dosy_2d()).unwrap(); assert!(ds.build_dosy_map()); let action = crate::actions::Action::insert_dataset_with_default_canvas( &app, @@ -552,7 +558,7 @@ fn a_snapshot_is_not_replayed_when_the_stored_map_could_not_be_restored() { #[test] fn the_missing_map_complaint_does_not_survive_selecting_a_method_that_has_one() { let mut app = PlotxApp::new(); - let mut ds = Nmr2DDataset::load(synthetic_dosy_2d()); + let mut ds = Nmr2DDataset::load(synthetic_dosy_2d()).unwrap(); let params = IltParams { lambda: 0.02, d_min: 1e-11, diff --git a/crates/core/src/project/scheme.rs b/crates/core/src/project/scheme.rs index e44cf84c..50533ac2 100644 --- a/crates/core/src/project/scheme.rs +++ b/crates/core/src/project/scheme.rs @@ -201,9 +201,9 @@ pub fn apply_scheme( .first() .ok_or_else(|| incompatible("scheme carries no pipeline"))?; let mut pipeline = pipeline_from_dto(dto); + remint_pipeline(&mut pipeline, &mut dataset_next_step_id(dataset)); validate_1d_pipeline(&n.data, &pipeline, scheme.group_delay_correct) .map_err(ProjectError::Invalid)?; - remint_pipeline(&mut pipeline, &mut dataset_next_step_id(dataset)); Ok(DatasetProcessingState::Nmr { pipeline, group_delay_correct: scheme.group_delay_correct, @@ -231,18 +231,13 @@ pub fn apply_scheme( f2: pipeline_from_dto(f2), f1: pipeline_from_dto(f1), }; - params - .f2 - .output_domain(n.data.domain) - .map_err(|error| incompatible(&error.to_string()))?; - params - .f1 - .output_domain(n.data.domain) - .map_err(|error| incompatible(&error.to_string()))?; + plotx_processing::nmr_execution::validate_2d_domains(&n.data, ¶ms) + .map_err(|error| incompatible(&error))?; let mut next = dataset_next_step_id(dataset); remint_pipeline(&mut params.f2, &mut next); remint_pipeline(&mut params.f1, &mut next); Ok(DatasetProcessingState::Nmr2D { + nus_request: n.nus_request, params, preset: n.preset, group_delay_correct: scheme.group_delay_correct, @@ -287,13 +282,14 @@ fn remint_pipeline(pipeline: &mut AxisPipeline, next: &mut u64) { pub fn reset_processing(dataset: &Dataset) -> Option { let mut state = match dataset { Dataset::Nmr(n) => Some(DatasetProcessingState::Nmr { - pipeline: AxisPipeline::default_1d(), - group_delay_correct: crate::state::default_group_delay_correct(n.data.domain), + pipeline: crate::state::default_nmr_pipeline(&n.data), + group_delay_correct: crate::state::default_group_delay_correct(&n.data), }), Dataset::Nmr2D(n) => Some(DatasetProcessingState::Nmr2D { - params: Params2D::default_for(n.preset), + nus_request: n.nus_request, + params: crate::state::default_nmr_params(&n.data, n.preset), preset: n.preset, - group_delay_correct: crate::state::default_group_delay_correct(n.data.domain), + group_delay_correct: crate::state::default_group_delay_correct(n.data.source_dataset()), }), Dataset::Table(_) => None, Dataset::Electrophysiology(_) => None, diff --git a/crates/core/src/project/step_identity_tests.rs b/crates/core/src/project/step_identity_tests.rs index 0623561b..68fa078f 100644 --- a/crates/core/src/project/step_identity_tests.rs +++ b/crates/core/src/project/step_identity_tests.rs @@ -38,7 +38,7 @@ fn step_ids_and_allocator_survive_project_roundtrip() { #[test] fn project_roundtrip_preserves_custom_pipeline_steps() { let mut app = PlotxApp::new(); - let mut dataset = NmrDataset::load(synthetic_1d()); + let mut dataset = NmrDataset::load(synthetic_1d()).unwrap(); // One time-side step (an exponential window before the FFT) and one // frequency-side step (a referencing shift) that must both survive. let fft_pos = dataset @@ -65,7 +65,7 @@ fn project_roundtrip_preserves_custom_pipeline_steps() { }), StepSource::User, )); - dataset.retransform(); + dataset.retransform().unwrap(); app.doc.datasets.push(Dataset::Nmr(Box::new(dataset))); let path = temp_project("pipeline"); diff --git a/crates/core/src/project/symmetry_tests.rs b/crates/core/src/project/symmetry_tests.rs index bb0a71a8..084ac5b1 100644 --- a/crates/core/src/project/symmetry_tests.rs +++ b/crates/core/src/project/symmetry_tests.rs @@ -4,7 +4,7 @@ use crate::state::{Peak2DOrigin, Peak2DPoint, Peak2DReview}; #[test] fn project_roundtrip_preserves_cross_peak_pairs_and_review_state() { let mut app = PlotxApp::new(); - let mut dataset = Nmr2DDataset::load(super::tests::synthetic_true_2d()); + let mut dataset = crate::nmr_test_support::load_2d(super::tests::synthetic_true_2d()).unwrap(); let ids = dataset .peaks .add_pair( diff --git a/crates/core/src/project/templates.rs b/crates/core/src/project/templates.rs index 9f1f3616..635f5b1e 100644 --- a/crates/core/src/project/templates.rs +++ b/crates/core/src/project/templates.rs @@ -114,16 +114,19 @@ mod tests { let points = (0..64) .map(|k| Complex64::from_polar((-(k as f64) / 16.0).exp(), 0.4 * k as f64)) .collect(); - Dataset::Nmr(Box::new(NmrDataset::load(NmrData { - points, - domain: Domain::Time, - spectral_width_hz: 4000.0, - observe_freq_mhz: 400.0, - carrier_ppm: 5.0, - nucleus: "1H".to_owned(), - source: "synthetic".to_owned(), - group_delay: 0.0, - }))) + Dataset::Nmr(Box::new( + NmrDataset::load(NmrData { + points, + domain: Domain::Time, + spectral_width_hz: 4000.0, + observe_freq_mhz: 400.0, + carrier_ppm: 5.0, + nucleus: "1H".to_owned(), + source: "synthetic".to_owned(), + group_delay: 0.0, + }) + .unwrap(), + )) } #[test] diff --git a/crates/core/src/project/tests.rs b/crates/core/src/project/tests.rs index 6fdddd4f..17f94c72 100644 --- a/crates/core/src/project/tests.rs +++ b/crates/core/src/project/tests.rs @@ -91,10 +91,10 @@ pub(super) fn synthetic_dosy_2d() -> NmrData2D { pub(super) fn sample_app() -> PlotxApp { let mut app = PlotxApp::new(); - let mut dataset = NmrDataset::load(synthetic_1d()); + let mut dataset = NmrDataset::load(synthetic_1d()).unwrap(); dataset.name = Some("sample data".to_owned()); set_manual_phase(&mut dataset.pipeline, 0.25, -0.5, 0.4); - dataset.rebuild(); + dataset.rebuild().unwrap(); app.doc.datasets.push(Dataset::Nmr(Box::new(dataset))); let chart = crate::state::ChartSpec::default_for(app.doc.datasets[0].domain()); @@ -369,7 +369,7 @@ fn project_roundtrip_preserves_data_recipe_and_view() { panic!("expected 1D NMR dataset"); }; assert_eq!(n.name.as_deref(), Some("sample data")); - assert_eq!(n.data.points.len(), 1024); + assert_eq!(n.data.len(), 1024); assert_eq!(n.peaks.marks.len(), 1); assert_eq!(n.peaks.marks[0].label.as_deref(), Some("2.00")); assert_eq!(n.integrals.len(), 1); @@ -416,7 +416,7 @@ fn project_roundtrip_preserves_axis_projections() { // dataset 0 = the 1D spectrum a projection attaches to; dataset 1 = the contour. let mut app = sample_app(); - let ds = Nmr2DDataset::load(synthetic_true_2d()); + let ds = crate::nmr_test_support::load_2d(synthetic_true_2d()).unwrap(); assert!(ds.is_true_2d()); app.doc.datasets.push(Dataset::Nmr2D(Box::new(ds))); let mut canvas = CanvasDocument::new("2d".to_owned(), [120.0, 80.0]); @@ -459,7 +459,7 @@ fn project_roundtrip_preserves_axis_projections() { #[test] fn project_roundtrip_preserves_pseudo2d_metadata() { let mut app = PlotxApp::new(); - let ds = Nmr2DDataset::load(synthetic_dosy_2d()); + let ds = Nmr2DDataset::load(synthetic_dosy_2d()).unwrap(); assert!(ds.is_pseudo(), "fixture should be a pseudo-2D dataset"); app.doc.datasets.push(Dataset::Nmr2D(Box::new(ds))); @@ -477,7 +477,7 @@ fn project_roundtrip_preserves_pseudo2d_metadata() { assert_eq!(axis.name, "g"); assert_eq!(axis.kind, PseudoKind::Gradient); assert_eq!(axis.unit, "mT/m"); - assert_eq!(axis.source, AxisSource::EmbeddedRamp); + assert_eq!(axis.source, AxisSource::LibraryEvidence); assert_eq!(axis.values.len(), 8); let meta = n.data.diffusion.as_ref().expect("diffusion meta preserved"); assert!((meta.delta - 2e-3).abs() < 1e-12); @@ -487,7 +487,7 @@ fn project_roundtrip_preserves_pseudo2d_metadata() { #[test] fn project_roundtrip_preserves_trace_item_sources_and_visibility() { - let dataset = Dataset::Nmr2D(Box::new(Nmr2DDataset::load(synthetic_dosy_2d()))); + let dataset = Dataset::Nmr2D(Box::new(Nmr2DDataset::load(synthetic_dosy_2d()).unwrap())); let mut app = PlotxApp::new(); app.doc.canvases.push(crate::workflow::build_default_canvas( &dataset, @@ -609,9 +609,9 @@ fn project_roundtrip_preserves_zorder() { fn project_roundtrip_preserves_overlay_binding() { let mut app = PlotxApp::new(); for _ in 0..2 { - app.doc - .datasets - .push(Dataset::Nmr(Box::new(NmrDataset::load(synthetic_1d())))); + app.doc.datasets.push(Dataset::Nmr(Box::new( + NmrDataset::load(synthetic_1d()).unwrap(), + ))); } app.doc.datasets[1].set_name(Some("treatment".to_owned())); let mut canvas = CanvasDocument::new("overlay".to_owned(), [120.0, 80.0]); @@ -665,9 +665,9 @@ fn project_roundtrip_preserves_overlay_binding() { fn project_roundtrip_preserves_stack_spec_and_series_fields() { let mut app = PlotxApp::new(); for _ in 0..2 { - app.doc - .datasets - .push(Dataset::Nmr(Box::new(NmrDataset::load(synthetic_1d())))); + app.doc.datasets.push(Dataset::Nmr(Box::new( + NmrDataset::load(synthetic_1d()).unwrap(), + ))); } let mut canvas = CanvasDocument::new("stack".to_owned(), [120.0, 80.0]); let [w, h] = canvas.size_pt(); @@ -731,7 +731,7 @@ fn plot_without_explicit_series_is_rejected_by_the_project_schema() { #[test] fn scheme_save_load_apply_roundtrips() { use crate::actions::DatasetProcessingState; - let mut source = NmrDataset::load(synthetic_1d()); + let mut source = NmrDataset::load(synthetic_1d()).unwrap(); set_manual_phase(&mut source.pipeline, 0.3, 0.1, 0.6); let source_ds = Dataset::Nmr(Box::new(source)); @@ -742,7 +742,7 @@ fn scheme_save_load_apply_roundtrips() { let _ = std::fs::remove_file(&path); assert_eq!(scheme.dimension_count, 1); - let target = Dataset::Nmr(Box::new(NmrDataset::load(synthetic_1d()))); + let target = Dataset::Nmr(Box::new(NmrDataset::load(synthetic_1d()).unwrap())); let DatasetProcessingState::Nmr { pipeline, .. } = apply_scheme(&scheme, &target).unwrap() else { panic!("expected a 1D processing state"); @@ -752,7 +752,9 @@ fn scheme_save_load_apply_roundtrips() { assert!((phase.phase0 - 0.3).abs() < 1e-9); assert!((phase.pivot_frac - 0.6).abs() < 1e-9); - let two_d = Dataset::Nmr2D(Box::new(Nmr2DDataset::load(synthetic_true_2d()))); + let two_d = Dataset::Nmr2D(Box::new( + crate::nmr_test_support::load_2d(synthetic_true_2d()).unwrap(), + )); assert!(apply_scheme(&scheme, &two_d).is_err()); let DatasetProcessingState::Nmr { pipeline, .. } = reset_processing(&source_ds).unwrap() else { diff --git a/crates/core/src/properties/apodization_tests.rs b/crates/core/src/properties/apodization_tests.rs index 7147befc..a8a5a3f8 100644 --- a/crates/core/src/properties/apodization_tests.rs +++ b/crates/core/src/properties/apodization_tests.rs @@ -64,7 +64,7 @@ fn time_domain_app_of(experiment: Option<&str>) -> PlotxApp { let mut app = PlotxApp::new(); app.doc .datasets - .push(Dataset::Nmr2D(Box::new(Nmr2DDataset::load(data)))); + .push(Dataset::Nmr2D(Box::new(Nmr2DDataset::load(data).unwrap()))); app } diff --git a/crates/core/src/properties/bin.rs b/crates/core/src/properties/bin.rs index bedd85e6..50856e18 100644 --- a/crates/core/src/properties/bin.rs +++ b/crates/core/src/properties/bin.rs @@ -75,14 +75,14 @@ impl PropertyProvider for BinProvider { let StepKind::Bin(current) = context.step.kind else { unreachable!("the shared context checked the step kind"); }; - let bounds = resolved_width_bounds(&context)?; + let (bounds, unit) = resolved_width_bounds(&context)?; Ok(ResolvedProperty { address: address.clone(), modified: None, value: AggregateValue::Uniform(value_of(definition, current)?), default_value: None, availability: Availability::Editable, - schema: schema_for(definition, bounds), + schema: schema_for(definition, bounds, unit), }) } @@ -97,7 +97,7 @@ impl PropertyProvider for BinProvider { let context = step_context(app, address, definition, |kind| { matches!(kind, StepKind::Bin(_)) })?; - let bounds = resolved_width_bounds(&context)?; + let (bounds, _) = resolved_width_bounds(&context)?; let value = match operation { EditOp::Set(value) => checked_value(definition, bounds, value)?, EditOp::Reset => { @@ -140,11 +140,15 @@ fn value_of( } } -fn schema_for(definition: &'static PropertyDefinition, bounds: FloatBounds) -> ResolvedSchema { +fn schema_for( + definition: &'static PropertyDefinition, + bounds: FloatBounds, + unit: &'static str, +) -> ResolvedSchema { if definition.id == WIDTH { ResolvedSchema::Float { bounds, - display: FloatDisplay::Linear("ppm"), + display: FloatDisplay::Linear(unit), } } else { ResolvedSchema::Enum { @@ -179,12 +183,15 @@ fn checked_value( fn resolved_width_bounds( context: &super::processing_common::StepContext<'_>, -) -> Result { +) -> Result<(FloatBounds, &'static str), PropertyError> { let spectrum = spectrum_before_step(context).ok_or_else(|| { PropertyError::NotApplicable( "Binning needs a one-dimensional input spectrum with an axis.".to_owned(), ) })?; - let axis_step = plotx_processing::cleanup::axis_step(&spectrum.ppm); - Ok(FloatBounds::above(1.5 * axis_step, f64::MAX)) + let axis_step = spectrum.coordinate_spacing().unwrap_or(0.0); + Ok(( + FloatBounds::above(1.5 * axis_step, f64::MAX), + plotx_processing::axis_unit_label(Some(spectrum.unit)), + )) } diff --git a/crates/core/src/properties/group_delay.rs b/crates/core/src/properties/group_delay.rs index 9e7dd7bd..28ce6634 100644 --- a/crates/core/src/properties/group_delay.rs +++ b/crates/core/src/properties/group_delay.rs @@ -130,8 +130,10 @@ impl NmrDatasetContext<'_> { fn factory_value(self) -> bool { match self { - Self::One(dataset) => crate::state::default_group_delay_correct(dataset.data.domain), - Self::Two(dataset) => crate::state::default_group_delay_correct(dataset.data.domain), + Self::One(dataset) => crate::state::default_group_delay_correct(&dataset.data), + Self::Two(dataset) => { + crate::state::default_group_delay_correct(dataset.data.source_dataset()) + } } } } diff --git a/crates/core/src/properties/group_delay_tests.rs b/crates/core/src/properties/group_delay_tests.rs index 11dca4ed..44313a71 100644 --- a/crates/core/src/properties/group_delay_tests.rs +++ b/crates/core/src/properties/group_delay_tests.rs @@ -33,7 +33,7 @@ fn time_domain_2d_app() -> PlotxApp { let mut app = PlotxApp::new(); app.doc .datasets - .push(Dataset::Nmr2D(Box::new(Nmr2DDataset::load(data)))); + .push(Dataset::Nmr2D(Box::new(Nmr2DDataset::load(data).unwrap()))); app } @@ -78,10 +78,19 @@ fn two_dimensional_group_delay_is_in_the_typed_action_and_is_undoable() { ); let disabled_input = app.doc.datasets[0] .as_nmr2d() - .expect("the dataset remains 2D NMR") - .processing_data(); - assert_eq!(disabled_input.direct.group_delay, 0.0); - assert_eq!(disabled_input.indirect.group_delay, 4.0); + .unwrap() + .data + .source_dataset(); + let delay = disabled_input + .dataset() + .as_raw() + .unwrap() + .descriptor() + .axes()[1] + .group_delay(); + assert!( + matches!(delay, nmr::acquisition::GroupDelayState::Pending(value) if value.delay_points() == 4.0) + ); app.undo(); assert!( @@ -101,7 +110,20 @@ fn two_dimensional_group_delay_settings_produce_different_real_spectra() { }; let corrected = { let dataset = app.doc.datasets[0].as_nmr2d().unwrap(); - plotx_processing::process_2d(&dataset.processing_data(), &dataset.params) + plotx_processing::nmr_execution::execute_2d( + dataset.data.source_dataset(), + &dataset.params, + if dataset.group_delay_correct { + plotx_processing::nmr_bridge::DelayPolicy::AxisEvidence + } else { + plotx_processing::nmr_bridge::DelayPolicy::Disabled + }, + plotx_processing::nmr_bridge::RecipeRange::Base, + None, + &mut nmr::ExecutionContext::default(), + ) + .unwrap() + .view }; let changed = app .plan_property_write( @@ -113,7 +135,20 @@ fn two_dimensional_group_delay_settings_produce_different_real_spectra() { app.commit_property(changed); let uncorrected = { let dataset = app.doc.datasets[0].as_nmr2d().unwrap(); - plotx_processing::process_2d(&dataset.processing_data(), &dataset.params) + plotx_processing::nmr_execution::execute_2d( + dataset.data.source_dataset(), + &dataset.params, + if dataset.group_delay_correct { + plotx_processing::nmr_bridge::DelayPolicy::AxisEvidence + } else { + plotx_processing::nmr_bridge::DelayPolicy::Disabled + }, + plotx_processing::nmr_bridge::RecipeRange::Base, + None, + &mut nmr::ExecutionContext::default(), + ) + .unwrap() + .view }; let ( plotx_processing::Processed2D::Ft(corrected), @@ -147,3 +182,51 @@ fn group_delay_reset_uses_the_same_factory_rule_as_dataset_construction() { app.commit_property(reset); assert!(app.doc.datasets[0].as_nmr2d().unwrap().group_delay_correct); } + +#[test] +fn reset_preserves_unknown_delay_and_processed_input_defaults() { + let fixtures = + std::path::Path::new(env!("CARGO_MANIFEST_DIR")).join("../io/tests/fixtures/nmr"); + for name in ["jeol-complex.jdf", "bruker-1d/pdata/1/1r"] { + let loaded = plotx_io::load_path(fixtures.join(name)).unwrap(); + let plotx_io::Acquisition::Nmr(source) = loaded.acquisition else { + panic!("expected NMR fixture"); + }; + let mut app = PlotxApp::new(); + app.doc.datasets.push(Dataset::Nmr(Box::new( + crate::state::NmrDataset::load(source).unwrap(), + ))); + let target = TargetRef { + resource: ResourceRef::from(app.doc.datasets[0].resource_id()), + component: None, + }; + let reset = app + .plan_property_reset(group_delay::CORRECT, &[target]) + .unwrap(); + app.commit_property(reset); + assert!(!app.doc.datasets[0].as_nmr().unwrap().group_delay_correct); + let reset = crate::project::reset_processing(&app.doc.datasets[0]).unwrap(); + reset.apply_to(&mut app.doc.datasets[0]).unwrap(); + let dataset = app.doc.datasets[0].as_nmr().unwrap(); + assert!(!dataset.group_delay_correct); + assert!(!dataset.pipeline.has_enabled_fft()); + assert_eq!(dataset.output_domain(), dataset.input_domain()); + } +} + +#[test] +fn reset_of_an_imported_2d_spectrum_does_not_add_time_domain_steps() { + let app = time_domain_2d_app(); + let source = app.doc.datasets[0] + .as_nmr2d() + .unwrap() + .native_processed + .clone(); + let mut dataset = Dataset::Nmr2D(Box::new(Nmr2DDataset::load(source).unwrap())); + let reset = crate::project::reset_processing(&dataset).unwrap(); + reset.apply_to(&mut dataset).unwrap(); + let dataset = dataset.as_nmr2d().unwrap(); + assert!(!dataset.params.f1.has_enabled_fft()); + assert!(!dataset.params.f2.has_enabled_fft()); + assert!(!dataset.group_delay_correct); +} diff --git a/crates/core/src/properties/ilt_tests.rs b/crates/core/src/properties/ilt_tests.rs index a9249057..f5e12f23 100644 --- a/crates/core/src/properties/ilt_tests.rs +++ b/crates/core/src/properties/ilt_tests.rs @@ -2,7 +2,7 @@ use super::*; use crate::automation::{ResourceRef, TargetRef}; use crate::properties::ilt; use crate::settings::{MAX_ILT_LAMBDA, MIN_ILT_LAMBDA, Settings}; -use crate::state::{Dataset, Nmr2DDataset, PlotxApp}; +use crate::state::{Dataset, PlotxApp}; use crate::{DosyInvocation, DosyResultProvenance, IltParams}; use num_complex::Complex64; use plotx_io::{ @@ -49,7 +49,7 @@ fn data() -> NmrData2D { pub(crate) fn ilt_app(lambda: f64) -> (PlotxApp, TargetRef) { let mut app = PlotxApp::new_with_settings(Settings::default()); - let mut dataset = Nmr2DDataset::load(data()); + let mut dataset = crate::nmr_test_support::load_2d(data()).unwrap(); dataset.ilt_provenance = Some(DosyResultProvenance { algorithm: "ilt_map".to_owned(), version: 1, diff --git a/crates/core/src/properties/object_tests.rs b/crates/core/src/properties/object_tests.rs index f79561d1..b102a193 100644 --- a/crates/core/src/properties/object_tests.rs +++ b/crates/core/src/properties/object_tests.rs @@ -78,9 +78,9 @@ fn stack_app() -> (PlotxApp, ObjectId) { source: source.to_owned(), group_delay: 0.0, }; - app.doc - .datasets - .push(Dataset::Nmr(Box::new(crate::state::NmrDataset::load(data)))); + app.doc.datasets.push(Dataset::Nmr(Box::new( + crate::state::NmrDataset::load(data).unwrap(), + ))); } let mut canvas = CanvasDocument::new("stack".to_owned(), [120.0, 80.0]); let id = canvas.allocate_object_id(); diff --git a/crates/core/src/properties/processing_common.rs b/crates/core/src/properties/processing_common.rs index 1fd90e11..fedb0062 100644 --- a/crates/core/src/properties/processing_common.rs +++ b/crates/core/src/properties/processing_common.rs @@ -154,10 +154,7 @@ pub(super) fn raw_point_count(dataset: &Dataset, axis: PhaseAxis) -> usize { match dataset { Dataset::Nmr(n) => n.data.len(), Dataset::Nmr2D(n) => match axis { - PhaseAxis::F1 => n.data.nus.as_ref().map_or_else( - || plotx_processing::fft2::f1_increments(n.data.rows, n.data.quad), - |nus| nus.grid, - ), + PhaseAxis::F1 => n.data.source_dataset().axes()[0].points, PhaseAxis::F2 | PhaseAxis::Direct => n.data.cols, }, Dataset::Table(_) @@ -176,19 +173,26 @@ pub(super) fn spectrum_before_step(context: &StepContext<'_>) -> Option result.view.as_frequency().cloned(), + Err(error) => { + eprintln!("Cannot resolve NMR property bounds: {error}"); + None } } - None } diff --git a/crates/core/src/properties/processing_test_support.rs b/crates/core/src/properties/processing_test_support.rs index 250fc397..b2a11687 100644 --- a/crates/core/src/properties/processing_test_support.rs +++ b/crates/core/src/properties/processing_test_support.rs @@ -27,11 +27,18 @@ pub(super) fn time_domain_app() -> PlotxApp { let mut app = PlotxApp::new(); app.doc .datasets - .push(Dataset::Nmr(Box::new(NmrDataset::load(data)))); + .push(Dataset::Nmr(Box::new(NmrDataset::load(data).unwrap()))); app } pub(super) fn states_2d_app(rows: usize, cols: usize) -> PlotxApp { + states_2d_app_with_sampling(rows, cols, None) +} +pub(super) fn states_2d_app_with_sampling( + rows: usize, + cols: usize, + nus: Option, +) -> PlotxApp { let dim = |nucleus: &str, width| Dim { spectral_width_hz: width, observe_freq_mhz: 400.0, @@ -53,13 +60,23 @@ pub(super) fn states_2d_app(rows: usize, cols: usize) -> PlotxApp { experiment: Some("hsqc".to_owned()), pseudo_axis: None, diffusion: None, - nus: None, + nus, source: "States property test".to_owned(), }; let mut app = PlotxApp::new(); - app.doc - .datasets - .push(Dataset::Nmr2D(Box::new(Nmr2DDataset::load(data)))); + app.doc.datasets.push(Dataset::Nmr2D(Box::new( + Nmr2DDataset::load_with_pipeline( + data, + Some(plotx_processing::Params2D::default()), + Some(true), + Some(plotx_processing::nmr_execution::NusRequest { + noise_standard_deviation: Some(0.0), + ..Default::default() + }), + true, + ) + .unwrap(), + ))); app } diff --git a/crates/core/src/properties/provider_tests.rs b/crates/core/src/properties/provider_tests.rs index 86c58b82..0726da3b 100644 --- a/crates/core/src/properties/provider_tests.rs +++ b/crates/core/src/properties/provider_tests.rs @@ -167,11 +167,9 @@ fn a_same_value_write_is_reported_without_an_empty_commit() { #[test] fn line_stroke_width_reports_mixed_values_and_skips_other_encodings() { let (mut app, contour) = contour_app(); - app.doc - .datasets - .push(Dataset::Nmr(Box::new(NmrDataset::load(nmr1d_with( - "lines", - ))))); + app.doc.datasets.push(Dataset::Nmr(Box::new( + NmrDataset::load(nmr1d_with("lines")).unwrap(), + ))); let mut line_targets = Vec::new(); for name in ["Line A", "Line B"] { let id = app.doc.canvases[0].allocate_object_id(); @@ -274,11 +272,9 @@ fn line_stroke_width_reports_mixed_values_and_skips_other_encodings() { #[test] fn a_line_readout_dispatches_by_property_address() { let (mut app, _) = contour_app(); - app.doc - .datasets - .push(Dataset::Nmr(Box::new(NmrDataset::load(nmr1d_with( - "line readout", - ))))); + app.doc.datasets.push(Dataset::Nmr(Box::new( + NmrDataset::load(nmr1d_with("line readout")).unwrap(), + ))); let id = app.doc.canvases[0].allocate_object_id(); let object = app.build_plot_object( 1, diff --git a/crates/core/src/properties/step_enabled.rs b/crates/core/src/properties/step_enabled.rs index bedb24ea..6ecc112e 100644 --- a/crates/core/src/properties/step_enabled.rs +++ b/crates/core/src/properties/step_enabled.rs @@ -78,8 +78,10 @@ impl PropertyProvider for StepEnabledProvider { EditOp::Step(_) => return Err(no_step_gesture(definition)), }; let input_domain = match context.dataset { - crate::state::Dataset::Nmr(dataset) => dataset.data.domain, - crate::state::Dataset::Nmr2D(dataset) => dataset.data.domain, + crate::state::Dataset::Nmr(dataset) => dataset.input_domain(), + crate::state::Dataset::Nmr2D(dataset) => dataset + .input_domain(context.axis) + .map_err(PropertyError::NotApplicable)?, crate::state::Dataset::Table(_) | crate::state::Dataset::Electrophysiology(_) | crate::state::Dataset::Afm(_) diff --git a/crates/core/src/properties/tests.rs b/crates/core/src/properties/tests.rs index 1fe45aee..34910044 100644 --- a/crates/core/src/properties/tests.rs +++ b/crates/core/src/properties/tests.rs @@ -6,8 +6,7 @@ use crate::automation::{ }; use crate::state::{ CONTOUR_BASE_ABSOLUTE, CONTOUR_BASE_FRACTION_OF_RANGE, CONTOUR_BASE_NOISE_FLOOR, - CanvasDocument, Dataset, Nmr2DDataset, NmrDataset, ObjectFrame, PlotxApp, SeriesBinding, - SeriesId, + CanvasDocument, Dataset, NmrDataset, ObjectFrame, PlotxApp, SeriesBinding, SeriesId, }; #[path = "tests_fixture.rs"] diff --git a/crates/core/src/properties/tests_fixture.rs b/crates/core/src/properties/tests_fixture.rs index 5dfc8fa0..fadfcbff 100644 --- a/crates/core/src/properties/tests_fixture.rs +++ b/crates/core/src/properties/tests_fixture.rs @@ -61,11 +61,9 @@ pub(crate) fn contour_app() -> (PlotxApp, TargetRef) { /// a given dynamic range in front of the catalog. pub(crate) fn contour_app_with_plane(values: &[f64]) -> (PlotxApp, TargetRef) { let mut app = PlotxApp::new(); - app.doc - .datasets - .push(Dataset::Nmr2D(Box::new(Nmr2DDataset::load(nmr2d_with( - "contour", values, - ))))); + app.doc.datasets.push(Dataset::Nmr2D(Box::new( + crate::nmr_test_support::load_2d(nmr2d_with("contour", values)).unwrap(), + ))); let mut canvas = CanvasDocument::new("page".to_owned(), [120.0, 80.0]); let id = canvas.allocate_object_id(); let object = app.build_plot_object( diff --git a/crates/core/src/properties/zero_fill_tests.rs b/crates/core/src/properties/zero_fill_tests.rs index 1bd07dc0..eec49659 100644 --- a/crates/core/src/properties/zero_fill_tests.rs +++ b/crates/core/src/properties/zero_fill_tests.rs @@ -2,7 +2,6 @@ use super::processing_test_support::{ spectrum, states_2d_app, step, step_mut, target_for, target_for_axis, time_domain_app, }; use super::*; -use crate::state::Dataset; use plotx_processing::{StepKind, ZeroFill}; #[test] @@ -133,18 +132,15 @@ fn states_f1_uses_complex_increments_as_its_raw_point_count() { #[test] fn nus_f1_uses_the_nominal_reconstruction_grid_as_its_raw_count() { - let mut app = states_2d_app(10, 6); - let Dataset::Nmr2D(dataset) = &mut app.doc.datasets[0] else { - panic!("the fixture is 2D NMR"); - }; - std::sync::Arc::make_mut(&mut dataset.data).nus = Some(plotx_io::NusMeta { - grid: 17, - acquired: 5, - idx_base: 0, - mode: "test".to_owned(), - echo_antiecho: false, - schedule: Some(vec![0, 2, 5, 9, 16]), - }); + let mut app = super::processing_test_support::states_2d_app_with_sampling( + 10, + 6, + Some(plotx_io::NusMeta { + grid: 17, + acquired: 5, + schedule: Some(vec![0, 2, 5, 9, 16]), + }), + ); let target = target_for_axis(&app, crate::state::PhaseAxis::F1, |kind| { matches!(kind, StepKind::ZeroFill(_)) }); diff --git a/crates/core/src/state/app_impl.rs b/crates/core/src/state/app_impl.rs index 7857f688..f9a6b448 100644 --- a/crates/core/src/state/app_impl.rs +++ b/crates/core/src/state/app_impl.rs @@ -79,6 +79,7 @@ impl PlotxApp { ..Default::default() }, compute: ComputeService::new(), + data_imports: DataImports::default(), updates: crate::update::UpdateService::new(&settings.updates), line_fit_job: None, xps_fit_job: None, @@ -145,23 +146,25 @@ impl PlotxApp { let Some(d2) = self.doc.datasets.get(dataset).and_then(Dataset::as_nmr2d) else { return; }; - let Processed2D::Ft(spec) = &d2.processed else { + let Processed2D::Ft(_) = &d2.processed else { return; }; - fig.top_projection = self.build_axis_trace(spec, SliceKind::Row, &projections.top); - fig.left_projection = self.build_axis_trace(spec, SliceKind::Column, &projections.left); + fig.top_projection = + self.build_axis_trace(&d2.native_processed, SliceKind::Row, &projections.top); + fig.left_projection = + self.build_axis_trace(&d2.native_processed, SliceKind::Column, &projections.left); } fn build_axis_trace( &self, - spec: &plotx_processing::Spectrum2D, + source: &plotx_io::nmr_view::NmrSource, kind: SliceKind, cfg: &AxisProjection, ) -> Option { if !cfg.is_shown() { return None; } - let slice = match &cfg.source { + let reduction = match &cfg.source { ProjectionSource::None => return None, ProjectionSource::Attached(other) => { return self @@ -169,9 +172,20 @@ impl PlotxApp { .dataset_index(*other) .and_then(|index| self.attached_axis_trace(index)); } - ProjectionSource::Sum => spec.project(kind, ProjectionMode::Sum), - ProjectionSource::Skyline => spec.project(kind, ProjectionMode::Skyline), - ProjectionSource::Slice(index) => spec.slice(kind, *index), + ProjectionSource::Sum => { + plotx_processing::slice::Reduction::Projection(ProjectionMode::Sum) + } + ProjectionSource::Skyline => { + plotx_processing::slice::Reduction::Projection(ProjectionMode::Skyline) + } + ProjectionSource::Slice(index) => plotx_processing::slice::Reduction::Slice(*index), + }; + let (_, slice) = match plotx_processing::slice::extract(source, kind, reduction) { + Ok(output) => output, + Err(error) => { + eprintln!("NMR axis projection unavailable: {error}"); + return None; + } }; let points = slice .coordinates @@ -592,7 +606,10 @@ impl PlotxApp { /// Secondary Side Bar tool widgets. pub fn apply_dataset_edit(&mut self, dataset: usize) { if let Some(n) = self.doc.datasets[dataset].as_nmr_mut() { - n.rebuild(); + if let Err(error) = n.rebuild() { + self.session.status = format!("NMR processing failed: {error}"); + return; + } n.recompute_integrals(); } else if self.doc.datasets[dataset].as_nmr2d().is_some() { self.schedule_2d_processing(dataset, false); @@ -618,7 +635,10 @@ impl PlotxApp { /// for dragging a time-domain step parameter, where the cached base changes. pub fn apply_dataset_retransform(&mut self, dataset: usize) { if let Some(n) = self.doc.datasets[dataset].as_nmr_mut() { - n.retransform(); + if let Err(error) = n.retransform() { + self.session.status = format!("NMR processing failed: {error}"); + return; + } n.recompute_integrals(); } else if self.doc.datasets[dataset].as_nmr2d().is_some() { self.schedule_2d_processing(dataset, true); diff --git a/crates/core/src/state/app_impl_analysis.rs b/crates/core/src/state/app_impl_analysis.rs index 73a32e54..4c782c52 100644 --- a/crates/core/src/state/app_impl_analysis.rs +++ b/crates/core/src/state/app_impl_analysis.rs @@ -1,31 +1,6 @@ use super::*; impl PlotxApp { - /// Apply a user-entered non-uniform-sampling schedule to a 2D dataset and - /// re-run the reconstruction. Returns the validation error (if any) so the - /// caller can surface it next to the input field. - pub fn apply_nus_schedule( - &mut self, - dataset: usize, - values: &[usize], - base: usize, - ) -> Result<(), String> { - let Some(d2) = self - .doc - .datasets - .get_mut(dataset) - .and_then(Dataset::as_nmr2d_mut) - else { - return Err("NUS reconstruction needs a 2D dataset.".into()); - }; - d2.set_nus_schedule(values, base)?; - self.schedule_2d_processing(dataset, true); - self.mark_document_dirty(); - self.session.status = - "Reconstructing the NUS spectrum from the entered sampling list…".into(); - Ok(()) - } - /// Fit every column to build the DOSY contour map (diffusion datasets only). pub fn build_dosy_map_for(&mut self, dataset: usize) { let Some(d2) = self @@ -37,6 +12,10 @@ impl PlotxApp { self.session.status = "DOSY maps need a diffusion dataset.".into(); return; }; + if let Some(error) = d2.dosy_input_error() { + self.session.status = error.into(); + return; + } if d2.data.diffusion.is_none() { self.session.status = "This dataset has no diffusion parameters (not a DOSY array).".into(); @@ -77,6 +56,10 @@ impl PlotxApp { self.session.status = "ILT DOSY maps need a diffusion dataset.".into(); return; }; + if let Some(error) = d2.dosy_input_error() { + self.session.status = error.into(); + return; + } if d2.data.diffusion.is_none() { self.session.status = "This dataset has no diffusion parameters (not a DOSY array).".into(); diff --git a/crates/core/src/state/app_impl_analysis_tests.rs b/crates/core/src/state/app_impl_analysis_tests.rs index f67b8e4c..c564e9e9 100644 --- a/crates/core/src/state/app_impl_analysis_tests.rs +++ b/crates/core/src/state/app_impl_analysis_tests.rs @@ -35,7 +35,7 @@ fn live_and_frozen_region_tables_record_lineage() { nus: None, source: "series".to_owned(), }; - let mut source = Nmr2DDataset::load(data); + let mut source = crate::nmr_test_support::load_2d(data).unwrap(); source.region_analysis.regions.push(Region { id: RegionId::new(0), lo: 4.0, diff --git a/crates/core/src/state/app_impl_arithmetic.rs b/crates/core/src/state/app_impl_arithmetic.rs index 17bd5357..31c9c655 100644 --- a/crates/core/src/state/app_impl_arithmetic.rs +++ b/crates/core/src/state/app_impl_arithmetic.rs @@ -1,5 +1,5 @@ use super::*; -use plotx_processing::Slice1D; +use plotx_io::nmr_view::NmrSource; use plotx_processing::arithmetic::{ SpectrumBinaryOp, combine_spectra, same_grid, scale_offset_spectrum, }; @@ -34,6 +34,17 @@ impl PlotxApp { sa.nucleus, sb.nucleus )); } + let left = self.doc.datasets[a] + .as_nmr() + .ok_or("Select an NMR spectrum")?; + let right = self.doc.datasets[b] + .as_nmr() + .ok_or("Select an NMR spectrum")?; + plotx_processing::arithmetic::validate_combination( + &left.native_processed, + &right.native_processed, + ) + .map_err(|error| error.to_string())?; if same_grid(sa, sb) { Ok(None) } else { @@ -46,12 +57,14 @@ impl PlotxApp { } pub fn combine_spectra_datasets(&mut self, a: usize, b: usize, op: SpectrumBinaryOp, k: f64) { - let (Some(sa), Some(sb)) = (self.arithmetic_spectrum(a), self.arithmetic_spectrum(b)) - else { + let (Some(sa), Some(sb)) = ( + self.doc.datasets.get(a).and_then(Dataset::as_nmr), + self.doc.datasets.get(b).and_then(Dataset::as_nmr), + ) else { self.session.status = "Spectrum arithmetic needs two 1D NMR spectra.".into(); return; }; - let result = match combine_spectra(sa, sb, op, k) { + let result = match combine_spectra(&sa.native_processed, &sb.native_processed, op, k) { Ok(result) => result, Err(error) => { self.session.status = error.to_string(); @@ -70,7 +83,7 @@ impl PlotxApp { } pub fn scale_spectrum_dataset(&mut self, a: usize, scale: f64, offset: f64) { - let Some(sa) = self.arithmetic_spectrum(a) else { + let Some(sa) = self.doc.datasets.get(a).and_then(Dataset::as_nmr) else { self.session.status = "Spectrum arithmetic needs a 1D NMR spectrum.".into(); return; }; @@ -78,7 +91,13 @@ impl PlotxApp { self.session.status = "Nothing to compute: scale is 1 and offset is 0.".into(); return; } - let result = scale_offset_spectrum(sa, scale, offset); + let result = match scale_offset_spectrum(&sa.native_processed, scale, offset) { + Ok(result) => result, + Err(error) => { + self.session.status = error.to_string(); + return; + } + }; let name_a = self.doc.datasets[a].display_name(); let scaled = if scale == 1.0 { name_a @@ -108,7 +127,7 @@ impl PlotxApp { /// dataset on its own page, as one undoable step (same path as slices). fn insert_arithmetic_dataset( &mut self, - result: Spectrum, + result: NmrSource, name: String, sources: impl IntoIterator, ) { @@ -116,16 +135,20 @@ impl PlotxApp { .into_iter() .filter_map(|index| self.doc.datasets.get(index).map(Dataset::resource_id)) .collect::>(); - let slice = Slice1D { - coordinates: result.ppm, - domain: plotx_io::Domain::Frequency, - values: result.values, - nucleus: result.nucleus, - observe_freq_mhz: result.observe_freq_mhz, - position: None, - position_domain: plotx_io::Domain::Frequency, + let dataset = match NmrDataset::load_with_pipeline( + result, + Some(AxisPipeline { steps: Vec::new() }), + Some(false), + ) { + Ok(dataset) => dataset, + Err(error) => { + self.session.status = format!("Spectrum arithmetic failed: {error}"); + return; + } }; - let mut ds = Dataset::Nmr(Box::new(NmrDataset::from_slice(slice, name.clone()))); + let mut dataset = dataset; + dataset.name = Some(name.clone()); + let mut ds = Dataset::Nmr(Box::new(dataset)); ds.set_lineage(Some(DatasetLineage::new( DerivationKind::SpectrumArithmetic, sources, diff --git a/crates/core/src/state/app_impl_compute.rs b/crates/core/src/state/app_impl_compute.rs index f986d200..9c85cecf 100644 --- a/crates/core/src/state/app_impl_compute.rs +++ b/crates/core/src/state/app_impl_compute.rs @@ -57,6 +57,10 @@ impl PlotxApp { self.session.status = "DOSY maps need a diffusion dataset.".into(); return; }; + if let Some(error) = d2.dosy_input_error() { + self.session.status = error.into(); + return; + } if d2.data.diffusion.is_none() { self.session.status = "This dataset has no diffusion parameters (not a DOSY array).".into(); @@ -104,6 +108,10 @@ impl PlotxApp { self.session.status = "ILT DOSY maps need a diffusion dataset.".into(); return; }; + if let Some(error) = d2.dosy_input_error() { + self.session.status = error.into(); + return; + } if d2.data.diffusion.is_none() { self.session.status = "This dataset has no diffusion parameters (not a DOSY array).".into(); @@ -184,18 +192,33 @@ impl PlotxApp { self.session.status = "CRAFT requires a one-dimensional NMR dataset.".into(); return false; }; - if nmr.data.domain != Domain::Time { + if nmr.input_domain() != Domain::Time { self.session.status = "CRAFT requires the original time-domain FID.".into(); return false; } let dataset_id = nmr.resource_id; - let reference = nmr.craft_reference(); + let data = match nmr.data.craft_fid() { + Ok(data) => data, + Err(error) => { + let message = error.to_string(); + self.session.status = message.clone(); + self.session + .ui + .craft_feedback + .insert(dataset_id, CraftRunFeedback::Failed { message }); + return false; + } + }; + let Some(reference) = nmr.craft_reference() else { + self.session.status = "CRAFT requires chemical-shift reference evidence".into(); + return false; + }; let provenance = base_run.and_then(|id| nmr.craft_run(id).map(|run| &run.provenance.invocation)); let invocation = plotx_processing::craft::resolve_craft_invocation( - &nmr.data, reference, &overrides, provenance, + &data, reference, &overrides, provenance, ); - if let Err(error) = invocation.validate(&nmr.data) { + if let Err(error) = invocation.validate(&data) { self.session.status = error.to_string(); self.session.ui.craft_feedback.insert( dataset_id, @@ -205,7 +228,7 @@ impl PlotxApp { ); return false; } - let data = std::sync::Arc::new(nmr.data.clone()); + let data = std::sync::Arc::new(data); let parent_run = invocation .sources .uses_result_provenance() @@ -536,6 +559,7 @@ impl PlotxApp { let Some(dataset) = self.doc.dataset_index(dataset) else { continue; }; + let previous_field = self.doc.datasets[dataset].default_field_id(); let Some(d2) = self .doc .datasets @@ -551,11 +575,15 @@ impl PlotxApp { // `params` may also lag `d2.params` for a paused edit, which is // the intended display-trails-recipe contract. if let Some(base) = base { - d2.base = base; + d2.native_base = base.source; + d2.base = base.view; d2.base_params = params; d2.base_stale = false; } - d2.processed = processed; + d2.native_processed = processed.source; + d2.reconstruction_warning = None; + d2.phase_reports = processed.phases; + d2.processed = processed.view; d2.processed_figure = std::sync::Arc::new(build_processed_figure(&d2.processed, d2.preset)); d2.invalidate_dosy_results( @@ -566,11 +594,19 @@ impl PlotxApp { .compute .promote_field_version(field.source, field.summary); } + self.initialize_nmr_result_bindings(dataset, previous_field); self.recompute_integrals_2d_after_processing(dataset); self.rebuild_canvases_for(dataset); self.mark_document_dirty(); self.session.status = "Updated 2D processing.".into(); } + Done::Processing2DFailed { + dataset, message, .. + } => { + if self.doc.dataset_index(dataset).is_some() { + self.session.status = format!("2D processing failed: {message}"); + } + } Done::EstimateField { key, result } => { let dataset = self.doc @@ -672,8 +708,8 @@ impl PlotxApp { let Some(d2) = self.doc.datasets.get(dataset).and_then(Dataset::as_nmr2d) else { return false; }; - // `base_stale` covers a mutation of `data` itself, which the recipe - // comparison cannot see. It stays set until a fresh base lands, so an + // `base_stale` covers NUS and delay inputs outside the axis recipes. + // It stays set until a fresh base lands, so an // intervening frequency-only edit cannot downgrade the pending retransform // to a re-apply and strand the reconstruction. let full = force_full @@ -699,13 +735,27 @@ impl PlotxApp { .flatten() .collect::>(); let outcome = if full { - self.session - .compute - .request_2d_full(dataset_id, &fields, d2.processing_data(), params) + self.session.compute.request_2d_full( + dataset_id, + &fields, + super::compute::Full2DInput { + source: d2.data.source_dataset().clone(), + delay: if d2.group_delay_correct { + plotx_processing::nmr_bridge::DelayPolicy::AxisEvidence + } else { + plotx_processing::nmr_bridge::DelayPolicy::Disabled + }, + nus: d2.nus_request, + }, + params, + ) } else { - self.session - .compute - .request_2d_reapply(dataset_id, &fields, d2.base.clone(), params) + self.session.compute.request_2d_reapply( + dataset_id, + &fields, + d2.native_base.clone(), + params, + ) }; let aborted = match outcome { Ok(aborted) => aborted, diff --git a/crates/core/src/state/app_impl_compute_tests.rs b/crates/core/src/state/app_impl_compute_tests.rs index 2bf74bde..90a1b0ff 100644 --- a/crates/core/src/state/app_impl_compute_tests.rs +++ b/crates/core/src/state/app_impl_compute_tests.rs @@ -54,12 +54,51 @@ fn craft_data() -> plotx_io::NmrData { } } +#[test] +fn native_processing_failure_reaches_status_without_replacing_the_display() { + let mut app = PlotxApp::new(); + let mut dataset = Nmr2DDataset::load(data_2d("failure target")).unwrap(); + let original = dataset.native_processed.dataset().canonical_digests(); + let step = dataset.allocate_step_id(); + dataset.params.f2.steps.push(ProcessingStep::new( + step, + StepKind::Phase(plotx_processing::PhaseParams { + phase0: f64::NAN, + ..plotx_processing::PhaseParams::MANUAL_ZERO + }), + StepSource::User, + )); + app.doc.datasets.push(Dataset::Nmr2D(Box::new(dataset))); + assert!(app.schedule_2d_processing(0, false)); + let deadline = Instant::now() + Duration::from_secs(3); + while app.compute_busy() && Instant::now() < deadline { + app.poll_compute(); + std::thread::sleep(Duration::from_millis(5)); + } + app.poll_compute(); + assert!(!app.compute_busy()); + assert!( + app.session.status.contains("2D processing failed"), + "{}", + app.session.status + ); + assert_eq!( + app.doc.datasets[0] + .as_nmr2d() + .unwrap() + .native_processed + .dataset() + .canonical_digests(), + original + ); +} + #[test] fn craft_result_is_installed_with_provenance_by_dataset_identity() { let mut app = PlotxApp::new(); - app.doc - .datasets - .push(Dataset::Nmr(Box::new(NmrDataset::load(craft_data())))); + app.doc.datasets.push(Dataset::Nmr(Box::new( + NmrDataset::load(craft_data()).unwrap(), + ))); let nmr = app.doc.datasets[0].as_nmr_mut().unwrap(); let reference_id = nmr.allocate_step_id(); nmr.pipeline.steps.push(ProcessingStep::new( @@ -70,7 +109,7 @@ fn craft_result_is_installed_with_provenance_by_dataset_identity() { }), StepSource::User, )); - nmr.rebuild(); + nmr.rebuild().unwrap(); let target = app.doc.datasets[0].resource_id(); app.session.ui.craft_task_dataset = Some(target); let mut params = plotx_processing::craft::CraftParams::conventional(); @@ -159,16 +198,16 @@ fn craft_result_is_installed_with_provenance_by_dataset_identity() { }) .unwrap(); reference.target_ppm += 0.1; - nmr.rebuild(); + nmr.rebuild().unwrap(); assert!(nmr.craft_runs[0].is_stale_for(&nmr.data, nmr.craft_reference())); } #[test] fn craft_rerun_keeps_requested_parent_without_hijacking_another_task() { let mut app = PlotxApp::new(); - app.doc - .datasets - .push(Dataset::Nmr(Box::new(NmrDataset::load(craft_data())))); + app.doc.datasets.push(Dataset::Nmr(Box::new( + NmrDataset::load(craft_data()).unwrap(), + ))); let target = app.doc.datasets[0].resource_id(); app.session.ui.craft_task_dataset = Some(target); let mut params = plotx_processing::craft::CraftParams::conventional(); @@ -193,9 +232,9 @@ fn craft_rerun_keeps_requested_parent_without_hijacking_another_task() { plotx_processing::craft::CraftParamOverrides::default(), Some(CraftRunId(0)), )); - app.doc - .datasets - .push(Dataset::Nmr(Box::new(NmrDataset::load(craft_data())))); + app.doc.datasets.push(Dataset::Nmr(Box::new( + NmrDataset::load(craft_data()).unwrap(), + ))); let other = app.doc.datasets[1].resource_id(); app.session.ui.craft_task_dataset = Some(other); app.session.ui.craft_base_run = None; @@ -225,16 +264,12 @@ fn craft_rerun_keeps_requested_parent_without_hijacking_another_task() { #[test] fn process_2d_result_follows_dataset_identity_after_earlier_deletion() { let mut app = PlotxApp::new(); - app.doc - .datasets - .push(Dataset::Nmr2D(Box::new(Nmr2DDataset::load(data_2d( - "unrelated", - ))))); - app.doc - .datasets - .push(Dataset::Nmr2D(Box::new(Nmr2DDataset::load(data_2d( - "target", - ))))); + app.doc.datasets.push(Dataset::Nmr2D(Box::new( + Nmr2DDataset::load(data_2d("unrelated")).unwrap(), + ))); + app.doc.datasets.push(Dataset::Nmr2D(Box::new( + Nmr2DDataset::load(data_2d("target")).unwrap(), + ))); let target_id = app.doc.datasets[1].resource_id(); let before = app.doc.datasets[1].as_nmr2d().unwrap().processed.clone(); let target = app.doc.datasets[1].as_nmr2d_mut().unwrap(); @@ -273,11 +308,9 @@ fn process_2d_result_follows_dataset_identity_after_earlier_deletion() { #[test] fn successful_processing_promotes_fresh_runtime_versions_for_each_scalar_field() { let mut app = PlotxApp::new(); - app.doc - .datasets - .push(Dataset::Nmr2D(Box::new(Nmr2DDataset::load(data_2d( - "versioned target", - ))))); + app.doc.datasets.push(Dataset::Nmr2D(Box::new( + Nmr2DDataset::load(data_2d("versioned target")).unwrap(), + ))); let resource = app.doc.datasets[0].resource_id(); let fields = app.doc.datasets[0] .field_descriptors() diff --git a/crates/core/src/state/app_impl_figures.rs b/crates/core/src/state/app_impl_figures.rs index 6928234d..9235539c 100644 --- a/crates/core/src/state/app_impl_figures.rs +++ b/crates/core/src/state/app_impl_figures.rs @@ -10,7 +10,17 @@ impl PlotxApp { /// datasets are plot-owned and remain visible in encoding-compatible modes. pub fn display_binding(&self, owner: Option, binding: &DataBinding) -> DataBinding { let Some((resource, field)) = self.live_display_source(owner) else { - return binding.clone(); + let mut displayed = binding.clone(); + if let Some(dataset @ Dataset::Nmr2D(_)) = + owner.and_then(|id| self.doc.dataset_by_id(id)) + { + let active = dataset.field_descriptors(); + displayed.series.retain(|series| { + series.source.resource != dataset.resource_id() + || active.iter().any(|field| field.id == series.source.field) + }); + } + return displayed; }; let mut owner_series = binding .series @@ -54,9 +64,14 @@ impl PlotxApp { persisted: &DataBinding, displayed: DataBinding, ) -> DataBinding { - let Some((_resource, _field)) = self.live_display_source(owner) else { + let projected_owner = self.live_display_source(owner).is_some() + || matches!( + owner.and_then(|id| self.doc.dataset_by_id(id)), + Some(Dataset::Nmr2D(_)) + ); + if !projected_owner { return displayed; - }; + } let previous_display = self.display_binding(owner, persisted); let previous_ids = previous_display .series @@ -87,6 +102,45 @@ impl PlotxApp { Some((dataset.resource_id(), field)) } + /// A reconstructed grid is a different field from its acquired observations. + /// Retain authored bindings for switching back and allocate fresh series only + /// when a completed result first exposes another field on a live owner plot. + pub(crate) fn initialize_nmr_result_bindings( + &mut self, + dataset: usize, + previous: Option, + ) { + let dataset = &self.doc.datasets[dataset]; + let Some(field) = dataset + .default_field_id() + .filter(|field| Some(*field) != previous) + else { + return; + }; + let resource = dataset.resource_id(); + let additions = SeriesBinding::from_field_all(dataset, field); + for canvas in &mut self.doc.canvases { + for object in &mut canvas.objects { + let Some(plot) = object.plot_mut().filter(|plot| { + plot.display_owner == Some(resource) + && plot.binding.series.iter().any(|series| { + series.source.resource == resource + && Some(series.source.field) == previous + }) + && !plot.binding.series.iter().any(|series| { + series.source.resource == resource && series.source.field == field + }) + }) else { + continue; + }; + for mut series in additions.clone() { + series.id = plot.allocate_series_id(); + plot.binding.series.push(series); + } + } + } + } + /// Build a dataset's figure through the chart registry: resolve `chart`'s /// type for the dataset's domain (falling back to the domain default when the /// recorded id doesn't apply), then dispatch to its builder. The default chart diff --git a/crates/core/src/state/app_impl_io.rs b/crates/core/src/state/app_impl_io.rs index 001056b3..f7aa469c 100644 --- a/crates/core/src/state/app_impl_io.rs +++ b/crates/core/src/state/app_impl_io.rs @@ -218,14 +218,73 @@ impl PlotxApp { self.load_archive_from(path); return; } + self.install_import_result(path, plotx_io::load_path(path)); + } + + pub fn load_nmr_with_sampling( + &mut self, + path: &std::path::Path, + declaration: plotx_io::nmr_sampling::SamplingDeclaration, + ) -> bool { + self.install_import_result(path, plotx_io::nmr_sampling::load(path, declaration)) + } + + fn install_import_result( + &mut self, + path: &std::path::Path, + result: Result, + ) -> bool { + let prepared = result + .map_err(|error| error.to_string()) + .and_then(|loaded| { + super::data_import::PreparedImport::new( + loaded, + self.settings.general.equal_scale_homonuclear_2d_imports, + ) + }); + self.install_prepared_import(path, prepared) + } + + pub(super) fn install_prepared_import( + &mut self, + path: &std::path::Path, + result: Result, + ) -> bool { let operation_id = self.session.begin_operation(); - match plotx_io::load_path(path) { - Ok(result) => { - let (acquisition, acquisition_identity, format, _, nmr_origin, warnings) = - result.into_parts(); - let format = format.as_str(); - let source = self.insert_acquisition(acquisition, acquisition_identity, nmr_origin); - let mut report = if warnings.is_empty() { + match result { + Ok(prepared) => { + let super::data_import::PreparedImport { + dataset, + source, + format, + warnings, + } = prepared; + let reconstruction_warning = dataset + .as_nmr2d() + .and_then(|data| data.reconstruction_warning.clone()); + let before = self.doc.datasets.len(); + if let Err(error) = self.insert_prepared_dataset(dataset, &source) { + return self.install_prepared_import(path, Err(error)); + } + if self.doc.datasets.len() == before { + return self.install_prepared_import(path, Err(self.session.status.clone())); + } + let mut report = if let Some(warning) = reconstruction_warning { + OperationReport::warning( + operation_id, + OperationKind::DatasetLoad, + format!("Loaded {source}. {warning}"), + (), + ) + .with_diagnostic( + Diagnostic::new( + Severity::Warning, + DiagnosticCode::DatasetLoadWarning, + warning, + ) + .with_source("core.nmr_reconstruction"), + ) + } else if warnings.is_empty() { OperationReport::success( operation_id, OperationKind::DatasetLoad, @@ -246,7 +305,7 @@ impl PlotxApp { DiagnosticCode::DatasetLoadSucceeded, "Dataset loaded", ) - .with_context("format", format) + .with_context("format", format.as_str()) .with_context("path", path.display().to_string()) .with_source("core.loading"), ); @@ -255,6 +314,7 @@ impl PlotxApp { } self.session.status = report.summary.clone(); self.session.record_operation(report); + true } Err(e) => { self.session.status = format!("Failed to load {}: {e}", path.display()); @@ -271,6 +331,7 @@ impl PlotxApp { .with_context("path", path.display().to_string()) .with_source("core.loading"), )); + false } } } @@ -297,13 +358,29 @@ impl PlotxApp { self.session.record_operation(report); return; } - let count = result.items.len(); + let mut count = 0; let mut warnings = result.warnings; for item in result.items { - let (acquisition, acquisition_identity, _, _, nmr_origin, item_warnings) = + let (acquisition, acquisition_identity, _, _, item_warnings) = item.into_parts(); warnings.extend(item_warnings); - self.insert_acquisition(acquisition, acquisition_identity, nmr_origin); + match self.insert_acquisition(acquisition, acquisition_identity) { + Ok((_, warning)) => { + count += 1; + if let Some(message) = warning { + warnings.push(plotx_io::LoadWarning { + code: plotx_io::LoadWarningCode::UnsupportedFunction, + message, + path: None, + }); + } + } + Err(error) => warnings.push(plotx_io::LoadWarning { + code: plotx_io::LoadWarningCode::InvalidMetadata, + message: format!("Archive dataset could not be opened: {error}"), + path: Some(path.to_owned()), + }), + } } let summary = if warnings.is_empty() { format!("Loaded {count} spectra from {archive}") @@ -369,22 +446,29 @@ impl PlotxApp { &mut self, acq: plotx_io::Acquisition, acquisition_identity: plotx_io::AcquisitionIdentity, - nmr_origin: Option, - ) -> String { + ) -> Result<(String, Option), crate::workflow::WorkflowError> { let (dataset, source) = crate::workflow::dataset_from_loaded_acquisition( acq, acquisition_identity, - nmr_origin, self.settings.general.equal_scale_homonuclear_2d_imports, - ); - let name = Self::short_name(&source); - self.execute_action(Action::insert_dataset_with_default_canvas( + )?; + let warning = dataset + .as_nmr2d() + .and_then(|data| data.reconstruction_warning.clone()); + self.insert_prepared_dataset(dataset, &source) + .map_err(crate::workflow::WorkflowError::FieldRuntime)?; + Ok((source, warning)) + } + + fn insert_prepared_dataset(&mut self, dataset: Dataset, source: &str) -> Result<(), String> { + let name = Self::short_name(source); + self.try_execute_action(Action::insert_dataset_with_default_canvas( self, dataset, format!("Canvas {} — {}", self.doc.canvases.len() + 1, name), DEFAULT_CANVAS_SIZE_MM, - )); - source + )) + .map_err(|error| error.to_string()) } pub fn request_export(&mut self, format: ExportFormat) { @@ -565,159 +649,5 @@ fn export_status(format: ExportFormat, paths: &[std::path::PathBuf]) -> String { } #[cfg(test)] -mod export_operation_tests { - use super::*; - use crate::operation::{DiagnosticCode, OperationOutcome}; - - #[test] - fn unavailable_export_is_recorded_and_projects_its_summary() { - let mut app = PlotxApp::new_with_settings(crate::settings::Settings::default()); - - app.request_export(ExportFormat::Svg); - - let operation = app - .session - .operation_history - .operations() - .next_back() - .unwrap(); - assert_eq!(operation.kind, OperationKind::Export); - assert_eq!(operation.outcome, OperationOutcome::Failure); - assert_eq!(operation.summary, app.session.status); - assert_eq!(operation.diagnostics.len(), 1); - assert_eq!( - operation.diagnostics[0].code, - DiagnosticCode::ExportUnavailable - ); - } - - #[test] - fn typed_export_error_is_mapped_at_the_workflow_boundary() { - let mut app = PlotxApp::new_with_settings(crate::settings::Settings::default()); - app.doc.canvases.push(CanvasDocument::new( - "page".to_owned(), - DEFAULT_CANVAS_SIZE_MM, - )); - app.session.active_canvas = Some(0); - - app.export_to( - ExportSettings { - format: ExportFormat::Svg, - scope: crate::export::ExportPageScope::Range { start: 2, end: 1 }, - dpi: crate::export::DEFAULT_BITMAP_DPI, - target_width_mm: None, - trim_to_visible_content: false, - allow_missing_images: false, - }, - std::path::Path::new("unused.svg"), - ); - - let operation = app - .session - .operation_history - .operations() - .next_back() - .unwrap(); - assert_eq!(operation.outcome, OperationOutcome::Failure); - assert_eq!(operation.summary, app.session.status); - assert_eq!(operation.diagnostics[0].code, DiagnosticCode::ExportFailed); - assert_eq!( - operation.diagnostics[0] - .context - .get("error_kind") - .map(String::as_str), - Some("invalid_page_range") - ); - } - - #[test] - fn image_pages_open_export_options_for_precheck_and_placeholder_choice() { - let mut app = PlotxApp::new_with_settings(crate::settings::Settings::default()); - app.doc.canvases.push(CanvasDocument::new( - "clean".to_owned(), - DEFAULT_CANVAS_SIZE_MM, - )); - let mut raster_page = CanvasDocument::new("raster".to_owned(), DEFAULT_CANVAS_SIZE_MM); - let id = raster_page.allocate_object_id(); - raster_page.objects.push(crate::state::CanvasObject { - id, - name: "image".to_owned(), - frame: crate::state::ObjectFrame::new(0.0, 0.0, 10.0, 10.0), - locked: false, - visible: true, - kind: crate::state::CanvasObjectKind::RasterImage( - crate::state::RasterImageContent::new(crate::state::AssetId::new()), - ), - }); - app.doc.canvases.push(raster_page); - app.session.active_canvas = Some(0); - - app.request_export(ExportFormat::Svg); - assert!(app.session.ui.export_options.is_some()); - app.session.ui.export_options = None; - app.session.active_canvas = Some(1); - app.request_export(ExportFormat::Svg); - assert!(app.session.ui.export_options.is_some()); - } -} - -#[cfg(test)] -mod install_loaded_project_tests { - use super::*; - - fn record_failure(app: &mut PlotxApp) -> OperationId { - let id = app.session.begin_operation(); - app.session.record_operation(OperationReport::<()>::failure( - id, - OperationKind::DatasetLoad, - "boom", - Diagnostic::new(Severity::Error, DiagnosticCode::DatasetLoadFailed, "boom"), - )); - id - } - - /// The invariant the feedback watermark hinges on: a project swap carries - /// the operation history *including its counters*, so reports recorded - /// after the load always come after a pre-load acknowledgement. - #[test] - fn project_swap_carries_history_counter_and_watermark() { - let mut app = PlotxApp::new_with_settings(crate::settings::Settings::default()); - let before = record_failure(&mut app); - let before_order = app - .session - .operation_history - .operations() - .next_back() - .expect("failure recorded") - .completion_order; - app.session.ui.dismissed_feedback_order = Some(before_order); - - let loaded = PlotxApp::new_with_settings(crate::settings::Settings::default()); - app.install_loaded_project(loaded); - - assert_eq!(app.session.ui.dismissed_feedback_order, Some(before_order)); - let after = record_failure(&mut app); - let after_order = app - .session - .operation_history - .operations() - .next_back() - .expect("failure recorded") - .completion_order; - assert!( - after > before, - "post-load ids must stay above the watermark" - ); - assert!( - after_order > before_order, - "post-load reports must stay after the acknowledgement" - ); - assert!( - app.session - .operation_history - .operations() - .any(|operation| operation.id == before), - "pre-load history is carried across the swap" - ); - } -} +#[path = "app_impl_io_tests.rs"] +mod tests; diff --git a/crates/core/src/state/app_impl_io_tests.rs b/crates/core/src/state/app_impl_io_tests.rs new file mode 100644 index 00000000..ae9f07cc --- /dev/null +++ b/crates/core/src/state/app_impl_io_tests.rs @@ -0,0 +1,159 @@ +use super::*; + +#[cfg(test)] +mod export_operation_tests { + use super::*; + use crate::operation::{DiagnosticCode, OperationOutcome}; + + #[test] + fn unavailable_export_is_recorded_and_projects_its_summary() { + let mut app = PlotxApp::new_with_settings(crate::settings::Settings::default()); + + app.request_export(ExportFormat::Svg); + + let operation = app + .session + .operation_history + .operations() + .next_back() + .unwrap(); + assert_eq!(operation.kind, OperationKind::Export); + assert_eq!(operation.outcome, OperationOutcome::Failure); + assert_eq!(operation.summary, app.session.status); + assert_eq!(operation.diagnostics.len(), 1); + assert_eq!( + operation.diagnostics[0].code, + DiagnosticCode::ExportUnavailable + ); + } + + #[test] + fn typed_export_error_is_mapped_at_the_workflow_boundary() { + let mut app = PlotxApp::new_with_settings(crate::settings::Settings::default()); + app.doc.canvases.push(CanvasDocument::new( + "page".to_owned(), + DEFAULT_CANVAS_SIZE_MM, + )); + app.session.active_canvas = Some(0); + + app.export_to( + ExportSettings { + format: ExportFormat::Svg, + scope: crate::export::ExportPageScope::Range { start: 2, end: 1 }, + dpi: crate::export::DEFAULT_BITMAP_DPI, + target_width_mm: None, + trim_to_visible_content: false, + allow_missing_images: false, + }, + std::path::Path::new("unused.svg"), + ); + + let operation = app + .session + .operation_history + .operations() + .next_back() + .unwrap(); + assert_eq!(operation.outcome, OperationOutcome::Failure); + assert_eq!(operation.summary, app.session.status); + assert_eq!(operation.diagnostics[0].code, DiagnosticCode::ExportFailed); + assert_eq!( + operation.diagnostics[0] + .context + .get("error_kind") + .map(String::as_str), + Some("invalid_page_range") + ); + } + + #[test] + fn image_pages_open_export_options_for_precheck_and_placeholder_choice() { + let mut app = PlotxApp::new_with_settings(crate::settings::Settings::default()); + app.doc.canvases.push(CanvasDocument::new( + "clean".to_owned(), + DEFAULT_CANVAS_SIZE_MM, + )); + let mut raster_page = CanvasDocument::new("raster".to_owned(), DEFAULT_CANVAS_SIZE_MM); + let id = raster_page.allocate_object_id(); + raster_page.objects.push(crate::state::CanvasObject { + id, + name: "image".to_owned(), + frame: crate::state::ObjectFrame::new(0.0, 0.0, 10.0, 10.0), + locked: false, + visible: true, + kind: crate::state::CanvasObjectKind::RasterImage( + crate::state::RasterImageContent::new(crate::state::AssetId::new()), + ), + }); + app.doc.canvases.push(raster_page); + app.session.active_canvas = Some(0); + + app.request_export(ExportFormat::Svg); + assert!(app.session.ui.export_options.is_some()); + app.session.ui.export_options = None; + app.session.active_canvas = Some(1); + app.request_export(ExportFormat::Svg); + assert!(app.session.ui.export_options.is_some()); + } +} + +#[cfg(test)] +mod install_loaded_project_tests { + use super::*; + + fn record_failure(app: &mut PlotxApp) -> OperationId { + let id = app.session.begin_operation(); + app.session.record_operation(OperationReport::<()>::failure( + id, + OperationKind::DatasetLoad, + "boom", + Diagnostic::new(Severity::Error, DiagnosticCode::DatasetLoadFailed, "boom"), + )); + id + } + + /// The invariant the feedback watermark hinges on: a project swap carries + /// the operation history *including its counters*, so reports recorded + /// after the load always come after a pre-load acknowledgement. + #[test] + fn project_swap_carries_history_counter_and_watermark() { + let mut app = PlotxApp::new_with_settings(crate::settings::Settings::default()); + let before = record_failure(&mut app); + let before_order = app + .session + .operation_history + .operations() + .next_back() + .expect("failure recorded") + .completion_order; + app.session.ui.dismissed_feedback_order = Some(before_order); + + let loaded = PlotxApp::new_with_settings(crate::settings::Settings::default()); + app.install_loaded_project(loaded); + + assert_eq!(app.session.ui.dismissed_feedback_order, Some(before_order)); + let after = record_failure(&mut app); + let after_order = app + .session + .operation_history + .operations() + .next_back() + .expect("failure recorded") + .completion_order; + assert!( + after > before, + "post-load ids must stay above the watermark" + ); + assert!( + after_order > before_order, + "post-load reports must stay after the acknowledgement" + ); + assert!( + app.session + .operation_history + .operations() + .any(|operation| operation.id == before), + "pre-load history is carried across the swap" + ); + } +} diff --git a/crates/core/src/state/app_impl_multiplet.rs b/crates/core/src/state/app_impl_multiplet.rs index fe53e336..6663ea42 100644 --- a/crates/core/src/state/app_impl_multiplet.rs +++ b/crates/core/src/state/app_impl_multiplet.rs @@ -35,7 +35,16 @@ impl PlotxApp { let Some(n) = self.doc.datasets.get(dataset).and_then(Dataset::as_nmr) else { return Err("Multiplet analysis needs a 1D NMR dataset.".to_owned()); }; - let obs = n.data.observe_freq_mhz; + let spectrum = n + .spectrum() + .ok_or("Multiplet analysis requires a spectrum")?; + if spectrum.unit != nmr::axis::AxisUnit::Ppm { + return Err("Use a spectrum calibrated in ppm before analyzing multiplets.".into()); + } + let obs = n + .native_processed + .reference_frequency_mhz(0) + .ok_or("Multiplet analysis in ppm requires chemical-shift reference evidence")?; let mut peaks: Vec = Vec::new(); let mut areas: Vec = Vec::new(); diff --git a/crates/core/src/state/app_impl_slice.rs b/crates/core/src/state/app_impl_slice.rs index 71644f41..8ecc319f 100644 --- a/crates/core/src/state/app_impl_slice.rs +++ b/crates/core/src/state/app_impl_slice.rs @@ -1,114 +1,86 @@ use super::*; -use plotx_processing::{Processed1D, ProjectionMode, Slice1D, SliceKind}; +use plotx_processing::{ProjectionMode, Slice1D, SliceKind}; +use std::sync::Arc; impl NmrDataset { - /// Build a standalone 1D trace from a slice/projection lifted out of a 2D - /// dataset without changing its scientific domain. - pub fn from_slice(slice: Slice1D, source: String) -> Self { - let Slice1D { - coordinates, - domain, - values, - nucleus, - observe_freq_mhz, - .. - } = slice; - let (spectral_width_hz, carrier_ppm) = match domain { - plotx_io::Domain::Frequency => linear_axis_params(&coordinates, observe_freq_mhz), - plotx_io::Domain::Time => (time_axis_spectral_width(&coordinates), 0.0), + /// Explicit coordinates are retained for a standalone programmatic trace. + pub fn from_slice(slice: Slice1D, source: String) -> Result { + use nmr::axis::{AxisCoordinates, AxisDomain, AxisRole, AxisUnit, FrequencyEvidence}; + use nmr::processed::{ + ComponentBasis, ProcessedAxis, ProcessedDataset, ProcessedOrigin, ProcessedProvenance, }; - let data = NmrData { - points: values.clone(), - domain, - spectral_width_hz, - observe_freq_mhz, - carrier_ppm, - nucleus: nucleus.clone(), - source: source.clone(), - group_delay: 0.0, + let fail = |error: &dyn std::fmt::Display| error.to_string(); + let (domain, unit) = match slice.domain { + Domain::Time => (AxisDomain::Time, AxisUnit::Second), + Domain::Frequency => (AxisDomain::Frequency, slice.unit), }; - let group_delay_correct = super::default_group_delay_correct(data.domain); - let pipeline = AxisPipeline { steps: Vec::new() }; - let processed = match domain { - plotx_io::Domain::Frequency => { - let n = coordinates.len().max(1); - Processed1D::Frequency(Spectrum { - ppm: coordinates, - values, - hz_per_point: (spectral_width_hz / n as f64).abs(), - observe_freq_mhz, - nucleus, - }) - } - plotx_io::Domain::Time => Processed1D::Time(plotx_processing::TimeTrace { - time_s: coordinates, - values, - nucleus, - source: source.clone(), + let reference = (unit == AxisUnit::Ppm) + .then_some(slice.reference_freq_mhz) + .flatten(); + let coordinates = if let Some(frequency) = reference { + slice + .coordinates + .into_iter() + .map(|value| value * frequency) + .collect() + } else { + slice.coordinates + }; + let axis = ProcessedAxis::new( + AxisRole::Signal, + domain, + Some(if reference.is_some() { + AxisUnit::Hertz + } else { + unit }), + slice.values.len(), + AxisCoordinates::Explicit(coordinates), + ComponentBasis::Cartesian, + ) + .map_err(|error| fail(&error))? + .with_nucleus((!slice.nucleus.is_empty()).then_some(slice.nucleus)) + .map_err(|error| fail(&error))? + .with_frequency_evidence(Some( + FrequencyEvidence::new(slice.observe_freq_mhz, None).map_err(|error| fail(&error))?, + )) + .map_err(|error| fail(&error))?; + let data = ProcessedDataset::from_complex_trace( + axis, + slice.values, + ProcessedProvenance::new(ProcessedOrigin::Unknown, Vec::new()) + .map_err(|error| fail(&error))?, + ) + .map_err(|error| fail(&error))?; + let data = if let Some(frequency) = reference { + use nmr::processing::{ + FrequencyFrame, ProcessingOperation, ProcessingPlan, ReferenceSource, + }; + ProcessingPlan::new(vec![ProcessingOperation::ResolveFrequencyFrame { + axis: 0, + frame: FrequencyFrame::Ppm(ReferenceSource::Explicit( + nmr::raw::ChemicalShiftReference::user_constructed(0.0, frequency) + .map_err(|error| fail(&error))?, + )), + }]) + .map_err(|error| fail(&error))? + .apply(&data.into()) + .map_err(|error| fail(&error))? + } else { + data.into() }; - let mut field_catalog = nmr_field_catalog(); - field_catalog.attach_provenance(&data.source, None); - Self { - resource_id: DatasetId::new(), - field_catalog, - acquisition_identity: plotx_io::AcquisitionIdentity { - subject: None, - acquisition: None, - source_label: source.clone(), - }, - data, - origin: plotx_io::NmrOrigin::Derived, - base: processed.clone(), - pipeline, - next_step_id: 0, - group_delay_correct, - has_imaginary: true, - processed, - name: Some(source), - lineage: None, - peaks: PeakSet::default(), - integrals: Vec::new(), - next_integral_id: 0, - line_fits: Vec::new(), - next_line_fit_id: 0, - multiplets: Vec::new(), - next_multiplet_id: 0, - craft_runs: Vec::new(), - next_craft_run_id: 0, - craft_spectrum_cache: Default::default(), - } - } -} - -/// Spectral width and carrier (ppm) that make [`fft::transform_base`] reproduce a -/// linear ppm axis `p`: `ppm[i] = carrier + (i − n/2)·sw/(n·obs)`. -fn linear_axis_params(ppm: &[f64], obs: f64) -> (f64, f64) { - let n = ppm.len(); - if n < 2 { - return ( - obs.max(f64::MIN_POSITIVE), - ppm.first().copied().unwrap_or(0.0), - ); - } - let dp = (ppm[n - 1] - ppm[0]) / (n - 1) as f64; - let sw = dp * n as f64 * obs; - let carrier = ppm[0] + (n as f64 / 2.0) * dp; - (sw, carrier) -} - -fn time_axis_spectral_width(time_s: &[f64]) -> f64 { - let Some((&first, &last)) = time_s.first().zip(time_s.last()) else { - return 1.0; - }; - if time_s.len() < 2 { - return 1.0; - } - let dwell = (last - first).abs() / (time_s.len() - 1) as f64; - if dwell.is_finite() && dwell > f64::MIN_POSITIVE { - 1.0 / dwell - } else { - 1.0 + let input = plotx_io::nmr_view::NmrSource::new(Arc::new(data)) + .map_err(|error| fail(&error))? + .with_display_label(source.clone()); + let mut dataset = + Self::load_with_pipeline(input, Some(AxisPipeline { steps: Vec::new() }), Some(false))?; + dataset.acquisition_identity = plotx_io::AcquisitionIdentity { + source_label: source.clone(), + subject: None, + acquisition: None, + }; + dataset.name = Some(source); + Ok(dataset) } } @@ -125,12 +97,25 @@ impl PlotxApp { return; }; let parent = self.doc.datasets[dataset].display_name(); - let (slice, is_stack) = match &d2.processed { - Processed2D::Ft(s) => (s.slice(cursor.kind, cursor.index), false), - Processed2D::Stack(s) => (s.slice(cursor.index), true), + let is_stack = matches!(d2.processed, Processed2D::Stack(_)); + let kind = if is_stack { + SliceKind::Row + } else { + cursor.kind + }; + let (source, slice) = match plotx_processing::slice::extract( + &d2.native_processed, + kind, + plotx_processing::slice::Reduction::Slice(cursor.index), + ) { + Ok(output) => output, + Err(error) => { + self.session.status = format!("Slice extraction failed: {error}"); + return; + } }; - let name = slice_name(&parent, &slice, cursor.kind, is_stack, cursor.index); - self.insert_slice_dataset(slice, name, dataset, DerivationKind::Slice); + let name = slice_name(&parent, &slice, kind, is_stack, cursor.index); + self.insert_slice_dataset(source, name, dataset, DerivationKind::Slice); } /// Materialize a whole-axis projection of a true-2D spectrum as a new 1D @@ -144,28 +129,51 @@ impl PlotxApp { let Some(d2) = self.doc.datasets.get(dataset).and_then(Dataset::as_nmr2d) else { return; }; - let Processed2D::Ft(s) = &d2.processed else { + let Processed2D::Ft(_) = &d2.processed else { self.session.status = "Projections are available for true-2D spectra.".into(); return; }; let parent = self.doc.datasets[dataset].display_name(); - let slice = s.project(kind, mode); + let source = match plotx_processing::slice::extract( + &d2.native_processed, + kind, + plotx_processing::slice::Reduction::Projection(mode), + ) { + Ok((source, _)) => source, + Err(error) => { + self.session.status = format!("Projection failed: {error}"); + return; + } + }; let word = match mode { ProjectionMode::Sum => "sum", ProjectionMode::Skyline => "skyline", }; let name = format!("{parent} — {} {word} projection", slice_axis_label(kind)); - self.insert_slice_dataset(slice, name, dataset, DerivationKind::Projection); + self.insert_slice_dataset(source, name, dataset, DerivationKind::Projection); } fn insert_slice_dataset( &mut self, - slice: Slice1D, + source_data: plotx_io::nmr_view::NmrSource, name: String, source: usize, kind: DerivationKind, ) { - let mut ds = Dataset::Nmr(Box::new(NmrDataset::from_slice(slice, name.clone()))); + let dataset = match NmrDataset::load_with_pipeline( + source_data, + Some(AxisPipeline { steps: Vec::new() }), + Some(false), + ) { + Ok(dataset) => dataset, + Err(error) => { + self.session.status = format!("Slice extraction failed: {error}"); + return; + } + }; + let mut dataset = dataset; + dataset.name = Some(name.clone()); + let mut ds = Dataset::Nmr(Box::new(dataset)); ds.set_lineage(Some(DatasetLineage::new( kind, [self.doc.datasets[source].resource_id()], @@ -227,7 +235,9 @@ mod tests { domain: plotx_io::Domain::Frequency, values: vec![Complex64::new(1.0, 0.0), Complex64::new(0.5, 0.0)], nucleus: "1H".to_owned(), - observe_freq_mhz: 400.0, + observe_freq_mhz: Some(400.0), + reference_freq_mhz: Some(400.0), + unit: nmr::axis::AxisUnit::Ppm, position: Some(3.0), position_domain: plotx_io::Domain::Frequency, } @@ -236,16 +246,26 @@ mod tests { #[test] fn slice_and_projection_insertions_record_the_source() { let mut app = PlotxApp::new(); - app.doc - .datasets - .push(Dataset::Nmr(Box::new(NmrDataset::from_slice( - slice(), - "source".to_owned(), - )))); + app.doc.datasets.push(Dataset::Nmr(Box::new( + NmrDataset::from_slice(slice(), "source".to_owned()).unwrap(), + ))); - app.insert_slice_dataset(slice(), "slice".to_owned(), 0, DerivationKind::Slice); app.insert_slice_dataset( - slice(), + app.doc.datasets[0] + .as_nmr() + .unwrap() + .native_processed + .clone(), + "slice".to_owned(), + 0, + DerivationKind::Slice, + ); + app.insert_slice_dataset( + app.doc.datasets[0] + .as_nmr() + .unwrap() + .native_processed + .clone(), "projection".to_owned(), 0, DerivationKind::Projection, @@ -269,11 +289,11 @@ mod tests { #[test] fn frequency_domain_slices_share_the_factory_group_delay_default() { - let dataset = NmrDataset::from_slice(slice(), "slice".to_owned()); + let dataset = NmrDataset::from_slice(slice(), "slice".to_owned()).unwrap(); assert!(!dataset.group_delay_correct); assert_eq!( dataset.group_delay_correct, - default_group_delay_correct(dataset.data.domain) + default_group_delay_correct(&dataset.data) ); } @@ -282,8 +302,8 @@ mod tests { let mut time = slice(); time.coordinates = vec![0.0, 0.002]; time.domain = plotx_io::Domain::Time; - let dataset = NmrDataset::from_slice(time, "FID slice".to_owned()); - assert_eq!(dataset.data.domain, plotx_io::Domain::Time); + let dataset = NmrDataset::from_slice(time, "FID slice".to_owned()).unwrap(); + assert_eq!(dataset.input_domain(), plotx_io::Domain::Time); assert_eq!(dataset.output_domain(), plotx_io::Domain::Time); assert_eq!(dataset.time_trace().unwrap().time_s, vec![0.0, 0.002]); } diff --git a/crates/core/src/state/compute.rs b/crates/core/src/state/compute.rs index ca051496..3765ede5 100644 --- a/crates/core/src/state/compute.rs +++ b/crates/core/src/state/compute.rs @@ -1,5 +1,5 @@ +use nmr::CancellationToken; use std::collections::HashMap; -use std::sync::atomic::{AtomicBool, Ordering}; use std::sync::mpsc::{Receiver, Sender}; use std::sync::{Arc, Mutex}; use std::thread; @@ -8,11 +8,12 @@ use std::time::{Duration, Instant}; use plotx_analysis::diffusion::{DiffusionMap, diffusion_map_cancellable}; use plotx_analysis::ilt::{IltResult, ilt_map_cancellable}; use plotx_figure::Figure; -use plotx_io::{DiffusionMeta, NmrData, NmrData2D}; +use plotx_io::{DiffusionMeta, NmrData, nmr_view::NmrSource}; use plotx_processing::{ Params2D, Processed2D, StackSpectrum, craft::{CraftInvocation, CraftResult, process_craft_cancellable}, - process_2d_cancellable, reapply_2d_cancellable, + nmr_bridge::{DelayPolicy, RecipeRange}, + nmr_execution::{NusRequest, Output2D, execute_2d}, }; use super::{ @@ -93,7 +94,7 @@ enum Job { generation: u64, dataset: DatasetId, epoch: u64, - token: Arc, + token: CancellationToken, stack: Arc, b_factors: Vec, d_grid: Vec, @@ -110,7 +111,7 @@ enum Job { generation: u64, dataset: DatasetId, epoch: u64, - token: Arc, + token: CancellationToken, stack: Arc, values: Vec, meta: DiffusionMeta, @@ -121,7 +122,7 @@ enum Job { generation: u64, dataset: DatasetId, epoch: u64, - token: Arc, + token: CancellationToken, data: Arc, invocation: Box, parent_run: Option, @@ -129,7 +130,7 @@ enum Job { Process2D { version: FieldVersion, dataset: DatasetId, - token: Arc, + token: CancellationToken, input: ProcessingInput, params: Params2D, fields: Vec, @@ -150,9 +151,15 @@ enum ProcessingInputKind { Reapply, } +pub(crate) struct Full2DInput { + pub source: NmrSource, + pub delay: DelayPolicy, + pub nus: Option, +} + enum ProcessingInput { - Full(Arc), - Reapply(Processed2D), + Full(Full2DInput), + Reapply(NmrSource), } impl ProcessingInput { @@ -175,7 +182,7 @@ struct DeferredProcessing { struct ActiveJob { generation: u64, started_at: Instant, - token: Arc, + token: CancellationToken, processing_input: Option, } @@ -217,11 +224,16 @@ pub enum Done { Processing2D { version: FieldVersion, dataset: DatasetId, - base: Option, - processed: Processed2D, + base: Option, + processed: Output2D, fields: Vec, params: Params2D, }, + Processing2DFailed { + version: FieldVersion, + dataset: DatasetId, + message: String, + }, EstimateField { key: EstimateKey, result: EstimateResult, @@ -316,13 +328,13 @@ impl ComputeService { return Err(EnqueueError::Busy(kind)); } let generation = self.next_generation(dataset, ComputeKind::Ilt); - let token = Arc::new(AtomicBool::new(false)); + let token = CancellationToken::new(); self.active.insert( (dataset, ComputeKind::Ilt), ActiveJob { generation, started_at: Instant::now(), - token: Arc::clone(&token), + token: token.clone(), processing_input: None, }, ); @@ -366,13 +378,13 @@ impl ComputeService { return Err(EnqueueError::Busy(kind)); } let generation = self.next_generation(dataset, ComputeKind::Dosy); - let token = Arc::new(AtomicBool::new(false)); + let token = CancellationToken::new(); self.active.insert( (dataset, ComputeKind::Dosy), ActiveJob { generation, started_at: Instant::now(), - token: Arc::clone(&token), + token: token.clone(), processing_input: None, }, ); @@ -409,13 +421,13 @@ impl ComputeService { return Err(EnqueueError::Busy(kind)); } let generation = self.next_generation(dataset, ComputeKind::Craft); - let token = Arc::new(AtomicBool::new(false)); + let token = CancellationToken::new(); self.active.insert( (dataset, ComputeKind::Craft), ActiveJob { generation, started_at: Instant::now(), - token: Arc::clone(&token), + token: token.clone(), processing_input: None, }, ); @@ -444,7 +456,7 @@ impl ComputeService { &mut self, dataset: DatasetId, fields: &[ProcessingField], - data: Arc, + data: Full2DInput, params: Params2D, ) -> Result, FieldEnqueueError> { self.request_2d(dataset, fields, ProcessingInput::Full(data), params) @@ -456,7 +468,7 @@ impl ComputeService { &mut self, dataset: DatasetId, fields: &[ProcessingField], - base: Processed2D, + base: NmrSource, params: Params2D, ) -> Result, FieldEnqueueError> { self.request_2d(dataset, fields, ProcessingInput::Reapply(base), params) @@ -533,14 +545,14 @@ impl ComputeService { let Some(request) = self.deferred_processing.remove(&dataset) else { continue; }; - let token = Arc::new(AtomicBool::new(false)); + let token = CancellationToken::new(); let input_kind = request.input.kind(); self.active.insert( (dataset, ComputeKind::Processing2D), ActiveJob { generation: request.version.0, started_at: Instant::now(), - token: Arc::clone(&token), + token: token.clone(), processing_input: Some(input_kind), }, ); @@ -586,6 +598,7 @@ impl ComputeService { | Done::Craft { .. } | Done::CraftFailed { .. } | Done::Processing2D { .. } + | Done::Processing2DFailed { .. } | Done::Cancelled { .. } | Done::Failed { .. } => {} } @@ -596,12 +609,15 @@ impl ComputeService { .active .get(&(dataset, kind)) .filter(|active| active.generation == generation); + if kind == ComputeKind::Processing2D && matching_active.is_none() { + continue; + } // A worker can send success immediately before cancellation. Check // the shared token again on the receiving side so explicit cancel, // Full/Reapply replacement, and dataset invalidation cannot install // that already-queued success. let cancelled_after_send = - matching_active.is_some_and(|active| active.token.load(Ordering::Relaxed)); + matching_active.is_some_and(|active| active.token.is_cancelled()); if matching_active.is_some() { self.active.remove(&(dataset, kind)); } @@ -621,9 +637,9 @@ impl ComputeService { } pub fn progress(&self, dataset: DatasetId, kind: ComputeKind) -> Option { - self.active.get(&(dataset, kind)).and_then(|active| { - (!active.token.load(Ordering::Relaxed)).then(|| active.started_at.elapsed()) - }) + self.active + .get(&(dataset, kind)) + .and_then(|active| (!active.token.is_cancelled()).then(|| active.started_at.elapsed())) } /// Return the active DOSY computation regardless of which method the UI is @@ -645,7 +661,7 @@ impl ComputeService { self.active .iter() .find(|((active_dataset, _), active)| { - *active_dataset == dataset && !active.token.load(Ordering::Relaxed) + *active_dataset == dataset && !active.token.is_cancelled() }) .map(|((_, kind), _)| *kind) } @@ -671,7 +687,8 @@ impl ComputeService { if compatible_reapply { continue; } - let running = !active.token.swap(true, Ordering::Relaxed); + let running = !active.token.is_cancelled(); + active.token.cancel(); if running && *kind != ComputeKind::Processing2D { aborted.push(*kind); } @@ -685,7 +702,7 @@ impl ComputeService { pub fn cancel(&mut self, dataset: DatasetId, kind: ComputeKind) -> bool { let mut cancelled = false; if let Some(active) = self.active.get(&(dataset, kind)) { - active.token.store(true, Ordering::Relaxed); + active.token.cancel(); cancelled = true; } if kind == ComputeKind::Processing2D && self.deferred_processing.remove(&dataset).is_some() @@ -745,6 +762,9 @@ fn done_identity(done: &Done) -> Option<(DatasetId, ComputeKind, u64)> { } => Some((*dataset, ComputeKind::Craft, *generation)), Done::Processing2D { dataset, version, .. + } + | Done::Processing2DFailed { + dataset, version, .. } => Some((*dataset, ComputeKind::Processing2D, version.0)), Done::Cancelled { dataset, diff --git a/crates/core/src/state/compute/tests.rs b/crates/core/src/state/compute/tests.rs index 14de6ccb..24065107 100644 --- a/crates/core/src/state/compute/tests.rs +++ b/crates/core/src/state/compute/tests.rs @@ -1,12 +1,12 @@ use super::*; use num_complex::Complex64; -use plotx_io::{Dim, Domain, QuadMode}; -use plotx_processing::{PhaseParams, Preset2D, ProcessingStep, StepKind, process_2d}; +use plotx_io::{Dim, Domain, NmrData2D, QuadMode}; +use plotx_processing::{PhaseParams, Preset2D, ProcessingStep, StepKind}; fn dataset(value: u128) -> DatasetId { DatasetId::from_uuid(uuid::Uuid::from_u128(value)) } -fn data_2d() -> Arc { +fn data_2d() -> Full2DInput { let dim = Dim { spectral_width_hz: 1000.0, observe_freq_mhz: 100.0, @@ -14,7 +14,7 @@ fn data_2d() -> Arc { nucleus: "X".into(), group_delay: 0.0, }; - Arc::new(NmrData2D { + let data = NmrData2D { data: (0..16) .map(|i| Complex64::new((i + 1) as f64, 0.0)) .collect(), @@ -30,7 +30,13 @@ fn data_2d() -> Arc { diffusion: None, nus: None, source: "test".into(), - }) + }; + let source = plotx_io::nmr_series::NmrSeriesSource::try_from(data).unwrap(); + Full2DInput { + source: source.source_dataset().clone(), + delay: DelayPolicy::AxisEvidence, + nus: None, + } } fn stack_spectrum() -> Arc { @@ -40,7 +46,8 @@ fn stack_spectrum() -> Arc { traces: vec![vec![Complex64::new(1.0, 0.0); 1]; 3], direct: plotx_processing::AxisMeta { nucleus: "X".into(), - observe_freq_mhz: 100.0, + observe_freq_mhz: Some(100.0), + unit: Some(nmr::axis::AxisUnit::Ppm), }, source: "test".into(), }) @@ -140,15 +147,24 @@ fn reapply_to_reapply_keeps_the_active_job_and_replaces_the_deferred_recipe() { let mut service = ComputeService::new(); let preset = Preset2D::Cosy; let mut first = Params2D::default_for(preset); - let base = process_2d(&data_2d(), &first); + let base = execute_2d( + &data_2d().source, + &first, + DelayPolicy::AxisEvidence, + RecipeRange::Base, + None, + &mut nmr::ExecutionContext::default(), + ) + .unwrap() + .source; - let token = Arc::new(AtomicBool::new(false)); + let token = CancellationToken::new(); service.active.insert( (dataset(0), ComputeKind::Processing2D), ActiveJob { generation: 10, started_at: Instant::now(), - token: Arc::clone(&token), + token: token.clone(), processing_input: Some(ProcessingInputKind::Reapply), }, ); @@ -162,26 +178,26 @@ fn reapply_to_reapply_keeps_the_active_job_and_replaces_the_deferred_recipe() { service .request_2d_reapply(dataset(0), &fields, base.clone(), first) .unwrap(); - assert!(!token.load(Ordering::Relaxed)); + assert!(!token.is_cancelled()); let first_version = service.deferred_processing[&dataset(0)].version; service .request_2d_reapply(dataset(0), &fields, base, Params2D::default_for(preset)) .unwrap(); - assert!(!token.load(Ordering::Relaxed)); + assert!(!token.is_cancelled()); assert!(service.deferred_processing[&dataset(0)].version > first_version); } #[test] fn any_full_retransform_cancels_an_active_reapply() { let mut service = ComputeService::new(); - let token = Arc::new(AtomicBool::new(false)); + let token = CancellationToken::new(); service.active.insert( (dataset(0), ComputeKind::Processing2D), ActiveJob { generation: 10, started_at: Instant::now(), - token: Arc::clone(&token), + token: token.clone(), processing_input: Some(ProcessingInputKind::Reapply), }, ); @@ -196,7 +212,7 @@ fn any_full_retransform_cancels_an_active_reapply() { Params2D::default_for(preset), ) .unwrap(); - assert!(token.load(Ordering::Relaxed)); + assert!(token.is_cancelled()); assert!(matches!( service.deferred_processing[&dataset(0)].input, ProcessingInput::Full(_) @@ -320,7 +336,8 @@ fn cancelling_processing_discards_its_result_and_releases_the_service() { #[test] fn cancelled_ilt_job_reports_acknowledgement_without_a_result() { - let token = Arc::new(AtomicBool::new(true)); + let token = CancellationToken::new(); + token.cancel(); let stack = stack_spectrum(); let done = run_job(Job::Ilt { generation: 7, diff --git a/crates/core/src/state/compute_worker.rs b/crates/core/src/state/compute_worker.rs index cee37552..6a89da45 100644 --- a/crates/core/src/state/compute_worker.rs +++ b/crates/core/src/state/compute_worker.rs @@ -20,7 +20,7 @@ pub(super) fn run_job(job: Job) -> Done { nucleus, source, } => { - let cancelled = || token.load(Ordering::Relaxed); + let cancelled = || token.is_cancelled(); let provenance = ilt_provenance(&stack, &values, &meta, params); match ilt_map_cancellable(&*stack, &b_factors, &d_grid, lambda, &cancelled) { Some(result) if !cancelled() => { @@ -62,7 +62,7 @@ pub(super) fn run_job(job: Job) -> Done { nucleus, source, } => { - let cancelled = || token.load(Ordering::Relaxed); + let cancelled = || token.is_cancelled(); let provenance = mono_exp_provenance(&stack, &values, &meta); match diffusion_map_cancellable(&*stack, &values, &meta, MONO_EXP_SNR_FRAC, &cancelled) { @@ -102,7 +102,7 @@ pub(super) fn run_job(job: Job) -> Done { invocation, parent_run, } => { - let cancelled = || token.load(Ordering::Relaxed); + let cancelled = || token.is_cancelled(); match process_craft_cancellable(&data, &invocation, &cancelled) { Ok(result) if !cancelled() => Done::Craft { generation, @@ -133,28 +133,61 @@ pub(super) fn run_job(job: Job) -> Done { params, fields, } => { - let cancelled = || token.load(Ordering::Relaxed); - let (base, processed) = match input { - ProcessingInput::Full(data) => { - let Some(base) = process_2d_cancellable(&data, ¶ms, &cancelled) else { - return cancelled_done(version.0, dataset); - }; - let Some(processed) = reapply_2d_cancellable(&base, ¶ms, &cancelled) else { - return cancelled_done(version.0, dataset); - }; - (Some(base), processed) + let cancelled = || token.is_cancelled(); + let mut work = plotx_processing::nmr_execution::processing_2d_work_ledger(); + let mut context = + nmr::ExecutionContext::new(&mut work).with_cancellation(token.clone()); + let result = (|| match input { + ProcessingInput::Full(input) => { + let base = execute_2d( + &input.source, + ¶ms, + input.delay, + RecipeRange::Base, + input.nus, + &mut context, + )?; + let processed = execute_2d( + &base.source, + ¶ms, + DelayPolicy::Disabled, + RecipeRange::Frequency, + None, + &mut context, + )?; + Ok((Some(base), processed)) } ProcessingInput::Reapply(base) => { - let Some(processed) = reapply_2d_cancellable(&base, ¶ms, &cancelled) else { - return cancelled_done(version.0, dataset); + let processed = execute_2d( + &base, + ¶ms, + DelayPolicy::Disabled, + RecipeRange::Frequency, + None, + &mut context, + )?; + Ok((None, processed)) + } + })(); + let (base, processed) = match result { + Ok(output) => output, + Err(error) + if plotx_processing::nmr_execution::ExecutionError::is_cancelled(&error) => + { + return cancelled_done(version.0, dataset); + } + Err(error) => { + return Done::Processing2DFailed { + version, + dataset, + message: error.to_string(), }; - (None, processed) } }; if cancelled() { return cancelled_done(version.0, dataset); } - let fields = processed_field_artifacts(&processed, &fields); + let fields = processed_field_artifacts(&processed.view, &fields); Done::Processing2D { version, dataset, diff --git a/crates/core/src/state/contour_budget_tests.rs b/crates/core/src/state/contour_budget_tests.rs index fc43dc34..1cb8c1fa 100644 --- a/crates/core/src/state/contour_budget_tests.rs +++ b/crates/core/src/state/contour_budget_tests.rs @@ -12,8 +12,8 @@ use super::compute_field::run_build_contour; use crate::state::{ AxisSampling, ChartSpec, ContourGeometryCacheKey, DataBinding, DataDomain, Dataset, DatasetId, - FieldId, FieldRef, FieldVersion, FiniteF64, Nmr2DDataset, PlotxApp, ResolvedContourLevels, - ScalarGrid2D, StackSpec, VersionedFieldRef, + FieldId, FieldRef, FieldVersion, FiniteF64, PlotxApp, ResolvedContourLevels, ScalarGrid2D, + StackSpec, VersionedFieldRef, }; use num_complex::Complex64; use plotx_figure::{ @@ -180,24 +180,27 @@ fn dense_dataset(label: &str, values: &[f32]) -> Dataset { nucleus: nucleus.to_owned(), group_delay: 0.0, }; - Dataset::Nmr2D(Box::new(Nmr2DDataset::load(NmrData2D { - data: values - .iter() - .map(|value| Complex64::new(f64::from(*value), 0.0)) - .collect(), - rows: SIDE, - cols: SIDE, - domain: Domain::Frequency, - direct: dimension("1H"), - indirect: dimension("13C"), - quad: QuadMode::Complex, - indirect_conjugate: false, - experiment: None, - pseudo_axis: None, - diffusion: None, - nus: None, - source: label.to_owned(), - }))) + Dataset::Nmr2D(Box::new( + crate::nmr_test_support::load_2d(NmrData2D { + data: values + .iter() + .map(|value| Complex64::new(f64::from(*value), 0.0)) + .collect(), + rows: SIDE, + cols: SIDE, + domain: Domain::Frequency, + direct: dimension("1H"), + indirect: dimension("13C"), + quad: QuadMode::Complex, + indirect_conjugate: false, + experiment: None, + pseudo_axis: None, + diffusion: None, + nus: None, + source: label.to_owned(), + }) + .unwrap(), + )) } #[test] diff --git a/crates/core/src/state/craft.rs b/crates/core/src/state/craft.rs index a2da6781..7b54d5b4 100644 --- a/crates/core/src/state/craft.rs +++ b/crates/core/src/state/craft.rs @@ -1,12 +1,11 @@ use super::{FloatSeries, NmrDataset, TableDataset, materialized_float_series_table}; -use plotx_io::NmrData; +use plotx_io::nmr_view::NmrSource; use plotx_processing::craft::{ CRAFT_ALGORITHM, CRAFT_ALGORITHM_VERSION, CraftAmplitudeReport, CraftComponent, CraftDiagnostics, CraftInvocation, CraftReference, CraftRegionRatio, CraftRegionSummary, CraftReportDefinition, CraftResult, calculate_craft_report, }; use serde::{Deserialize, Serialize}; -use sha2::{Digest, Sha256}; #[derive(Clone, Copy, Debug, PartialEq, Eq, Hash, Serialize, Deserialize)] pub struct CraftRunId(pub u64); @@ -46,7 +45,7 @@ impl StoredCraftRun { } pub fn from_result( id: CraftRunId, - data: &NmrData, + data: &NmrSource, invocation: CraftInvocation, parent_run: Option, result: CraftResult, @@ -68,20 +67,27 @@ impl StoredCraftRun { } } - pub fn is_stale_for(&self, data: &NmrData, reference: CraftReference) -> bool { + pub fn is_stale_for(&self, data: &NmrSource, reference: Option) -> bool { self.provenance.input_sha256 != craft_input_sha256(data) - || self.provenance.invocation.reference != reference + || Some(self.provenance.invocation.reference) != reference } } impl NmrDataset { /// Reference context used by analyses that fit the original FID but report /// chemical shifts on the processed spectrum's visible axis. - pub fn craft_reference(&self) -> CraftReference { - CraftReference::new( - self.data.carrier_ppm, + pub fn craft_reference(&self) -> Option { + let raw = self.data.dataset().as_raw()?; + let reference = raw + .descriptor() + .axes() + .first()? + .chemical_shift_reference()?; + Some(CraftReference::new( + reference.carrier_ppm(), + reference.reference_frequency_mhz(), self.pipeline.chemical_shift_reference_offset_ppm(), - ) + )) } pub fn allocate_craft_run_id(&mut self) -> CraftRunId { @@ -114,27 +120,14 @@ impl NmrDataset { } } -pub fn craft_input_sha256(data: &NmrData) -> String { - let mut digest = Sha256::new(); - digest.update(b"plotx.craft.input.v1\0"); - digest.update([match data.domain { - plotx_io::Domain::Time => 0, - plotx_io::Domain::Frequency => 1, - }]); - for value in [ - data.spectral_width_hz, - data.observe_freq_mhz, - data.carrier_ppm, - data.group_delay, - ] { - digest.update(value.to_le_bytes()); - } - digest.update((data.points.len() as u64).to_le_bytes()); - for point in &data.points { - digest.update(point.re.to_le_bytes()); - digest.update(point.im.to_le_bytes()); - } - format!("{:x}", digest.finalize()) +pub fn craft_input_sha256(data: &NmrSource) -> String { + data.dataset() + .canonical_digests() + .dataset() + .as_bytes() + .iter() + .map(|byte| format!("{byte:02x}")) + .collect() } pub fn craft_component_table(run: &StoredCraftRun) -> Result { diff --git a/crates/core/src/state/craft_fields.rs b/crates/core/src/state/craft_fields.rs index 23b0e05c..87140cdc 100644 --- a/crates/core/src/state/craft_fields.rs +++ b/crates/core/src/state/craft_fields.rs @@ -98,7 +98,7 @@ impl NmrDataset { .chain(self.craft_field_specs().map(CraftFieldSpec::key)) .collect::>(); self.field_catalog - .reconcile_keys(keys, &self.data.source, None); + .reconcile_keys(keys, self.data.source(), None); self.attach_craft_trace_collections(); } @@ -116,7 +116,7 @@ impl NmrDataset { continue; }; let collection = - TraceCollectionId::derived(self.data.source.as_bytes(), key.as_bytes()); + TraceCollectionId::derived(self.data.source().as_bytes(), key.as_bytes()); let items = run .region_summaries .iter() @@ -192,7 +192,7 @@ impl NmrDataset { let mut figure = Figure::new( "", Axis::new( - crate::figures::axis_label(&self.data.nucleus), + crate::figures::axis_label(self.data.nucleus()), observed.ppm_bounds().0, observed.ppm_bounds().1, ) @@ -241,7 +241,7 @@ impl NmrDataset { CraftFieldKind::Residual | CraftFieldKind::Groups => { let (x, y) = self.craft_curve(spec)?; Some(single_curve_figure( - &self.data.nucleus, + self.data.nucleus(), if spec.kind == CraftFieldKind::Residual { "Complex residual" } else { @@ -264,7 +264,7 @@ impl NmrDataset { self.craft_run(spec.run)?; let spectrum = self.cached_model_spectrum(spec.run, Some(region))?; single_curve_figure( - &self.data.nucleus, + self.data.nucleus(), &label, spectrum.ppm, channel_values(&spectrum.values, spec.channel), @@ -299,7 +299,9 @@ impl NmrDataset { .derived_plan .reconstruction_points .max(1), - ); + ) + .map_err(|error| eprintln!("CRAFT model display: {error}")) + .ok()?; if let Ok(mut cache) = self.craft_spectrum_cache.lock() { cache.models.insert((run, region), spectrum.clone()); } @@ -313,19 +315,25 @@ impl NmrDataset { return Some(spectrum.clone()); } let stored = self.craft_run(run)?; + let data = self + .data + .craft_fid() + .map_err(|error| eprintln!("CRAFT residual display: {error}")) + .ok()?; let model = synthesize_craft_fid( &stored.components, - self.data.points.len(), - self.data.spectral_width_hz, + data.points.len(), + data.spectral_width_hz, ); - let residual = self - .data + let residual = data .points .iter() .zip(model) .map(|(observed, model)| observed - model) .collect(); - let spectrum = transformed_points(self, residual); + let spectrum = transformed_points(self, residual) + .map_err(|error| eprintln!("CRAFT residual display: {error}")) + .ok()?; if let Ok(mut cache) = self.craft_spectrum_cache.lock() { cache.residuals.insert(run, spectrum.clone()); } @@ -337,22 +345,46 @@ fn transformed_fid( dataset: &NmrDataset, components: &[plotx_processing::craft::CraftComponent], point_count: usize, -) -> plotx_processing::Spectrum { +) -> Result { + let data = dataset + .data + .craft_fid() + .map_err(|error| error.to_string())?; transformed_points( dataset, - synthesize_craft_fid(components, point_count, dataset.data.spectral_width_hz), + synthesize_craft_fid(components, point_count, data.spectral_width_hz), ) } fn transformed_points( dataset: &NmrDataset, points: Vec, -) -> plotx_processing::Spectrum { - let mut data = dataset.data.clone(); - data.points = points; - let base = - plotx_processing::transform_base(&data, dataset.pipeline(), dataset.group_delay_correct); - plotx_processing::reapply(&base, dataset.pipeline()) +) -> Result { + use plotx_processing::nmr_bridge::{DelayPolicy, RecipeRange}; + let mut view = dataset + .data + .craft_fid() + .map_err(|error| error.to_string())?; + view.points = points; + let source = + plotx_io::nmr_view::NmrSource::try_from(view).map_err(|error| error.to_string())?; + let output = plotx_processing::nmr_execution::execute_1d( + &source, + dataset.pipeline(), + if dataset.group_delay_correct { + DelayPolicy::AxisEvidence + } else { + DelayPolicy::Disabled + }, + RecipeRange::All, + &mut nmr::ExecutionContext::default(), + ) + .map_err(|error| error.to_string())?; + output + .view + .as_frequency() + .cloned() + .ok_or_else(|| "CRAFT spectrum display requires an FFT".into()) } fn channel_values(values: &[num_complex::Complex64], channel: CraftSpectrumChannel) -> Vec { diff --git a/crates/core/src/state/data_import.rs b/crates/core/src/state/data_import.rs new file mode 100644 index 00000000..8dd3c935 --- /dev/null +++ b/crates/core/src/state/data_import.rs @@ -0,0 +1,200 @@ +//! Bounded, document-scoped background data preparation. +use super::{Dataset, PlotxApp}; +use std::{collections::VecDeque, path::PathBuf, sync::mpsc}; + +pub(super) struct PreparedImport { + pub dataset: Dataset, + pub source: String, + pub format: plotx_io::DataFormat, + pub warnings: Vec, +} + +#[cfg(test)] +#[path = "data_import_tests.rs"] +mod tests; + +impl PreparedImport { + pub(super) fn new(loaded: plotx_io::LoadResult, equal_scale: bool) -> Result { + let (dataset, source) = crate::workflow::dataset_from_loaded_acquisition( + loaded.acquisition, + loaded.acquisition_identity, + equal_scale, + ) + .map_err(|error| error.to_string())?; + Ok(Self { + dataset, + source, + format: loaded.format, + warnings: loaded.warnings, + }) + } +} + +type Discover = Box Result, String> + Send>; +struct Request { + recent: PathBuf, + discover: Discover, + equal_scale: bool, +} + +enum Event { + Started(PathBuf, usize), + Item(PathBuf, Result), + Finished, +} + +struct Job { + receiver: mpsc::Receiver, + recent: PathBuf, + loaded: usize, + failed: usize, +} + +/// Dropping a session disconnects the bounded channel. The worker then exits +/// without joining the UI thread or publishing results into the next document. +#[derive(Default)] +pub struct DataImports { + pending: VecDeque, + active: Option, +} + +impl DataImports { + pub fn is_pending(&self) -> bool { + self.active.is_some() || !self.pending.is_empty() + } +} + +impl PlotxApp { + /// Discovery, parsing, default processing and figure preparation run on one + /// worker. Additional gestures queue behind it instead of multiplying RAM use. + pub fn queue_data_import( + &mut self, + recent: PathBuf, + discover: impl FnOnce() -> Result, String> + Send + 'static, + ) { + self.session.data_imports.pending.push_back(Request { + recent, + discover: Box::new(discover), + equal_scale: self.settings.general.equal_scale_homonuclear_2d_imports, + }); + self.session.status = "Data import queued; you can continue working.".into(); + } + + /// Call once per frame. Never drain the channel: each insertion gets its own + /// frame even when the worker produces many small acquisitions immediately. + pub fn poll_data_import(&mut self) -> bool { + let mut imports = std::mem::take(&mut self.session.data_imports); + if imports.active.is_none() + && let Some(request) = imports.pending.pop_front() + { + let (sender, receiver) = mpsc::sync_channel(1); + let recent = request.recent.clone(); + match std::thread::Builder::new() + .name("data-import".into()) + .spawn(move || { + let paths = match (request.discover)() { + Ok(paths) => paths, + Err(error) => { + if sender + .send(Event::Item(request.recent, Err(error))) + .is_err() + { + return; + } + if sender.send(Event::Finished).is_err() { + return; // The owning document was closed. + } + return; + } + }; + let total = paths.len(); + for path in paths { + if sender.send(Event::Started(path.clone(), total)).is_err() { + return; + } + let result = plotx_io::load_path(&path) + .map_err(|error| error.to_string()) + .and_then(|loaded| PreparedImport::new(loaded, request.equal_scale)); + if sender.send(Event::Item(path, result)).is_err() { + return; + } + } + // A disconnected receiver means the document was closed. + if sender.send(Event::Finished).is_err() { + // Document closure is normal cancellation, not an import failure. + } + }) { + Ok(_) => { + imports.active = Some(Job { + receiver, + recent, + loaded: 0, + failed: 0, + }) + } + Err(error) => { + self.install_prepared_import( + &recent, + Err(format!("Could not start import worker: {error}")), + ); + } + } + } + if let Some(job) = imports.active.as_mut() { + match job.receiver.try_recv() { + Ok(Event::Started(path, total)) => { + self.session.status = format!( + "Importing {}/{}: {} ({} loaded, {} failed)", + job.loaded + job.failed + 1, + total, + path.display(), + job.loaded, + job.failed + ); + } + Ok(Event::Item(path, result)) => { + // Completion must not steal the user's current page or Data + // selection while they edit something else during the batch. + let active_canvas = self.session.active_canvas; + let selection = self.session.ui.data_selection.clone(); + let view = self.session.view; + if self.install_prepared_import(&path, result) { + job.loaded += 1; + } else { + job.failed += 1; + } + self.session.active_canvas = active_canvas; + self.session.ui.data_selection = selection; + self.session.view = view; + self.session.status = format!( + "Importing: {} loaded, {} failed. {}", + job.loaded, + job.failed, + path.display() + ); + } + Ok(Event::Finished) => { + self.session.status = format!( + "Import complete: {} loaded, {} failed.", + job.loaded, job.failed + ); + if job.loaded > 0 { + self.note_recent_file(&job.recent); + } + imports.active = None; + } + Err(mpsc::TryRecvError::Empty) => {} + Err(mpsc::TryRecvError::Disconnected) => { + self.install_prepared_import( + &job.recent, + Err("The import worker stopped unexpectedly; retry the import.".into()), + ); + imports.active = None; + } + } + } + let busy = imports.is_pending(); + self.session.data_imports = imports; + busy + } +} diff --git a/crates/core/src/state/data_import_tests.rs b/crates/core/src/state/data_import_tests.rs new file mode 100644 index 00000000..503e8a44 --- /dev/null +++ b/crates/core/src/state/data_import_tests.rs @@ -0,0 +1,131 @@ +use super::*; +use crate::state::XrdDataset; + +fn prepared() -> PreparedImport { + PreparedImport { + dataset: Dataset::Xrd(Box::new(XrdDataset::load(plotx_io::XrdData { + two_theta_deg: vec![1.0, 2.0, 3.0], + intensity: vec![2.0, 4.0, 2.0], + attenuation: None, + source: "sample.raw".into(), + instrument: None, + target: None, + wavelength_angstrom: None, + voltage_kv: None, + current_ma: None, + scan_step_deg: None, + scan_speed_deg_min: None, + }))), + source: "sample.raw".into(), + format: plotx_io::DataFormat::Xrd(plotx_io::XrdFormat::RigakuRaw), + warnings: Vec::new(), + } +} + +fn app_with_channel() -> (PlotxApp, mpsc::Sender) { + let mut app = PlotxApp::new_with_settings(Default::default()); + let (sender, receiver) = mpsc::channel(); + app.session.data_imports.active = Some(Job { + receiver, + recent: "batch".into(), + loaded: 0, + failed: 0, + }); + (app, sender) +} + +#[test] +fn commits_only_one_item_per_poll_and_preserves_selection_and_undo() { + let (mut app, sender) = app_with_channel(); + assert!(app.install_prepared_import(std::path::Path::new("existing"), Ok(prepared()))); + let selected = app.session.ui.data_selection.clone(); + let active = app.session.active_canvas; + for _ in 0..2 { + sender + .send(Event::Item("next".into(), Ok(prepared()))) + .unwrap(); + } + assert!(app.poll_data_import()); + assert_eq!(app.doc.datasets.len(), 2); + assert_eq!(app.session.active_canvas, active); + assert_eq!(app.session.ui.data_selection, selected); + assert!(app.poll_data_import()); + assert_eq!(app.doc.datasets.len(), 3); + app.undo(); + assert_eq!(app.doc.datasets.len(), 2); + app.redo(); + assert_eq!(app.doc.datasets.len(), 3); +} + +#[test] +fn failed_item_is_reported_and_does_not_prevent_later_success() { + let (mut app, sender) = app_with_channel(); + sender + .send(Event::Item("bad".into(), Err("broken source".into()))) + .unwrap(); + sender + .send(Event::Item("good".into(), Ok(prepared()))) + .unwrap(); + app.poll_data_import(); + assert!(app.doc.datasets.is_empty()); + assert_eq!( + app.session + .operation_history + .operations() + .next_back() + .unwrap() + .outcome, + crate::operation::OperationOutcome::Failure + ); + app.poll_data_import(); + assert_eq!(app.doc.datasets.len(), 1); + assert!(app.session.status.contains("1 loaded, 1 failed")); +} + +#[test] +fn disconnected_worker_reaches_user_feedback() { + let (mut app, sender) = app_with_channel(); + drop(sender); + assert!(!app.poll_data_import()); + assert!(app.session.status.contains("stopped unexpectedly")); + assert_eq!(app.session.operation_history.operations().count(), 1); +} + +#[test] +fn document_swap_discards_ready_results_and_queued_requests() { + let (mut app, sender) = app_with_channel(); + sender + .send(Event::Item("old".into(), Ok(prepared()))) + .unwrap(); + app.queue_data_import("old queued".into(), || { + panic!("old request must be dropped") + }); + app.start_new_project(); + assert!(sender.send(Event::Finished).is_err()); + assert!(!app.poll_data_import()); + assert!(app.doc.datasets.is_empty()); +} + +#[test] +fn discovery_runs_off_thread_and_poll_does_not_wait_for_it() { + let mut app = PlotxApp::new_with_settings(Default::default()); + let main_thread = std::thread::current().id(); + let (entered, started) = mpsc::channel(); + let (release, wait) = mpsc::channel(); + app.queue_data_import("batch".into(), move || { + entered.send(std::thread::current().id()).unwrap(); + wait.recv().unwrap(); + Ok(vec![]) + }); + assert!(app.poll_data_import()); + assert_ne!( + started + .recv_timeout(std::time::Duration::from_secs(5)) + .unwrap(), + main_thread + ); + assert!(app.poll_data_import()); + app.start_new_project(); + release.send(()).unwrap(); + assert!(!app.poll_data_import()); +} diff --git a/crates/core/src/state/dataset_trace.rs b/crates/core/src/state/dataset_trace.rs index 9704686b..64768854 100644 --- a/crates/core/src/state/dataset_trace.rs +++ b/crates/core/src/state/dataset_trace.rs @@ -23,6 +23,9 @@ impl Dataset { data.craft_group_figure(spec, region, label) } Self::Nmr2D(data) => { + if data.field_catalog.id_for_key(data.stack_field_key()) != Some(field) { + return None; + } let plotx_processing::Processed2D::Stack(stack) = &data.processed else { return None; }; @@ -44,13 +47,7 @@ impl Dataset { y0 = -0.5; y1 = 0.5; } - let x_name = if stack.direct_domain == plotx_io::Domain::Frequency { - crate::figures::axis_label(&stack.direct.nucleus) - } else { - "Time (s)".to_owned() - }; - let x_axis = plotx_figure::Axis::new(x_name, x0, x1) - .reversed(stack.direct_domain == plotx_io::Domain::Frequency); + let x_axis = crate::figures::nmr_axis(&stack.direct, x0, x1); Some( plotx_figure::Figure::new( "", diff --git a/crates/core/src/state/datasets.rs b/crates/core/src/state/datasets.rs index ed4cfe5b..69fb9feb 100644 --- a/crates/core/src/state/datasets.rs +++ b/crates/core/src/state/datasets.rs @@ -1,10 +1,8 @@ use super::*; use std::sync::Arc; -/// Factory rule shared by dataset construction, reset, and property defaults. -pub(crate) fn default_group_delay_correct(domain: Domain) -> bool { - matches!(domain, Domain::Time) -} +mod nmr_defaults; +pub(crate) use nmr_defaults::*; #[derive(Clone, Copy, PartialEq, Eq)] pub enum PhaseDragKind { @@ -34,10 +32,10 @@ pub struct NmrDataset { pub resource_id: DatasetId, /// Persisted child-field identity allocator and key mapping. pub field_catalog: FieldCatalog, - pub data: NmrData, - /// Required v1 origin contract. `Derived` is a real scientific state, not - /// a fallback for projects that omitted the field. - pub origin: plotx_io::NmrOrigin, + pub data: plotx_io::nmr_view::NmrSource, + pub native_base: plotx_io::nmr_view::NmrSource, + pub native_processed: plotx_io::nmr_view::NmrSource, + pub phase_reports: Vec, pub acquisition_identity: plotx_io::AcquisitionIdentity, pub base: Processed1D, pub pipeline: AxisPipeline, @@ -70,36 +68,74 @@ pub struct NmrDataset { } impl NmrDataset { - pub fn load(data: NmrData) -> Self { - Self::load_with_origin(data, plotx_io::NmrOrigin::Derived) + pub fn load(input: T) -> Result + where + T: TryInto, + T::Error: std::fmt::Display, + { + Self::load_with_pipeline(input, None, None) } - pub fn load_with_origin(data: NmrData, origin: plotx_io::NmrOrigin) -> Self { - let acquisition_identity = - plotx_io::AcquisitionIdentity::from_path(std::path::Path::new(&data.source)); - let pipeline = match data.domain { - Domain::Time => AxisPipeline::default_1d(), - Domain::Frequency => AxisPipeline::frequency_1d(), - }; - let group_delay_correct = default_group_delay_correct(data.domain); - let has_imaginary = data.domain == Domain::Time || data.points.iter().any(|v| v.im != 0.0); - let base = transform_output_base(&data, &pipeline, group_delay_correct) - .expect("factory processing pipeline is domain-valid"); - let processed = reapply_output(&base, &pipeline); + pub fn load_with_pipeline( + input: T, + pipeline: Option, + correct_delay: Option, + ) -> Result + where + T: TryInto, + T::Error: std::fmt::Display, + { + use plotx_processing::nmr_bridge::{DelayPolicy, RecipeRange}; + let data = input.try_into().map_err(|error| error.to_string())?; + if data.axes().len() != 1 { + return Err("Select a one-dimensional NMR dataset".into()); + } + let acquisition_identity = data.identity(); + let domain = data.domain().map_err(|error| error.to_string())?; + let known_delay = default_group_delay_correct(&data); + let has_imaginary = data.has_imaginary(0); + let mut pipeline = pipeline.unwrap_or_else(|| default_nmr_pipeline(&data)); + for (index, step) in pipeline.steps.iter_mut().enumerate() { + step.id = StepId::new(index as u64); + } + let group_delay_correct = correct_delay.unwrap_or(domain == Domain::Time && known_delay); + let mut context = nmr::ExecutionContext::default(); + let base = plotx_processing::nmr_execution::execute_1d( + &data, + &pipeline, + if group_delay_correct { + DelayPolicy::AxisEvidence + } else { + DelayPolicy::Disabled + }, + RecipeRange::Base, + &mut context, + ) + .map_err(|error| error.to_string())?; + let processed = plotx_processing::nmr_execution::execute_1d( + &base.source, + &pipeline, + DelayPolicy::Disabled, + RecipeRange::Frequency, + &mut context, + ) + .map_err(|error| error.to_string())?; let mut field_catalog = nmr_field_catalog(); - field_catalog.attach_provenance(&data.source, None); + field_catalog.attach_provenance(data.source(), None); let mut result = Self { resource_id: DatasetId::new(), field_catalog, data, - origin, acquisition_identity, - base, + native_base: base.source, + native_processed: processed.source, + phase_reports: processed.phases, + base: base.view, pipeline, next_step_id: 0, group_delay_correct, has_imaginary, - processed, + processed: processed.view, name: None, lineage: None, peaks: PeakSet::default(), @@ -117,23 +153,65 @@ impl NmrDataset { // allocator starts at 0. Kept so `load` establishes the "ids are unique // and below next_step_id" invariant itself, rather than inheriting it // from whichever template `pipeline` happened to come from. - result.remint_all_steps(); - result + result.repair_step_allocator(); + Ok(result) } - /// Cheap re-apply of the frequency-domain steps from the cached `base`. - pub fn rebuild(&mut self) { + pub fn input_domain(&self) -> Domain { + // Construction validates rank and the direct signal domain. + match self.data.axes()[0].domain { + nmr::axis::AxisDomain::Time => Domain::Time, + _ => Domain::Frequency, + } + } + + pub fn rebuild(&mut self) -> Result<(), String> { + use plotx_processing::nmr_bridge::{DelayPolicy, RecipeRange}; + let output = plotx_processing::nmr_execution::execute_1d( + &self.native_base, + &self.pipeline, + DelayPolicy::Disabled, + RecipeRange::Frequency, + &mut nmr::ExecutionContext::default(), + ) + .map_err(|error| error.to_string())?; + self.native_processed = output.source; + self.processed = output.view; + self.phase_reports = output.phases; self.clear_craft_spectrum_cache(); - self.processed = reapply_output(&self.base, &self.pipeline); + Ok(()) } - /// Rebuild `base` from the acquisition, including a real output-domain - /// transition when FFT was added or removed. - pub fn retransform(&mut self) { + pub fn retransform(&mut self) -> Result<(), String> { + use plotx_processing::nmr_bridge::{DelayPolicy, RecipeRange}; + let mut context = nmr::ExecutionContext::default(); + let base = plotx_processing::nmr_execution::execute_1d( + &self.data, + &self.pipeline, + if self.group_delay_correct { + DelayPolicy::AxisEvidence + } else { + DelayPolicy::Disabled + }, + RecipeRange::Base, + &mut context, + ) + .map_err(|error| error.to_string())?; + let output = plotx_processing::nmr_execution::execute_1d( + &base.source, + &self.pipeline, + DelayPolicy::Disabled, + RecipeRange::Frequency, + &mut context, + ) + .map_err(|error| error.to_string())?; + self.native_base = base.source; + self.base = base.view; + self.native_processed = output.source; + self.processed = output.view; + self.phase_reports = output.phases; self.clear_craft_spectrum_cache(); - self.base = transform_output_base(&self.data, &self.pipeline, self.group_delay_correct) - .expect("live processing pipelines are reconciled before application"); - self.rebuild(); + Ok(()) } pub fn spectrum(&self) -> Option<&Spectrum> { @@ -172,13 +250,6 @@ impl NmrDataset { .unwrap_or(0); self.next_step_id = self.next_step_id.max(required); } - - fn remint_all_steps(&mut self) { - for step in &mut self.pipeline.steps { - step.id = StepId::new(self.next_step_id); - self.next_step_id = self.next_step_id.checked_add(1).expect("step id overflow"); - } - } } /// A loaded 2D acquisition and its processing recipe. `base` is the post-FFT, @@ -189,8 +260,13 @@ pub struct Nmr2DDataset { pub resource_id: DatasetId, /// Persisted child-field identity allocator and key mapping. pub field_catalog: FieldCatalog, - pub data: Arc, - pub origin: plotx_io::NmrOrigin, + pub data: Arc, + pub native_base: plotx_io::nmr_view::NmrSource, + pub native_processed: plotx_io::nmr_view::NmrSource, + pub phase_reports: Vec, + pub nus_request: Option, + /// Import diagnostic when automatic reconstruction could not produce a spectrum. + pub reconstruction_warning: Option, pub acquisition_identity: plotx_io::AcquisitionIdentity, pub params: Params2D, /// Persistent owner-local allocator shared by both axes. @@ -249,37 +325,96 @@ pub struct Nmr2DDataset { pub dosy_provenance_warning: Option, } impl Nmr2DDataset { - pub fn load(data: NmrData2D) -> Self { - Self::load_with_origin_and_equal_scale_preference(data, plotx_io::NmrOrigin::Derived, true) + pub fn load(input: T) -> Result + where + T: TryInto, + T::Error: std::fmt::Display, + { + Self::load_with_equal_scale_preference(input, true) } - pub fn load_with_equal_scale_preference( - data: NmrData2D, - equal_scale_homonuclear_2d_imports: bool, - ) -> Self { - Self::load_with_origin_and_equal_scale_preference( - data, - plotx_io::NmrOrigin::Derived, - equal_scale_homonuclear_2d_imports, - ) + pub fn load_with_equal_scale_preference(input: T, equal_scale: bool) -> Result + where + T: TryInto, + T::Error: std::fmt::Display, + { + let source = input.try_into().map_err(|error| error.to_string())?; + match Self::load_with_pipeline(source.clone(), None, None, None, equal_scale) { + Ok(dataset) => Ok(dataset), + Err(error) if source.nus.is_some() => { + // Preserve the acquisition when estimation/reconstruction is unsupported. + // Import surfaces must report this as a warning, never a completed spectrum. + let params = Params2D { + layout: plotx_processing::Layout2D::Stack, + f2: AxisPipeline { steps: vec![] }, + f1: AxisPipeline { steps: vec![] }, + }; + let mut dataset = + Self::load_with_pipeline(source, Some(params), Some(false), None, equal_scale)?; + dataset.reconstruction_warning = Some(format!( + "Automatic NUS reconstruction failed; showing acquired observations: {error}" + )); + Ok(dataset) + } + Err(error) => Err(error), + } } - pub fn load_with_origin_and_equal_scale_preference( - data: NmrData2D, - origin: plotx_io::NmrOrigin, + pub fn load_with_pipeline( + input: T, + params: Option, + correct_delay: Option, + nus_request: Option, equal_scale_homonuclear_2d_imports: bool, - ) -> Self { - let acquisition_identity = - plotx_io::AcquisitionIdentity::from_path(std::path::Path::new(&data.source)); + ) -> Result + where + T: TryInto, + T::Error: std::fmt::Display, + { + use plotx_processing::nmr_bridge::{DelayPolicy, RecipeRange}; + let data = input.try_into().map_err(|error| error.to_string())?; + let source = data.source_dataset(); + let acquisition_identity = source.identity(); let preset = recommend_preset(&data); - let params = match data.domain { - Domain::Time => Params2D::default_for(preset), - Domain::Frequency => Params2D::frequency_domain(preset), - }; - let group_delay_correct = default_group_delay_correct(data.domain); - let has_imaginary = data.domain == Domain::Time || data.data.iter().any(|v| v.im != 0.0); - let base = process_2d(&data, ¶ms); - let processed = reapply_2d(&base, ¶ms); + let known_delay = default_group_delay_correct(source); + let mut params = params.unwrap_or_else(|| default_nmr_params(&data, preset)); + for (index, step) in params + .f2 + .steps + .iter_mut() + .chain(&mut params.f1.steps) + .enumerate() + { + step.id = StepId::new(index as u64); + } + let group_delay_correct = correct_delay + .unwrap_or(known_delay && data.direct.domain == nmr::axis::AxisDomain::Time); + let has_imaginary = source.has_imaginary(1); + let mut work = plotx_processing::nmr_execution::processing_2d_work_ledger(); + let mut context = nmr::ExecutionContext::new(&mut work); + let base = plotx_processing::nmr_execution::execute_2d( + source, + ¶ms, + if group_delay_correct { + DelayPolicy::AxisEvidence + } else { + DelayPolicy::Disabled + }, + RecipeRange::Base, + nus_request, + &mut context, + ) + .map_err(|error| error.to_string())?; + let output = plotx_processing::nmr_execution::execute_2d( + &base.source, + ¶ms, + DelayPolicy::Disabled, + RecipeRange::Frequency, + None, + &mut context, + ) + .map_err(|error| error.to_string())?; + let processed = output.view; let mut processed_figure = build_processed_figure(&processed, preset); if !equal_scale_homonuclear_2d_imports { processed_figure.lock_aspect = false; @@ -298,16 +433,20 @@ impl Nmr2DDataset { resource_id: DatasetId::new(), field_catalog, data: Arc::new(data), - origin, acquisition_identity, + native_base: base.source, + native_processed: output.source, + phase_reports: output.phases, + nus_request, base_params: params.clone(), + reconstruction_warning: None, base_stale: false, params, next_step_id: 0, preset, group_delay_correct, has_imaginary, - base, + base: base.view, processed, processed_figure, name: None, @@ -327,40 +466,87 @@ impl Nmr2DDataset { integral_error: None, dosy_provenance_warning: None, }; - result.remint_all_steps(); - result + result.repair_step_allocator(); + Ok(result) } - /// Cheap re-apply of per-axis phase from the cached `base` (no FFT). - pub fn rebuild(&mut self) { - self.processed = reapply_2d(&self.base, &self.params); + + pub fn input_domain(&self, axis: PhaseAxis) -> Result { + self.data + .input_domain(if axis == PhaseAxis::F1 { 0 } else { 1 }) + .map_err(|error| error.to_string()) + } + + pub fn rebuild(&mut self) -> Result<(), String> { + use plotx_processing::nmr_bridge::{DelayPolicy, RecipeRange}; + let output = plotx_processing::nmr_execution::execute_2d( + &self.native_base, + &self.params, + DelayPolicy::Disabled, + RecipeRange::Frequency, + None, + &mut nmr::ExecutionContext::default(), + ) + .map_err(|error| error.to_string())?; + self.native_processed = output.source; + self.processed = output.view; + self.phase_reports = output.phases; self.processed_figure = Arc::new(build_processed_figure(&self.processed, self.preset)); self.invalidate_dosy_results("Processing changed and invalidated the selected DOSY map"); + Ok(()) } - /// Rebuild `base` from the FID (a time-domain step or the layout changed) then - /// re-derive the display result. - pub fn retransform(&mut self) { - let data = self.processing_data(); - self.base = process_2d(&data, &self.params); + + pub fn retransform(&mut self) -> Result<(), String> { + use plotx_processing::nmr_bridge::{DelayPolicy, RecipeRange}; + let mut work = plotx_processing::nmr_execution::processing_2d_work_ledger(); + let mut context = nmr::ExecutionContext::new(&mut work); + let base = plotx_processing::nmr_execution::execute_2d( + self.data.source_dataset(), + &self.params, + if self.group_delay_correct { + DelayPolicy::AxisEvidence + } else { + DelayPolicy::Disabled + }, + RecipeRange::Base, + self.nus_request, + &mut context, + ) + .map_err(|error| error.to_string())?; + let output = plotx_processing::nmr_execution::execute_2d( + &base.source, + &self.params, + DelayPolicy::Disabled, + RecipeRange::Frequency, + None, + &mut context, + ) + .map_err(|error| error.to_string())?; + self.native_base = base.source; + self.reconstruction_warning = None; + self.base = base.view; + self.native_processed = output.source; + self.processed = output.view; + self.phase_reports = output.phases; self.base_params = self.params.clone(); self.base_stale = false; - self.rebuild(); + self.processed_figure = Arc::new(build_processed_figure(&self.processed, self.preset)); + self.invalidate_dosy_results("Processing changed and invalidated the selected DOSY map"); + Ok(()) } - /// Input view for the 2D transform's existing unconditional direct-axis - /// delay removal. - /// - /// Keeping the switch here avoids a second FFT implementation: disabling - /// correction presents zero delay metadata to the same scientific kernel. - /// The uncommon disabled path owns one copy so the persisted acquisition - /// metadata remains untouched. - pub(crate) fn processing_data(&self) -> Arc { - if self.group_delay_correct { - return Arc::clone(&self.data); + pub(crate) fn stack_field_key(&self) -> &'static str { + if self + .native_processed + .dataset() + .as_raw() + .is_some_and(|raw| raw.data().is_sparse()) + { + "nmr.observations" + } else { + "nmr.stack" } - let mut data = (*self.data).clone(); - data.direct.group_delay = 0.0; - Arc::new(data) } + /// A true-2D (contour) result, as opposed to a pseudo-2D stack of slices. pub fn is_true_2d(&self) -> bool { matches!(self.processed, Processed2D::Ft(_)) @@ -408,7 +594,11 @@ impl Nmr2DDataset { // Auto steps have a placeholder pivot; show the peak the pass really // rotates about so the on-plot handle isn't pinned to an edge. StepKind::Phase(p) => Some(match p.auto { - Some(_) => self.auto_pivot_frac(axis), + Some(_) => self + .phase_reports + .iter() + .find(|report| report.step == s.id) + .map_or(p.pivot_frac, |report| report.recipe_parameters().2), None => p.pivot_frac, }), _ => None, @@ -416,17 +606,6 @@ impl Nmr2DDataset { .unwrap_or(0.0); Some(lo + (hi - lo) * frac) } - /// The peak the auto-phase pass rotates about, per axis, read from the cached - /// pre-phase `base`. - fn auto_pivot_frac(&self, axis: PhaseAxis) -> f64 { - match &self.base { - Processed2D::Ft(s) => { - let (f2, f1) = s.peak_pivot_fracs(); - if axis == PhaseAxis::F1 { f1 } else { f2 } - } - Processed2D::Stack(s) => s.peak_pivot_frac(), - } - } pub fn set_pivot_ppm(&mut self, axis: PhaseAxis, ppm: f64) { let Some((lo, hi)) = self.axis_ppm_ends(axis) else { return; @@ -466,19 +645,6 @@ impl Nmr2DDataset { .unwrap_or(0); self.next_step_id = self.next_step_id.max(required); } - - fn remint_all_steps(&mut self) { - for step in self - .params - .f2 - .steps - .iter_mut() - .chain(&mut self.params.f1.steps) - { - step.id = StepId::new(self.next_step_id); - self.next_step_id = self.next_step_id.checked_add(1).expect("step id overflow"); - } - } } #[derive(Clone)] diff --git a/crates/core/src/state/datasets/nmr_defaults.rs b/crates/core/src/state/datasets/nmr_defaults.rs new file mode 100644 index 00000000..190b8d90 --- /dev/null +++ b/crates/core/src/state/datasets/nmr_defaults.rs @@ -0,0 +1,66 @@ +//! Import, reset and property defaults use the same checked acquisition evidence. + +use super::*; +use nmr::{acquisition::GroupDelayState, axis::AxisDomain}; +use plotx_io::{nmr_series::NmrSeriesSource, nmr_view::NmrSource}; + +pub(crate) fn default_group_delay_correct(source: &NmrSource) -> bool { + source.dataset().as_raw().is_some_and(|raw| { + raw.descriptor().axes().last().is_some_and(|axis| { + axis.domain() == AxisDomain::Time + && matches!( + axis.group_delay(), + GroupDelayState::Pending(_) | GroupDelayState::NotApplicable + ) + }) + }) +} + +pub(crate) fn default_nmr_pipeline(source: &NmrSource) -> AxisPipeline { + if source.axes()[0].domain == AxisDomain::Time { + return if default_group_delay_correct(source) { + AxisPipeline::default_1d() + } else { + AxisPipeline { steps: Vec::new() } + }; + } + let mut pipeline = AxisPipeline::frequency_1d(); + disable_scalar_phase(&mut pipeline, source, 0); + pipeline +} + +pub(crate) fn default_nmr_params(data: &NmrSeriesSource, preset: Preset2D) -> Params2D { + let source = data.source_dataset(); + let mut params = Params2D::default_for(preset); + for (axis, pipeline) in [(1, &mut params.f2), (0, &mut params.f1)] { + match source.axes()[axis].domain { + AxisDomain::Frequency => { + *pipeline = AxisPipeline::frequency_2d(axis == 1); + disable_scalar_phase(pipeline, source, axis); + } + AxisDomain::Parameter => pipeline.steps.clear(), + _ => {} + } + } + if source.axes()[0].domain == AxisDomain::Parameter { + params.layout = plotx_processing::Layout2D::Stack; + } + if !default_group_delay_correct(source) + && data.direct.domain == AxisDomain::Time + && data.nus.is_none() + { + params.f2.steps.clear(); + params.f1.steps.clear(); + } + params +} + +fn disable_scalar_phase(pipeline: &mut AxisPipeline, source: &NmrSource, axis: usize) { + if !source.has_imaginary(axis) { + for step in &mut pipeline.steps { + if matches!(step.kind, StepKind::Phase(_)) { + step.enabled = false; + } + } + } +} diff --git a/crates/core/src/state/datasets/pseudo_display_binding_tests.rs b/crates/core/src/state/datasets/pseudo_display_binding_tests.rs index 3196e592..08a655ab 100644 --- a/crates/core/src/state/datasets/pseudo_display_binding_tests.rs +++ b/crates/core/src/state/datasets/pseudo_display_binding_tests.rs @@ -3,7 +3,7 @@ use super::*; #[test] fn trace_alignment_merges_stack_projection_without_changing_map_bindings() { - let mut owner = Nmr2DDataset::load(synthetic_dosy(1.2e-9)); + let mut owner = Nmr2DDataset::load(synthetic_dosy(1.2e-9)).unwrap(); assert!(owner.build_dosy_map()); owner.display = PseudoDisplay::Stack; let mut app = PlotxApp::new_with_settings(crate::settings::Settings::default()); @@ -61,10 +61,10 @@ fn trace_alignment_merges_stack_projection_without_changing_map_bindings() { #[test] fn live_binding_projects_the_current_field_and_keeps_external_series() { - let mut owner = Nmr2DDataset::load(synthetic_dosy(1.2e-9)); + let mut owner = Nmr2DDataset::load(synthetic_dosy(1.2e-9)).unwrap(); assert!(owner.build_dosy_map()); owner.display = PseudoDisplay::Stack; - let mut external = Nmr2DDataset::load(synthetic_dosy(1.5e-9)); + let mut external = Nmr2DDataset::load(synthetic_dosy(1.5e-9)).unwrap(); assert!(external.build_dosy_map()); let mut app = PlotxApp::new_with_settings(crate::settings::Settings::default()); app.doc.datasets.push(Dataset::Nmr2D(Box::new(owner))); @@ -249,7 +249,7 @@ fn dosy_map_honors_non_default_contour_levels_and_style() { PositiveFiniteF32, PositiveFiniteF64, SeriesEncoding, }; - let mut dataset = Nmr2DDataset::load(synthetic_dosy(1.2e-9)); + let mut dataset = Nmr2DDataset::load(synthetic_dosy(1.2e-9)).unwrap(); assert!(dataset.build_dosy_map()); let peak = dosy_scalar_grid(dataset.dosy_map.as_ref().unwrap()) .values @@ -307,7 +307,7 @@ fn ilt_map_honors_non_default_contour_style() { PositiveFiniteF32, PositiveFiniteF64, SeriesEncoding, }; - let mut dataset = Nmr2DDataset::load(synthetic_dosy(1.2e-9)); + let mut dataset = Nmr2DDataset::load(synthetic_dosy(1.2e-9)).unwrap(); assert!(dataset.build_ilt_map(IltParams { lambda: 1e-2, d_min: 1e-10, diff --git a/crates/core/src/state/datasets/pseudo_tests.rs b/crates/core/src/state/datasets/pseudo_tests.rs index 059f9555..08c5a074 100644 --- a/crates/core/src/state/datasets/pseudo_tests.rs +++ b/crates/core/src/state/datasets/pseudo_tests.rs @@ -84,7 +84,7 @@ pub(super) fn synthetic_dosy(d_true: f64) -> NmrData2D { #[test] fn dataset_builds_dosy_map() { let d_true = 1.2e-9; - let mut ds = Nmr2DDataset::load(synthetic_dosy(d_true)); + let mut ds = Nmr2DDataset::load(synthetic_dosy(d_true)).unwrap(); assert!(ds.is_pseudo()); assert_eq!(ds.preset, Preset2D::Dosy); @@ -95,13 +95,16 @@ fn dataset_builds_dosy_map() { #[test] fn ordered_series_supports_region_analysis() { - let series = Dataset::Nmr2D(Box::new(Nmr2DDataset::load(synthetic_dosy(1.2e-9)))); + let series = Dataset::Nmr2D(Box::new( + Nmr2DDataset::load(synthetic_dosy(1.2e-9)).unwrap(), + )); assert!(series.supports_region_analysis()); assert!(series.tool_groups().contains(&ToolGroup::RegionAnalysis)); let mut without_ruler = synthetic_dosy(1.2e-9); without_ruler.pseudo_axis = None; - let not_a_series = Dataset::Nmr2D(Box::new(Nmr2DDataset::load(without_ruler))); + without_ruler.diffusion = None; + let not_a_series = Dataset::Nmr2D(Box::new(Nmr2DDataset::load(without_ruler).unwrap())); assert!(!not_a_series.supports_region_analysis()); assert!( !not_a_series @@ -117,7 +120,7 @@ fn ordered_series_supports_region_analysis() { #[test] fn region_support_matches_what_the_table_builder_accepts() { let mut app = crate::state::PlotxApp::new_with_settings(crate::settings::Settings::default()); - let mut series = Nmr2DDataset::load(synthetic_dosy(1.2e-9)); + let mut series = Nmr2DDataset::load(synthetic_dosy(1.2e-9)).unwrap(); series.region_analysis.regions = vec![Region { id: RegionId::new(0), lo: 0.9, @@ -141,7 +144,8 @@ fn region_support_matches_what_the_table_builder_accepts() { // the Series Table command from offering a table that cannot be built. let mut ruler_less = synthetic_dosy(1.2e-9); ruler_less.pseudo_axis = None; - let mut stale = Nmr2DDataset::load(ruler_less); + ruler_less.diffusion = None; + let mut stale = Nmr2DDataset::load(ruler_less).unwrap(); stale.region_analysis.regions = vec![Region { id: RegionId::new(0), lo: 0.9, @@ -163,7 +167,7 @@ fn region_support_matches_what_the_table_builder_accepts() { #[test] fn dataset_builds_ilt_dosy_map() { - let mut ds = Nmr2DDataset::load(synthetic_dosy(1.2e-9)); + let mut ds = Nmr2DDataset::load(synthetic_dosy(1.2e-9)).unwrap(); let params = IltParams { lambda: 1e-2, d_min: 1e-10, @@ -182,7 +186,7 @@ fn dataset_builds_ilt_dosy_map() { #[test] fn pseudo_map_fields_are_truthful_scalar_grids_with_map_encodings() { - let mut dataset = Nmr2DDataset::load(synthetic_dosy(1.2e-9)); + let mut dataset = Nmr2DDataset::load(synthetic_dosy(1.2e-9)).unwrap(); assert!(dataset.build_dosy_map()); assert!(dataset.build_ilt_map(IltParams { lambda: 1e-2, @@ -228,7 +232,7 @@ fn switching_dosy_method_serves_that_methods_figure() { d_max: 1e-8, n_grid: 64, }; - let mut ds = Nmr2DDataset::load(synthetic_dosy(1.2e-9)); + let mut ds = Nmr2DDataset::load(synthetic_dosy(1.2e-9)).unwrap(); assert!(ds.build_dosy_map(), "per-column map should populate"); assert!(ds.build_ilt_map(params), "ILT map should populate"); assert!(ds.figure().title.starts_with("DOSY (ILT)")); @@ -247,41 +251,56 @@ fn switching_dosy_method_serves_that_methods_figure() { assert!(ds.figure().title.starts_with("DOSY (ILT)")); } -/// A NUS schedule mutates `data` while leaving the recipe untouched, so nothing in -/// `params` records that the cached base is void. Without the explicit flag, a -/// frequency-only edit arriving before the reconstruction lands would schedule a -/// re-apply from the pre-NUS base and strand the reconstruction forever. +/// Acquisition coordinates stay fixed; reconstruction invocation changes invalidate the base. #[test] -fn entering_a_nus_schedule_forces_a_retransform_until_a_base_lands() { +fn changing_nus_reconstruction_inputs_keeps_the_base_stale_until_a_result_lands() { let mut data = synthetic_dosy(1.2e-9); + data.pseudo_axis = None; + data.diffusion = None; data.nus = Some(plotx_io::NusMeta { grid: data.rows * 2, acquired: data.rows, - idx_base: 0, - mode: String::new(), - echo_antiecho: false, - schedule: None, + schedule: Some((0..data.rows).map(|index| 2 * index).collect()), }); - let mut ds = Nmr2DDataset::load(data); - assert!(!ds.base_stale); - - let rows = ds.data.rows; - ds.set_nus_schedule(&(0..rows).collect::>(), 0) - .expect("a full in-grid schedule is valid"); - assert!(ds.base_stale, "the cached base no longer derives from data"); - - ds.retransform(); - assert!(!ds.base_stale, "a fresh base clears the flag"); + let mut app = PlotxApp::new(); + app.doc + .datasets + .push(Dataset::Nmr2D(Box::new(Nmr2DDataset::load(data).unwrap()))); + let before = app.doc.datasets[0] + .as_nmr2d() + .unwrap() + .data + .source_dataset() + .dataset() + .canonical_digests(); + let mut state = DatasetProcessingState::from_dataset(&app.doc.datasets[0]); + if let DatasetProcessingState::Nmr2D { nus_request, .. } = &mut state { + *nus_request = Some(plotx_processing::nmr_execution::NusRequest { + noise_standard_deviation: Some(0.01), + ..Default::default() + }); + } + app.set_dataset_processing_state(0, &state).unwrap(); + let ds = app.doc.datasets[0].as_nmr2d().unwrap(); + assert!(ds.base_stale); + assert_eq!( + before, + ds.data.source_dataset().dataset().canonical_digests() + ); + app.doc.datasets[0] + .as_nmr2d_mut() + .unwrap() + .retransform() + .unwrap(); + assert!(!app.doc.datasets[0].as_nmr2d().unwrap().base_stale); } #[test] fn persisted_display_and_method_changes_mark_the_document_dirty() { let mut app = PlotxApp::new_with_settings(crate::settings::Settings::default()); - app.doc - .datasets - .push(Dataset::Nmr2D(Box::new(Nmr2DDataset::load( - synthetic_dosy(1.2e-9), - )))); + app.doc.datasets.push(Dataset::Nmr2D(Box::new( + Nmr2DDataset::load(synthetic_dosy(1.2e-9)).unwrap(), + ))); app.doc.dirty = false; app.set_pseudo_display(0, PseudoDisplay::DosyMap); @@ -302,7 +321,7 @@ fn persisted_display_and_method_changes_mark_the_document_dirty() { #[test] fn switching_an_existing_stack_canvas_to_dosy_rebuilds_it_as_a_map() { let mut app = PlotxApp::new_with_settings(crate::settings::Settings::default()); - let mut dataset = Nmr2DDataset::load(synthetic_dosy(1.2e-9)); + let mut dataset = Nmr2DDataset::load(synthetic_dosy(1.2e-9)).unwrap(); assert!(dataset.build_dosy_map()); dataset.display = PseudoDisplay::Stack; app.doc.datasets.push(Dataset::Nmr2D(Box::new(dataset))); @@ -444,11 +463,11 @@ fn switching_an_existing_stack_canvas_to_dosy_rebuilds_it_as_a_map() { #[test] fn processing_invalidation_explains_the_stack_fallback() { - let mut dataset = Nmr2DDataset::load(synthetic_dosy(1.2e-9)); + let mut dataset = Nmr2DDataset::load(synthetic_dosy(1.2e-9)).unwrap(); assert!(dataset.build_dosy_map()); assert_eq!(dataset.display, PseudoDisplay::DosyMap); - dataset.rebuild(); + dataset.rebuild().unwrap(); assert!(dataset.dosy_map.is_none()); assert!(dataset.figure().title.starts_with("Pseudo-2D stack —")); @@ -466,7 +485,7 @@ fn ilt_invocation_resolution_obeys_explicit_provenance_default_and_reports_empty let mut settings = crate::settings::Settings::default(); settings.processing.ilt_lambda = 0.8; let mut app = PlotxApp::new_with_settings(settings); - let mut dataset = Nmr2DDataset::load(synthetic_dosy(1.2e-9)); + let mut dataset = Nmr2DDataset::load(synthetic_dosy(1.2e-9)).unwrap(); let previous = IltParams { lambda: 0.03, d_min: 1e-10, @@ -488,10 +507,9 @@ fn ilt_invocation_resolution_obeys_explicit_provenance_default_and_reports_empty let mut empty = synthetic_dosy(1.2e-9); empty.data.fill(Complex64::new(0.0, 0.0)); let mut empty_app = PlotxApp::new_with_settings(crate::settings::Settings::default()); - empty_app - .doc - .datasets - .push(Dataset::Nmr2D(Box::new(Nmr2DDataset::load(empty)))); + empty_app.doc.datasets.push(Dataset::Nmr2D(Box::new( + crate::nmr_test_support::load_2d(empty).unwrap(), + ))); // Deliberately not a boundary value: at MIN or MAX the assertion below would // be satisfied by the range text the same message prints, and would still // pass with the value itself removed from the message. @@ -545,9 +563,9 @@ fn a_build_that_fits_nothing_still_marks_the_document_dirty() { .data .iter_mut() .for_each(|value| *value = Complex64::ZERO); - app.doc - .datasets - .push(Dataset::Nmr2D(Box::new(Nmr2DDataset::load(empty)))); + app.doc.datasets.push(Dataset::Nmr2D(Box::new( + crate::nmr_test_support::load_2d(empty).unwrap(), + ))); app.doc.dirty = false; app.build_dosy_map_for(0); @@ -580,7 +598,7 @@ fn a_build_that_fits_nothing_still_marks_the_document_dirty() { #[test] fn the_missing_map_note_tracks_the_current_selection_instead_of_persisting() { let mut app = PlotxApp::new_with_settings(crate::settings::Settings::default()); - let mut ds = Nmr2DDataset::load(synthetic_dosy(1.2e-9)); + let mut ds = Nmr2DDataset::load(synthetic_dosy(1.2e-9)).unwrap(); assert!(ds.build_dosy_map()); app.doc.datasets.push(Dataset::Nmr2D(Box::new(ds))); @@ -627,7 +645,7 @@ fn the_missing_map_note_tracks_the_current_selection_instead_of_persisting() { /// produce a different digest. #[test] fn the_data_fingerprint_covers_coordinates_and_diffusion_metadata() { - let mut ds = Nmr2DDataset::load(synthetic_dosy(1.2e-9)); + let mut ds = Nmr2DDataset::load(synthetic_dosy(1.2e-9)).unwrap(); let Processed2D::Stack(stack) = &ds.processed else { panic!("synthetic DOSY must process as a stack"); }; @@ -713,11 +731,9 @@ fn ilt_parameters_from_a_project_are_validated_before_the_inversion() { // And the build path must actually consult it rather than reaching the // inversion with the value. let mut app = PlotxApp::new_with_settings(crate::settings::Settings::default()); - app.doc - .datasets - .push(Dataset::Nmr2D(Box::new(Nmr2DDataset::load( - synthetic_dosy(1.2e-9), - )))); + app.doc.datasets.push(Dataset::Nmr2D(Box::new( + Nmr2DDataset::load(synthetic_dosy(1.2e-9)).unwrap(), + ))); app.build_ilt_map_for_with_params(0, Some(huge_grid)); assert!( app.doc.datasets[0].as_nmr2d().unwrap().ilt_map.is_none(), diff --git a/crates/core/src/state/datasets_2d_figure.rs b/crates/core/src/state/datasets_2d_figure.rs index df2558e2..048d9919 100644 --- a/crates/core/src/state/datasets_2d_figure.rs +++ b/crates/core/src/state/datasets_2d_figure.rs @@ -168,24 +168,8 @@ impl Nmr2DDataset { fn nmr_axes(spectrum: &plotx_processing::Spectrum2D) -> (Axis, Axis) { let (f2_lo, f2_hi) = spectrum.f2_bounds(); let (f1_lo, f1_hi) = spectrum.f1_bounds(); - let f2 = match spectrum.f2_domain { - plotx_io::Domain::Time => Axis::new("F2 acquisition time (s)", f2_lo, f2_hi), - plotx_io::Domain::Frequency => Axis::new( - format!("{} chemical shift (ppm)", spectrum.direct.nucleus), - f2_lo, - f2_hi, - ) - .reversed(true), - }; - let f1 = match spectrum.f1_domain { - plotx_io::Domain::Time => Axis::new("F1 acquisition time (s)", f1_lo, f1_hi), - plotx_io::Domain::Frequency => Axis::new( - format!("{} chemical shift (ppm)", spectrum.indirect.nucleus), - f1_lo, - f1_hi, - ) - .reversed(true), - }; + let f2 = crate::figures::nmr_axis(&spectrum.direct, f2_lo, f2_hi); + let f1 = crate::figures::nmr_axis(&spectrum.indirect, f1_lo, f1_hi); (f2, f1) } diff --git a/crates/core/src/state/datasets_2d_maps.rs b/crates/core/src/state/datasets_2d_maps.rs index 7a761312..d63d533f 100644 --- a/crates/core/src/state/datasets_2d_maps.rs +++ b/crates/core/src/state/datasets_2d_maps.rs @@ -102,9 +102,24 @@ impl Nmr2DDataset { }); } + pub fn dosy_input_error(&self) -> Option<&'static str> { + match &self.processed { + Processed2D::Stack(stack) + if stack.direct.unit == Some(nmr::axis::AxisUnit::Ppm) + && stack.direct_domain == plotx_io::Domain::Frequency => + { + None + } + _ => Some("DOSY maps require a stack of frequency-domain spectra calibrated in ppm."), + } + } + /// Fit every column to build a DOSY map. Only meaningful for diffusion /// datasets. pub fn build_dosy_map(&mut self) -> bool { + if self.dosy_input_error().is_some() { + return false; + } let (Processed2D::Stack(stack), Some(axis), Some(meta)) = ( &self.processed, &self.data.pseudo_axis, @@ -132,6 +147,9 @@ impl Nmr2DDataset { /// diffusion metadata and a gradient-encoded ruler; each gradient value is /// converted to a Stejskal–Tanner b-factor before inversion. pub fn build_ilt_map(&mut self, params: IltParams) -> bool { + if self.dosy_input_error().is_some() { + return false; + } let (Processed2D::Stack(stack), Some(axis), Some(meta)) = ( &self.processed, &self.data.pseudo_axis, diff --git a/crates/core/src/state/datasets_dispatch.rs b/crates/core/src/state/datasets_dispatch.rs index 98011aa3..a103a465 100644 --- a/crates/core/src/state/datasets_dispatch.rs +++ b/crates/core/src/state/datasets_dispatch.rs @@ -157,10 +157,13 @@ impl Dataset { pub fn summary(&self) -> String { match self { Dataset::Nmr(d) => format!( - "{} · {} pts · {:.2} MHz", - d.data.nucleus, + "{} · {} pts · {}", + d.data.nucleus(), d.data.len(), - d.data.observe_freq_mhz + d.data.axes()[0] + .observe_frequency_mhz() + .map(|value| format!("{value:.1} MHz")) + .unwrap_or_else(|| "frequency unknown".into()) ), Dataset::Nmr2D(d) => d.summary(), Dataset::Table(d) => d.summary(), @@ -496,12 +499,12 @@ impl Dataset { /// rather than two derivations that agree only until one of them changes. pub fn factory_pipeline(&self, axis: PhaseAxis) -> Option { match self { - Dataset::Nmr(n) if axis == PhaseAxis::Direct => Some(match n.data.domain { + Dataset::Nmr(n) if axis == PhaseAxis::Direct => Some(match n.input_domain() { Domain::Time => AxisPipeline::default_1d(), Domain::Frequency => AxisPipeline::frequency_1d(), }), Dataset::Nmr2D(n) => { - let params = match n.data.domain { + let params = match n.input_domain(axis).ok()? { Domain::Time => Params2D::default_for(n.preset), Domain::Frequency => Params2D::frequency_domain(n.preset), }; @@ -527,7 +530,7 @@ impl Dataset { } /// Parameters produced by the currently enabled automatic Phase step. - /// This mirrors the processing kernels so switching to manual is lossless. + /// Uses the retained library result so switching to manual is lossless. pub fn automatic_phase_params(&self, axis: PhaseAxis) -> Option<(f64, f64, f64)> { let pipe = self.axis_pipeline(axis)?; let method = @@ -538,27 +541,19 @@ impl Dataset { StepKind::Phase(params) => params.auto, _ => None, })?; - match self { - Dataset::Nmr(n) => Some(plotx_processing::auto_phase(n.base.as_frequency()?, method)), - Dataset::Nmr2D(n) => match &n.base { - Processed2D::Ft(s) => { - let peak_arg = s - .data - .iter() - .max_by(|a, b| a.norm().total_cmp(&b.norm())) - .map_or(0.0, |value| value.arg()); - let (f2, f1) = s.peak_pivot_fracs(); - Some((peak_arg, 0.0, if axis == PhaseAxis::F1 { f1 } else { f2 })) - } - Processed2D::Stack(s) if axis == PhaseAxis::F2 => { - let (phase0, phase1) = - plotx_processing::fft2::absorptive_phase(&s.traces).unwrap_or((0.0, 0.0)); - Some((phase0, phase1, s.peak_pivot_frac())) - } - Processed2D::Stack(_) => None, - }, - _ => None, - } + let step = pipe.steps.iter().find(|step| { + step.enabled + && matches!(step.kind, StepKind::Phase(params) if params.auto == Some(method)) + })?; + let reports = match self { + Dataset::Nmr(n) => &n.phase_reports, + Dataset::Nmr2D(n) => &n.phase_reports, + _ => return None, + }; + reports + .iter() + .find(|report| report.step == step.id) + .map(|report| report.recipe_parameters()) } pub fn pivot_ppm(&self, axis: PhaseAxis) -> Option { diff --git a/crates/core/src/state/field.rs b/crates/core/src/state/field.rs index 9b118ac3..4192c8c6 100644 --- a/crates/core/src/state/field.rs +++ b/crates/core/src/state/field.rs @@ -15,14 +15,19 @@ use crate::automation::{ CAP_FIELD_XPS_SPECTRUM, CapabilityId, }; use plotx_figure::{ContourStyle, SeriesEncoding}; -use std::collections::{BTreeMap, BTreeSet}; +use std::collections::BTreeMap; #[path = "field_contour.rs"] mod field_contour; pub use field_contour::*; #[path = "field_mass_spec.rs"] mod field_mass_spec; - +#[path = "field_metadata.rs"] +mod field_metadata; +#[path = "field_nmr.rs"] +mod field_nmr; +use field_metadata::LINE_X_UNIT_METADATA_KEY; +pub use field_metadata::{FieldCapabilities, FieldDescriptor, FieldMetadata}; impl super::Dataset { /// Describes stable child fields and their encoding capabilities. pub fn field_descriptors(&self) -> Vec { @@ -75,13 +80,17 @@ impl super::Dataset { }, capabilities(id, &[CAP_FIELD_NMR_SIGNAL]), vec![nmr.processed.values().len()], - vec![match nmr.output_domain() { - plotx_io::Domain::Time => "s".to_owned(), - plotx_io::Domain::Frequency => "ppm".to_owned(), - }], + vec![ + plotx_processing::axis_unit_label( + nmr.native_processed.axes()[0].unit, + ) + .to_owned(), + ], "line", ) - .with_line_x_unit(domain_unit(nmr.output_domain())) + .with_line_x_unit( + plotx_processing::axis_unit_label(nmr.native_processed.axes()[0].unit), + ) }) .collect::>(); fields.extend(nmr.craft_field_specs().filter_map(|spec| { @@ -115,8 +124,8 @@ impl super::Dataset { plotx_processing::Processed2D::Ft(spectrum) => ( vec![spectrum.f1_size, spectrum.f2_size], vec![ - domain_unit(spectrum.f1_domain), - domain_unit(spectrum.f2_domain), + spectrum.indirect.unit_label().to_owned(), + spectrum.direct.unit_label().to_owned(), ], ), plotx_processing::Processed2D::Stack(_) => unreachable!("true 2D is FT"), @@ -161,12 +170,16 @@ impl super::Dataset { unreachable!("pseudo 2D is stack") }; let mut fields = Vec::new(); - if let Some(id) = nmr.field_catalog.id_for_key("nmr.stack") { + if let Some(id) = nmr.field_catalog.id_for_key(nmr.stack_field_key()) { fields.push( descriptor( id, - "nmr.stack", - "Stack", + nmr.stack_field_key(), + if nmr.stack_field_key() == "nmr.observations" { + "Acquired NUS observations" + } else { + "Stack" + }, capabilities( id, &[ @@ -175,11 +188,11 @@ impl super::Dataset { CAP_FIELD_REGION_SERIES, ], ), - vec![nmr.data.rows, nmr.data.cols], - vec![String::new(), domain_unit(stack.direct_domain)], + vec![stack.increments(), stack.ppm.len()], + vec![String::new(), stack.direct.unit_label().to_owned()], "line", ) - .with_line_x_unit(domain_unit(stack.direct_domain)), + .with_line_x_unit(stack.direct.unit_label().to_owned()), ); } if let Some(id) = nmr.field_catalog.id_for_key("nmr.dosy_map") { @@ -193,7 +206,10 @@ impl super::Dataset { "DOSY map", capabilities(id, &[CAP_FIELD_BOUNDED, CAP_FIELD_SCALAR_GRID_2D_REGULAR]), dimensions, - vec!["log10(m2/s)".to_owned(), domain_unit(stack.direct_domain)], + vec![ + "log10(m2/s)".to_owned(), + stack.direct.unit_label().to_owned(), + ], "contour", )); } @@ -208,7 +224,10 @@ impl super::Dataset { "ILT map", capabilities(id, &[CAP_FIELD_BOUNDED, CAP_FIELD_SCALAR_GRID_2D_REGULAR]), dimensions, - vec!["log10(m2/s)".to_owned(), domain_unit(stack.direct_domain)], + vec![ + "log10(m2/s)".to_owned(), + stack.direct.unit_label().to_owned(), + ], "contour", )); } @@ -451,7 +470,6 @@ impl super::Dataset { .collect(), } } - pub fn default_field_id(&self) -> Option { if let Self::MassSpec(dataset) = self { return field_mass_spec::default_field_id(dataset); @@ -460,25 +478,23 @@ impl super::Dataset { && !dataset.is_true_2d() { let key = match dataset.display { - super::PseudoDisplay::Stack => "nmr.stack", + super::PseudoDisplay::Stack => dataset.stack_field_key(), super::PseudoDisplay::DosyMap => match dataset.dosy_method { super::DosyMethod::MonoExp if dataset.dosy_map.is_some() => "nmr.dosy_map", super::DosyMethod::Ilt(_) if dataset.ilt_map.is_some() => "nmr.ilt_map", - _ => "nmr.stack", + _ => dataset.stack_field_key(), }, }; return dataset.field_catalog.id_for_key(key); } self.field_descriptors().first().map(|field| field.id) } - pub fn has_field(&self, id: FieldId) -> bool { if let Self::MassSpec(dataset) = self { return dataset.field_catalog.key_for_id(id).is_some(); } - self.field_descriptors().iter().any(|field| field.id == id) + self.field_descriptor(id).is_some() } - pub fn field_descriptor(&self, id: FieldId) -> Option { if let Self::MassSpec(dataset) = self { return field_mass_spec::descriptor(dataset, id); @@ -486,6 +502,7 @@ impl super::Dataset { self.field_descriptors() .into_iter() .find(|field| field.id == id) + .or_else(|| field_nmr::inactive_descriptor(self, id)) } /// A persisted encoding is valid only when its source field exposes the @@ -628,6 +645,7 @@ impl super::Dataset { "nmr.stack".to_owned(), "nmr.dosy_map".to_owned(), "nmr.ilt_map".to_owned(), + "nmr.observations".to_owned(), ], Self::Table(_) => vec!["table.default_series".to_owned()], Self::Electrophysiology(dataset) => (0..dataset.data.channels.len()) @@ -656,15 +674,6 @@ impl super::Dataset { } } } - -fn domain_unit(domain: plotx_io::Domain) -> String { - match domain { - plotx_io::Domain::Time => "s", - plotx_io::Domain::Frequency => "ppm", - } - .to_owned() -} - /// Central capability gate for scalar fields. A provider must derive /// `regular` from its actual coordinate representation, not from its domain. pub fn scalar_grid_capabilities(regular: bool, extra: &[&str]) -> FieldCapabilities { @@ -680,78 +689,6 @@ pub fn scalar_grid_capabilities(regular: bool, extra: &[&str]) -> FieldCapabilit ) } -/// Stable child-resource metadata, including the capabilities used by encoding -/// and chart applicability checks. -#[derive(Clone, Debug, PartialEq, Eq)] -pub struct FieldDescriptor { - pub id: FieldId, - pub local_id: String, - pub name: String, - /// The scientific concept represented by this field. This is required so - /// every new field participates in the v1 summary contract by construction. - pub scientific_observation: SummaryPart, - pub capabilities: FieldCapabilities, - pub dimensions: Vec, - pub units: Vec, - pub metadata: FieldMetadata, -} - -impl FieldDescriptor { - pub(crate) fn with_line_x_unit(mut self, unit: impl Into) -> Self { - self.metadata - .0 - .insert(LINE_X_UNIT_METADATA_KEY.to_owned(), unit.into()); - self - } - - pub fn line_x_unit(&self) -> Option<&str> { - self.metadata.line_x_unit() - } -} - -#[derive(Clone, Debug, Default, PartialEq, Eq)] -pub struct FieldCapabilities(BTreeSet); - -impl FieldCapabilities { - pub fn new(values: impl IntoIterator) -> Self { - Self(values.into_iter().collect()) - } - - pub fn contains(&self, capability: &str) -> bool { - self.0.contains(capability) - } - - /// Reject scalar-grid renderers for a colored raster even when a malformed - /// provider advertises both mutually exclusive capabilities. - pub fn supports(&self, required: &[&str]) -> bool { - required.iter().all(|capability| self.contains(capability)) - && !(required.contains(&CAP_FIELD_SCALAR_GRID_2D_REGULAR) - && self.contains(CAP_FIELD_COLORED_RASTER_2D)) - } - - pub fn iter(&self) -> impl Iterator { - self.0.iter() - } -} - -#[derive(Clone, Debug, Default, PartialEq, Eq)] -pub struct FieldMetadata(pub BTreeMap); - -const LINE_X_UNIT_METADATA_KEY: &str = "line_x_unit"; - -impl FieldMetadata { - pub fn recommended_encoding(&self) -> Option<&str> { - self.0.get("recommended_encoding").map(String::as_str) - } - - pub fn line_x_unit(&self) -> Option<&str> { - self.0 - .get(LINE_X_UNIT_METADATA_KEY) - .map(String::as_str) - .filter(|unit| !unit.is_empty()) - } -} - /// A creation-time request. It is resolved to a concrete `SeriesEncoding` /// before a `SeriesBinding` enters document state. #[derive(Clone, Copy, Debug, Default, PartialEq, Eq)] diff --git a/crates/core/src/state/field_catalog.rs b/crates/core/src/state/field_catalog.rs index 3e20aa6e..a6871d63 100644 --- a/crates/core/src/state/field_catalog.rs +++ b/crates/core/src/state/field_catalog.rs @@ -220,8 +220,9 @@ pub(crate) fn pseudo_axis_display_scale(unit: &str) -> f64 { pub(crate) fn attach_pseudo_trace_collection( catalog: &mut FieldCatalog, - data: &plotx_io::NmrData2D, + data: &plotx_io::nmr_series::NmrSeriesSource, ) { + attach_nus_observations(catalog, data); let Some(field) = catalog.id_for_key("nmr.stack") else { return; }; @@ -273,6 +274,42 @@ pub(crate) fn attach_pseudo_trace_collection( ); } +fn attach_nus_observations( + catalog: &mut FieldCatalog, + data: &plotx_io::nmr_series::NmrSeriesSource, +) { + let (Some(nus), Some(field)) = (&data.nus, catalog.id_for_key("nmr.observations")) else { + return; + }; + let id = TraceCollectionId::derived(data.source.as_bytes(), b"nmr.observations"); + let items = nus + .schedule + .iter() + .enumerate() + .map(|(ordinal, coordinate)| TraceItemDescriptor { + id: TraceItemId::derived(id, &(ordinal as u64).to_le_bytes()), + parameters: vec![TraceItemParameter { + key: "observation".into(), + name: "Observation".into(), + value: TraceParameterValue::Text { + value: format!("Observation {} (grid index {})", ordinal + 1, coordinate), + }, + }], + primary_label_parameter: "observation".into(), + label_override: None, + }) + .collect(); + catalog.set_trace_collection( + field, + TraceCollectionCatalog { + id, + axis_quantity: "Acquired NUS observation".into(), + axis_unit: "".into(), + items, + }, + ); +} + pub(crate) fn attach_electrophysiology_trace_collections( catalog: &mut FieldCatalog, data: &plotx_io::ElectrophysiologyData, @@ -378,6 +415,7 @@ pub(crate) fn nmr2d_field_catalog() -> FieldCatalog { "nmr.stack".to_owned(), "nmr.dosy_map".to_owned(), "nmr.ilt_map".to_owned(), + "nmr.observations".to_owned(), ]) } diff --git a/crates/core/src/state/field_metadata.rs b/crates/core/src/state/field_metadata.rs new file mode 100644 index 00000000..f2807117 --- /dev/null +++ b/crates/core/src/state/field_metadata.rs @@ -0,0 +1,77 @@ +use super::{ + CAP_FIELD_COLORED_RASTER_2D, CAP_FIELD_SCALAR_GRID_2D_REGULAR, CapabilityId, FieldId, + SummaryPart, +}; +use std::collections::{BTreeMap, BTreeSet}; + +/// Stable child-resource metadata, including the capabilities used by encoding +/// and chart applicability checks. +#[derive(Clone, Debug, PartialEq, Eq)] +pub struct FieldDescriptor { + pub id: FieldId, + pub local_id: String, + pub name: String, + /// The scientific concept represented by this field. This is required so + /// every new field participates in the v1 summary contract by construction. + pub scientific_observation: SummaryPart, + pub capabilities: FieldCapabilities, + pub dimensions: Vec, + pub units: Vec, + pub metadata: FieldMetadata, +} + +impl FieldDescriptor { + pub(crate) fn with_line_x_unit(mut self, unit: impl Into) -> Self { + self.metadata + .0 + .insert(LINE_X_UNIT_METADATA_KEY.to_owned(), unit.into()); + self + } + + pub fn line_x_unit(&self) -> Option<&str> { + self.metadata.line_x_unit() + } +} + +#[derive(Clone, Debug, Default, PartialEq, Eq)] +pub struct FieldCapabilities(BTreeSet); + +impl FieldCapabilities { + pub fn new(values: impl IntoIterator) -> Self { + Self(values.into_iter().collect()) + } + + pub fn contains(&self, capability: &str) -> bool { + self.0.contains(capability) + } + + /// Reject scalar-grid renderers for a colored raster even when a malformed + /// provider advertises both mutually exclusive capabilities. + pub fn supports(&self, required: &[&str]) -> bool { + required.iter().all(|capability| self.contains(capability)) + && !(required.contains(&CAP_FIELD_SCALAR_GRID_2D_REGULAR) + && self.contains(CAP_FIELD_COLORED_RASTER_2D)) + } + + pub fn iter(&self) -> impl Iterator { + self.0.iter() + } +} + +#[derive(Clone, Debug, Default, PartialEq, Eq)] +pub struct FieldMetadata(pub BTreeMap); + +pub(super) const LINE_X_UNIT_METADATA_KEY: &str = "line_x_unit"; + +impl FieldMetadata { + pub fn recommended_encoding(&self) -> Option<&str> { + self.0.get("recommended_encoding").map(String::as_str) + } + + pub fn line_x_unit(&self) -> Option<&str> { + self.0 + .get(LINE_X_UNIT_METADATA_KEY) + .map(String::as_str) + .filter(|unit| !unit.is_empty()) + } +} diff --git a/crates/core/src/state/field_nmr.rs b/crates/core/src/state/field_nmr.rs new file mode 100644 index 00000000..cf5a11ed --- /dev/null +++ b/crates/core/src/state/field_nmr.rs @@ -0,0 +1,61 @@ +//! Stable NMR fields remain valid binding targets while another result is active. + +use super::*; + +pub(super) fn inactive_descriptor( + dataset: &super::super::Dataset, + id: FieldId, +) -> Option { + let super::super::Dataset::Nmr2D(nmr) = dataset else { + return None; + }; + let key = nmr.field_catalog.key_for_id(id)?; + let (name, recommended, capabilities) = match key { + "nmr.stack" => ( + "Stack", + "line", + vec![CAP_FIELD_CURVE_1D, CAP_FIELD_TRACE_COLLECTION], + ), + "nmr.observations" if nmr.data.nus.is_some() => ( + "Acquired NUS observations", + "line", + vec![CAP_FIELD_CURVE_1D, CAP_FIELD_TRACE_COLLECTION], + ), + "nmr.real" => ( + "Real", + "contour", + vec![CAP_FIELD_BOUNDED, CAP_FIELD_SCALAR_GRID_2D_REGULAR], + ), + "nmr.magnitude" => ( + "Magnitude", + "heatmap", + vec![CAP_FIELD_BOUNDED, CAP_FIELD_SCALAR_GRID_2D_REGULAR], + ), + "nmr.dosy_map" => ( + "DOSY map", + "contour", + vec![CAP_FIELD_BOUNDED, CAP_FIELD_SCALAR_GRID_2D_REGULAR], + ), + "nmr.ilt_map" => ( + "ILT map", + "contour", + vec![CAP_FIELD_BOUNDED, CAP_FIELD_SCALAR_GRID_2D_REGULAR], + ), + _ => return None, + }; + // These are the provider's declared rendering types, not a fabricated payload. + // Active field discovery and payload access still require an actual result. + Some(FieldDescriptor { + id, + local_id: key.to_owned(), + name: name.to_owned(), + scientific_observation: SummaryPart::new(format!("field:{key}"), name), + capabilities: FieldCapabilities::new(capabilities.into_iter().map(CapabilityId::new)), + dimensions: vec![], + units: vec![], + metadata: FieldMetadata(BTreeMap::from([ + ("recommended_encoding".into(), recommended.into()), + ("availability".into(), "inactive".into()), + ])), + }) +} diff --git a/crates/core/src/state/field_payload.rs b/crates/core/src/state/field_payload.rs index e0defaed..de1400e0 100644 --- a/crates/core/src/state/field_payload.rs +++ b/crates/core/src/state/field_payload.rs @@ -274,7 +274,7 @@ impl super::Dataset { .cloned() .unwrap_or_else(|| { let (source, algorithm) = match self { - Self::Nmr(dataset) => (dataset.data.source.as_str(), None), + Self::Nmr(dataset) => (dataset.data.source(), None), Self::Nmr2D(dataset) => ( dataset.data.source.as_str(), Some(FieldAlgorithmProvenance { @@ -318,7 +318,7 @@ fn nmr_field_payload(dataset: &super::Nmr2DDataset, id: FieldId) -> Option + if dataset.field_catalog.id_for_key(dataset.stack_field_key()) == Some(id) => { let values = stack .traces @@ -360,7 +360,7 @@ fn nmr_field_representation( }) } plotx_processing::Processed2D::Stack(_) - if dataset.field_catalog.id_for_key("nmr.stack") == Some(id) => + if dataset.field_catalog.id_for_key(dataset.stack_field_key()) == Some(id) => { Some(FieldRepresentation::Curve1D) } diff --git a/crates/core/src/state/field_runtime_tests.rs b/crates/core/src/state/field_runtime_tests.rs index 6ea5dc50..c0b2d2d4 100644 --- a/crates/core/src/state/field_runtime_tests.rs +++ b/crates/core/src/state/field_runtime_tests.rs @@ -1,5 +1,5 @@ use super::*; -use crate::state::{ComputeService, DataBinding, Dataset, Nmr2DDataset, PlotxApp}; +use crate::state::{ComputeService, DataBinding, Dataset, PlotxApp}; use num_complex::Complex64; use plotx_figure::{ Color, ColorSource, ContourBasePolicy, ContourLevelSpec, ContourSpec, ContourStyle, @@ -76,21 +76,24 @@ pub(super) fn grid_dataset(label: &str, values: &[f32]) -> Dataset { .copied() .map(|value| Complex64::new(f64::from(value), 0.0)) .collect(); - Dataset::Nmr2D(Box::new(Nmr2DDataset::load(NmrData2D { - data: values, - rows: 4, - cols: 4, - domain: Domain::Frequency, - direct: dimension("1H"), - indirect: dimension("13C"), - quad: QuadMode::Complex, - indirect_conjugate: false, - experiment: None, - pseudo_axis: None, - diffusion: None, - nus: None, - source: label.to_owned(), - }))) + Dataset::Nmr2D(Box::new( + crate::nmr_test_support::load_2d(NmrData2D { + data: values, + rows: 4, + cols: 4, + domain: Domain::Frequency, + direct: dimension("1H"), + indirect: dimension("13C"), + quad: QuadMode::Complex, + indirect_conjugate: false, + experiment: None, + pseudo_axis: None, + diffusion: None, + nus: None, + source: label.to_owned(), + }) + .unwrap(), + )) } fn absolute_signed_contour() -> ContourSpec { @@ -138,6 +141,7 @@ fn settle_estimates(service: &mut ComputeService) { | crate::state::Done::Craft { .. } | crate::state::Done::CraftFailed { .. } | crate::state::Done::Processing2D { .. } + | crate::state::Done::Processing2DFailed { .. } | crate::state::Done::Cancelled { .. } | crate::state::Done::Failed { .. } => { panic!("unexpected non-estimate job while settling estimates"); diff --git a/crates/core/src/state/field_tests.rs b/crates/core/src/state/field_tests.rs index c1625bdd..f12018a0 100644 --- a/crates/core/src/state/field_tests.rs +++ b/crates/core/src/state/field_tests.rs @@ -1,6 +1,6 @@ use super::*; use crate::state::{ - AfmDataset, Dataset, ElectrophysiologyDataset, Nmr2DDataset, ToolGroup, default_contour_spec, + AfmDataset, Dataset, ElectrophysiologyDataset, ToolGroup, default_contour_spec, default_encoding, }; use plotx_figure::HeatmapSpec; @@ -118,8 +118,8 @@ fn afm_dataset(scan_size_x: f64, raw: Vec, forces: bool) -> Dataset { #[test] fn cheap_representation_matches_the_materialized_payload() { - let nmr_1d = Dataset::Nmr(Box::new(crate::state::NmrDataset::load( - plotx_io::NmrData { + let nmr_1d = Dataset::Nmr(Box::new( + crate::nmr_test_support::load_1d(plotx_io::NmrData { points: vec![num_complex::Complex64::new(1.0, 0.0); 8], domain: plotx_io::Domain::Frequency, spectral_width_hz: 4_000.0, @@ -128,14 +128,19 @@ fn cheap_representation_matches_the_materialized_payload() { nucleus: "1H".to_owned(), source: "representation test".to_owned(), group_delay: 0.0, - }, - ))); + }) + .unwrap(), + )); assert_representation_matches_payload(&nmr_1d, "nmr 1d"); - let nmr_2d = Dataset::Nmr2D(Box::new(Nmr2DDataset::load(nmr2d_data("true 2d", None)))); + let nmr_2d = Dataset::Nmr2D(Box::new( + crate::nmr_test_support::load_2d(nmr2d_data("true 2d", None)).unwrap(), + )); assert_representation_matches_payload(&nmr_2d, "nmr 2d"); - let mut irregular = Dataset::Nmr2D(Box::new(Nmr2DDataset::load(nmr2d_data("explicit", None)))); + let mut irregular = Dataset::Nmr2D(Box::new( + crate::nmr_test_support::load_2d(nmr2d_data("explicit", None)).unwrap(), + )); let Dataset::Nmr2D(nmr) = &mut irregular else { panic!("fixture is NMR 2D"); }; @@ -145,16 +150,19 @@ fn cheap_representation_matches_the_materialized_payload() { Arc::make_mut(spectrum).f1_ppm[2] += 0.25; assert_representation_matches_payload(&irregular, "nmr 2d, explicitly sampled"); - let pseudo = Dataset::Nmr2D(Box::new(Nmr2DDataset::load(nmr2d_data( - "pseudo 2d", - Some(plotx_io::PseudoAxis { - name: "delay".to_owned(), - kind: plotx_io::PseudoKind::Delay, - values: vec![0.1, 0.2, 0.3, 0.4], - unit: "s".to_owned(), - source: plotx_io::AxisSource::EmbeddedList, - }), - )))); + let pseudo = Dataset::Nmr2D(Box::new( + crate::nmr_test_support::load_2d(nmr2d_data( + "pseudo 2d", + Some(plotx_io::PseudoAxis { + name: "delay".to_owned(), + kind: plotx_io::PseudoKind::Delay, + values: vec![0.1, 0.2, 0.3, 0.4], + unit: "s".to_owned(), + source: plotx_io::AxisSource::EmbeddedList, + }), + )) + .unwrap(), + )); assert!( !matches!(&pseudo, Dataset::Nmr2D(nmr) if nmr.is_true_2d()), "the pseudo-2D fixture must exercise the stack branch" @@ -587,26 +595,29 @@ fn magnitude_field_renders_magnitude_instead_of_falling_back_to_real() { nucleus: nucleus.to_owned(), group_delay: 0.0, }; - let dataset = Dataset::Nmr2D(Box::new(Nmr2DDataset::load(plotx_io::NmrData2D { - data: vec![ - num_complex::Complex64::new(-3.0, 4.0), - num_complex::Complex64::new(5.0, 12.0), - num_complex::Complex64::new(8.0, 15.0), - num_complex::Complex64::new(-7.0, 24.0), - ], - rows: 2, - cols: 2, - domain: plotx_io::Domain::Frequency, - direct: dimension("1H"), - indirect: dimension("13C"), - quad: plotx_io::QuadMode::Complex, - indirect_conjugate: false, - experiment: None, - pseudo_axis: None, - diffusion: None, - nus: None, - source: "magnitude test".to_owned(), - }))); + let dataset = Dataset::Nmr2D(Box::new( + crate::nmr_test_support::load_2d(plotx_io::NmrData2D { + data: vec![ + num_complex::Complex64::new(-3.0, 4.0), + num_complex::Complex64::new(5.0, 12.0), + num_complex::Complex64::new(8.0, 15.0), + num_complex::Complex64::new(-7.0, 24.0), + ], + rows: 2, + cols: 2, + domain: plotx_io::Domain::Frequency, + direct: dimension("1H"), + indirect: dimension("13C"), + quad: plotx_io::QuadMode::Complex, + indirect_conjugate: false, + experiment: None, + pseudo_axis: None, + diffusion: None, + nus: None, + source: "magnitude test".to_owned(), + }) + .unwrap(), + )); let magnitude = dataset .field_descriptors() .into_iter() @@ -632,21 +643,24 @@ fn default_nmr_contour_never_builds_geometry_inline() { nucleus: nucleus.to_owned(), group_delay: 0.0, }; - let dataset = Dataset::Nmr2D(Box::new(Nmr2DDataset::load(plotx_io::NmrData2D { - data: vec![num_complex::Complex64::new(1.0, 0.0); 16], - rows: 4, - cols: 4, - domain: plotx_io::Domain::Frequency, - direct: dimension("1H"), - indirect: dimension("13C"), - quad: plotx_io::QuadMode::Complex, - indirect_conjugate: false, - experiment: None, - pseudo_axis: None, - diffusion: None, - nus: None, - source: "cache test".to_owned(), - }))); + let dataset = Dataset::Nmr2D(Box::new( + crate::nmr_test_support::load_2d(plotx_io::NmrData2D { + data: vec![num_complex::Complex64::new(1.0, 0.0); 16], + rows: 4, + cols: 4, + domain: plotx_io::Domain::Frequency, + direct: dimension("1H"), + indirect: dimension("13C"), + quad: plotx_io::QuadMode::Complex, + indirect_conjugate: false, + experiment: None, + pseudo_axis: None, + diffusion: None, + nus: None, + source: "cache test".to_owned(), + }) + .unwrap(), + )); let real = dataset .field_descriptors() .into_iter() diff --git a/crates/core/src/state/mod.rs b/crates/core/src/state/mod.rs index a97258a7..1383d0a5 100644 --- a/crates/core/src/state/mod.rs +++ b/crates/core/src/state/mod.rs @@ -12,13 +12,10 @@ use crate::{ use plotx_analysis::diffusion::{DiffusionMap, diffusion_map}; use plotx_analysis::ilt::{IltResult, ilt_map, log_grid}; use plotx_figure::{Axis, Color, Figure}; -use plotx_io::{ - AfmData, Domain, ElectricalQuantity, ElectricalUnit, ElectrophysiologyData, NmrData, NmrData2D, -}; +use plotx_io::{AfmData, Domain, ElectricalQuantity, ElectricalUnit, ElectrophysiologyData}; use plotx_processing::{ AxisPipeline, DisplayMode, Params2D, PhaseParams, Preset2D, Processed1D, Processed2D, Spectrum, - StepId, StepKind, process_2d, reapply_2d, reapply_output, recommend_preset, - transform_output_base, + StepId, StepKind, recommend_preset, }; mod afm; @@ -58,7 +55,9 @@ mod content; mod craft; mod craft_fields; mod cursors; +mod data_import; mod dataset_identity; +pub use data_import::DataImports; mod dataset_trace; mod datasets; mod datasets_2d_figure; @@ -90,7 +89,6 @@ mod mass_spec_xic; mod multiplet; mod nmr_integrals; mod nmr_integrals_2d; -mod nus; mod page_fit; mod panel; mod panel_label; diff --git a/crates/core/src/state/nmr_integrals.rs b/crates/core/src/state/nmr_integrals.rs index eb225c9a..b071311a 100644 --- a/crates/core/src/state/nmr_integrals.rs +++ b/crates/core/src/state/nmr_integrals.rs @@ -85,9 +85,9 @@ impl NmrDataset { } pub fn pivot_ppm(&self) -> f64 { - let Some(base) = self.base.as_frequency() else { + if self.base.as_frequency().is_none() { return 0.0; - }; + } let (lo, hi) = self.ppm_ends(); let frac = self .pipeline @@ -99,7 +99,11 @@ impl NmrDataset { // show the peak the pass actually rotates about so the on-plot handle // sits where the user expects instead of pinned to an edge. StepKind::Phase(p) => Some(match p.auto { - Some(_) => plotx_processing::phase::peak_pivot_frac(&base.values), + Some(_) => self + .phase_reports + .iter() + .find(|report| report.step == s.id) + .map_or(p.pivot_frac, |report| report.recipe_parameters().2), None => p.pivot_frac, }), _ => None, diff --git a/crates/core/src/state/nmr_integrals_2d.rs b/crates/core/src/state/nmr_integrals_2d.rs index 9bf567d6..4d8def70 100644 --- a/crates/core/src/state/nmr_integrals_2d.rs +++ b/crates/core/src/state/nmr_integrals_2d.rs @@ -37,7 +37,13 @@ impl Nmr2DDataset { let grid: Vec = spectrum .data .iter() - .map(|value| mode.reduce(value)) + .enumerate() + .map(|(index, value)| match mode { + DisplayMode::Real => value.re, + DisplayMode::Magnitude => spectrum + .magnitude_at(index) + .expect("view planes have the same shape"), + }) .collect(); let prepared = plotx_analysis::integrate_2d::IntegrationGrid2D::new( &spectrum.f2_ppm, @@ -190,7 +196,7 @@ mod tests { let mut dataset = test_dataset(); dataset.integrals = vec![integral(0, 123.0, Some(1.0))]; - dataset.rebuild(); + dataset.rebuild().unwrap(); assert_eq!(dataset.integrals[0].volume, 123.0); dataset.recompute_integrals().unwrap(); @@ -241,7 +247,7 @@ mod tests { nucleus: "X".to_owned(), group_delay: 0.0, }; - Nmr2DDataset::load(NmrData2D { + crate::nmr_test_support::load_2d(NmrData2D { data: vec![Complex64::new(1.0, 0.0); 4], rows: 2, cols: 2, @@ -256,5 +262,6 @@ mod tests { nus: None, source: "test".to_owned(), }) + .unwrap() } } diff --git a/crates/core/src/state/nus.rs b/crates/core/src/state/nus.rs deleted file mode 100644 index 47ec8d10..00000000 --- a/crates/core/src/state/nus.rs +++ /dev/null @@ -1,46 +0,0 @@ -//! Non-uniform-sampling schedule entry for a 2D dataset. - -use super::*; - -impl Nmr2DDataset { - /// Sampling indices are `base`-indexed on input and stored 0-based; the list - /// must hold exactly one unique in-grid index per acquired increment. - pub fn set_nus_schedule(&mut self, values: &[usize], base: usize) -> Result<(), String> { - let Some(nus) = self.data.nus.as_ref() else { - return Err("This dataset is not non-uniformly sampled.".into()); - }; - let (grid, acquired) = (nus.grid, nus.acquired); - if values.len() != acquired { - return Err(format!( - "Expected {acquired} sampling indices, got {}.", - values.len() - )); - } - let mut zero_based = Vec::with_capacity(values.len()); - for &v in values { - if v < base || v >= base + grid { - return Err(format!( - "Index {v} is outside the grid [{base}, {}].", - base + grid - 1 - )); - } - zero_based.push(v - base); - } - let mut unique = zero_based.clone(); - unique.sort_unstable(); - unique.dedup(); - if unique.len() != zero_based.len() { - return Err("Sampling indices must be unique.".into()); - } - let meta = std::sync::Arc::make_mut(&mut self.data) - .nus - .as_mut() - .unwrap(); - meta.schedule = Some(zero_based); - meta.idx_base = base; - // The cached base was reconstructed from the previous schedule, so it must - // be rebuilt from the FID even though the recipe is unchanged. - self.base_stale = true; - Ok(()) - } -} diff --git a/crates/core/src/state/peaks_tests.rs b/crates/core/src/state/peaks_tests.rs index ca32ed40..c3223646 100644 --- a/crates/core/src/state/peaks_tests.rs +++ b/crates/core/src/state/peaks_tests.rs @@ -76,7 +76,7 @@ fn frequency_app() -> crate::state::PlotxApp { }; let mut app = crate::state::PlotxApp::new(); app.doc.datasets.push(crate::state::Dataset::Nmr(Box::new( - crate::state::NmrDataset::load(data), + crate::state::NmrDataset::load(data).unwrap(), ))); app } @@ -96,7 +96,7 @@ fn apply_reference(app: &mut crate::state::PlotxApp, at_ppm: f64, target_ppm: f6 plotx_processing::StepSource::User, )); let nmr = app.doc.datasets[0].as_nmr_mut().expect("NMR dataset"); - nmr.processed = plotx_processing::reapply_output(&nmr.base, &nmr.pipeline); + nmr.rebuild().unwrap(); } fn resolved_marks(app: &crate::state::PlotxApp) -> Vec { diff --git a/crates/core/src/state/scientific_summary/mod.rs b/crates/core/src/state/scientific_summary/mod.rs index 6f807142..4172947c 100644 --- a/crates/core/src/state/scientific_summary/mod.rs +++ b/crates/core/src/state/scientific_summary/mod.rs @@ -81,12 +81,12 @@ fn format_parts(parts: &[SummaryPart]) -> String { #[cfg(test)] mod tests { use super::*; - use crate::state::{Dataset, NmrDataset, PlotxApp}; + use crate::state::{Dataset, PlotxApp}; use num_complex::Complex64; use plotx_io::{AcquisitionIdentity, Domain, NmrData}; fn nmr_dataset() -> Dataset { - let mut dataset = NmrDataset::load(NmrData { + let mut dataset = crate::nmr_test_support::load_1d(NmrData { points: vec![Complex64::new(1.0, 0.0); 8], domain: Domain::Frequency, spectral_width_hz: 4_000.0, @@ -95,7 +95,8 @@ mod tests { nucleus: "1H".to_owned(), source: "raw/exp1/fid".to_owned(), group_delay: 0.0, - }); + }) + .unwrap(); dataset.acquisition_identity = AcquisitionIdentity { subject: Some("Sample A".to_owned()), acquisition: Some("zg30".to_owned()), diff --git a/crates/core/src/state/scientific_summary/resolver.rs b/crates/core/src/state/scientific_summary/resolver.rs index 957ab206..2a258e29 100644 --- a/crates/core/src/state/scientific_summary/resolver.rs +++ b/crates/core/src/state/scientific_summary/resolver.rs @@ -284,11 +284,11 @@ fn nmr_2d_observation(data: &crate::state::Nmr2DDataset) -> SummaryPart { fn nmr_1d_observation(data: &crate::state::NmrDataset) -> SummaryPart { let domain = data.output_domain(); SummaryPart::new( - format!("nmr:{domain:?}:{}", data.data.nucleus), + format!("nmr:{domain:?}:{}", data.data.nucleus()), if domain == plotx_io::Domain::Time { - format!("{} FID", data.data.nucleus) + format!("{} FID", data.data.nucleus()) } else { - data.data.nucleus.clone() + data.data.nucleus().to_owned() }, ) } @@ -555,7 +555,7 @@ mod tests { use plotx_processing::Slice1D; fn nmr(domain: Domain, subject: &str, acquisition: &str) -> Dataset { - let mut data = NmrDataset::load(NmrData { + let mut data = crate::nmr_test_support::load_1d(NmrData { points: vec![Complex64::new(1.0, 0.0); 8], domain, spectral_width_hz: 4_000.0, @@ -564,7 +564,8 @@ mod tests { nucleus: "1H".to_owned(), source: "fid".to_owned(), group_delay: 0.0, - }); + }) + .unwrap(); data.acquisition_identity = AcquisitionIdentity { subject: Some(subject.to_owned()), acquisition: Some(acquisition.to_owned()), @@ -581,7 +582,7 @@ mod tests { nucleus: nucleus.to_owned(), group_delay: 0.0, }; - Nmr2DDataset::load(NmrData2D { + crate::nmr_test_support::load_2d(NmrData2D { data: vec![Complex64::new(1.0, 0.0); 4], rows: 2, cols: 2, @@ -596,6 +597,7 @@ mod tests { nus: None, source: "2d".to_owned(), }) + .unwrap() } #[test] @@ -608,18 +610,23 @@ mod tests { fn derived_slice_inherits_the_source_subject() { let source = nmr(Domain::Frequency, "Specimen A", "HSQC"); let source_id = source.resource_id(); - let mut derived = Dataset::Nmr(Box::new(NmrDataset::from_slice( - Slice1D { - coordinates: vec![2.0, 1.0], - domain: Domain::Frequency, - values: vec![Complex64::new(1.0, 0.0); 2], - nucleus: "1H".to_owned(), - observe_freq_mhz: 400.0, - position: Some(3.0), - position_domain: Domain::Frequency, - }, - "F2 slice at 3 ppm".to_owned(), - ))); + let mut derived = Dataset::Nmr(Box::new( + NmrDataset::from_slice( + Slice1D { + coordinates: vec![2.0, 1.0], + domain: Domain::Frequency, + values: vec![Complex64::new(1.0, 0.0); 2], + nucleus: "1H".to_owned(), + observe_freq_mhz: Some(400.0), + reference_freq_mhz: Some(400.0), + unit: nmr::axis::AxisUnit::Ppm, + position: Some(3.0), + position_domain: Domain::Frequency, + }, + "F2 slice at 3 ppm".to_owned(), + ) + .unwrap(), + )); derived.set_lineage(Some(DatasetLineage::new( DerivationKind::Slice, [source_id], @@ -669,12 +676,15 @@ mod tests { domain: Domain::Time, values: vec![Complex64::new(1.0, 0.0); 2], nucleus: "1H".to_owned(), - observe_freq_mhz: 400.0, + observe_freq_mhz: Some(400.0), + reference_freq_mhz: Some(400.0), + unit: nmr::axis::AxisUnit::Ppm, position: None, position_domain: Domain::Time, }, "FID".to_owned(), - ); + ) + .unwrap(); let frequency = nmr(Domain::Frequency, "A", "zg30"); let time = nmr_1d_observation(&time); let frequency = nmr_1d_observation(frequency.as_nmr().unwrap()); diff --git a/crates/core/src/state/trace_alignment_tests.rs b/crates/core/src/state/trace_alignment_tests.rs index 588d48cd..745a7bb9 100644 --- a/crates/core/src/state/trace_alignment_tests.rs +++ b/crates/core/src/state/trace_alignment_tests.rs @@ -387,7 +387,9 @@ fn stacked_shift_bounds_union_each_provider_range() { #[test] fn pseudo_increment_uses_the_same_plot_owned_plan() { - let dataset = Dataset::Nmr2D(Box::new(Nmr2DDataset::load(pseudo_data()))); + let dataset = Dataset::Nmr2D(Box::new( + crate::nmr_test_support::load_2d(pseudo_data()).unwrap(), + )); let field = dataset.field_catalog().id_for_key("nmr.stack").unwrap(); let mut app = PlotxApp::new(); app.doc.datasets.push(dataset); @@ -507,7 +509,9 @@ fn selected_channel_projection_preserves_other_channel_bindings() { #[test] fn automatic_alignment_skips_incompatible_x_units() { let (mut app, canvas, object, ids) = alignment_recording_app(); - let pseudo = Dataset::Nmr2D(Box::new(Nmr2DDataset::load(pseudo_data()))); + let pseudo = Dataset::Nmr2D(Box::new( + crate::nmr_test_support::load_2d(pseudo_data()).unwrap(), + )); let field = pseudo.field_catalog().id_for_key("nmr.stack").unwrap(); let mut extra = SeriesBinding::from_field_all(&pseudo, field)[0].clone(); extra.id = SeriesId::new(99); @@ -544,7 +548,9 @@ fn provider_line_x_units_describe_plotted_x_axes() { Some("s") ); } - let pseudo = Dataset::Nmr2D(Box::new(Nmr2DDataset::load(pseudo_data()))); + let pseudo = Dataset::Nmr2D(Box::new( + crate::nmr_test_support::load_2d(pseudo_data()).unwrap(), + )); let field = pseudo.field_catalog().id_for_key("nmr.stack").unwrap(); assert_eq!( pseudo.field_descriptor(field).unwrap().line_x_unit(), @@ -553,16 +559,19 @@ fn provider_line_x_units_describe_plotted_x_axes() { } fn scalar_nmr(source: &str, carrier_ppm: f64) -> Dataset { - Dataset::Nmr(Box::new(NmrDataset::load(plotx_io::NmrData { - points: vec![num_complex::Complex64::new(1.0, 0.0); 8], - domain: plotx_io::Domain::Frequency, - spectral_width_hz: 4_000.0, - observe_freq_mhz: 400.0, - carrier_ppm, - nucleus: "1H".to_owned(), - source: source.to_owned(), - group_delay: 0.0, - }))) + Dataset::Nmr(Box::new( + crate::nmr_test_support::load_1d(plotx_io::NmrData { + points: vec![num_complex::Complex64::new(1.0, 0.0); 8], + domain: plotx_io::Domain::Frequency, + spectral_width_hz: 4_000.0, + observe_freq_mhz: 400.0, + carrier_ppm, + nucleus: "1H".to_owned(), + source: source.to_owned(), + group_delay: 0.0, + }) + .unwrap(), + )) } #[test] @@ -613,7 +622,8 @@ fn ordinary_scalar_line_stack_uses_the_same_alignment_planner() { .as_nmr() .unwrap() .data - .points + .trace() + .unwrap() .iter() .map(|point| (point.re.to_bits(), point.im.to_bits())) .collect::>() @@ -668,7 +678,8 @@ fn ordinary_scalar_line_stack_uses_the_same_alignment_planner() { .as_nmr() .unwrap() .data - .points + .trace() + .unwrap() .iter() .map(|point| (point.re.to_bits(), point.im.to_bits())) .collect::>() diff --git a/crates/core/src/state/trace_provider.rs b/crates/core/src/state/trace_provider.rs index 61da642b..a871fba3 100644 --- a/crates/core/src/state/trace_provider.rs +++ b/crates/core/src/state/trace_provider.rs @@ -70,6 +70,15 @@ impl Dataset { "NMR trace collection item count does not match the acquisition".to_owned(), ); } + if let Some(nus) = &dataset.data.nus { + let observations = catalog + .id_for_key("nmr.observations") + .and_then(|field| catalog.trace_collection(field)) + .ok_or("NUS observations are missing their trace catalog")?; + if observations.items.len() != nus.acquired { + return Err("NUS trace catalog differs from acquired observations".into()); + } + } } Self::Electrophysiology(dataset) => { for field in self diff --git a/crates/core/src/state/trace_provider_tests.rs b/crates/core/src/state/trace_provider_tests.rs index 766ae202..9760e20a 100644 --- a/crates/core/src/state/trace_provider_tests.rs +++ b/crates/core/src/state/trace_provider_tests.rs @@ -33,7 +33,7 @@ fn pseudo_data() -> plotx_io::NmrData2D { #[test] fn pseudo_trace_items_keep_identity_and_format_display_units() { - let mut dataset = Nmr2DDataset::load(pseudo_data()); + let mut dataset = crate::nmr_test_support::load_2d(pseudo_data()).unwrap(); let field = dataset.field_catalog.id_for_key("nmr.stack").unwrap(); let before = dataset .field_catalog @@ -49,7 +49,7 @@ fn pseudo_trace_items_keep_identity_and_format_display_units() { .as_deref(), Some("20 mT/m") ); - dataset.rebuild(); + dataset.rebuild().unwrap(); assert_eq!( before, dataset @@ -433,7 +433,7 @@ fn trace_composer_uses_each_recordings_selected_channel() { fn pseudo_map_display_composes_the_stable_stack_collection() { let mut app = PlotxApp::new(); for _ in 0..2 { - let mut dataset = Nmr2DDataset::load(pseudo_data()); + let mut dataset = crate::nmr_test_support::load_2d(pseudo_data()).unwrap(); dataset.display = PseudoDisplay::DosyMap; app.doc.datasets.push(Dataset::Nmr2D(Box::new(dataset))); } @@ -576,7 +576,9 @@ fn trace_contract_uses_capabilities_concrete_encoding_and_units_not_domain_polic .unwrap(); electrophysiology_descriptor.metadata = FieldMetadata::default(); - let pseudo = Dataset::Nmr2D(Box::new(Nmr2DDataset::load(pseudo_data()))); + let pseudo = Dataset::Nmr2D(Box::new( + crate::nmr_test_support::load_2d(pseudo_data()).unwrap(), + )); let pseudo_field = pseudo.active_trace_collection_field().unwrap(); let pseudo_binding = SeriesBinding::from_field_all(&pseudo, pseudo_field) .into_iter() @@ -618,9 +620,9 @@ fn trace_stack_forces_offset_even_when_the_primary_domain_is_field_stacked() { let mut true_2d = pseudo_data(); true_2d.pseudo_axis = None; let mut app = PlotxApp::new(); - app.doc - .datasets - .push(Dataset::Nmr2D(Box::new(Nmr2DDataset::load(true_2d)))); + app.doc.datasets.push(Dataset::Nmr2D(Box::new( + crate::nmr_test_support::load_2d(true_2d).unwrap(), + ))); app.doc.datasets.push(recording("pA", Some("mV"))); assert_eq!( app.doc.datasets[0].domain().stack_kind(), @@ -691,7 +693,9 @@ fn fixed_prepulse_is_skipped_for_the_varying_abf_test_pulse() { #[test] fn single_and_multi_item_materialization_apply_identical_line_style() { - let dataset = Dataset::Nmr2D(Box::new(Nmr2DDataset::load(pseudo_data()))); + let dataset = Dataset::Nmr2D(Box::new( + crate::nmr_test_support::load_2d(pseudo_data()).unwrap(), + )); let field = dataset.field_catalog().id_for_key("nmr.stack").unwrap(); let mut bindings = SeriesBinding::from_field_all(&dataset, field); for binding in bindings.iter_mut().take(2) { diff --git a/crates/core/src/state/ui_state.rs b/crates/core/src/state/ui_state.rs index 98769dd4..6acbec36 100644 --- a/crates/core/src/state/ui_state.rs +++ b/crates/core/src/state/ui_state.rs @@ -1,6 +1,9 @@ use super::*; +#[path = "ui_state_nmr_import.rs"] +mod nmr_import; use crate::actions::PendingWheelPropertyEdit; use crate::operation::{OperationHistory, OperationId, OperationReport}; +pub use nmr_import::NmrImportDraft; use std::collections::{HashMap, HashSet}; use std::ops::{Deref, DerefMut}; use std::sync::Arc; @@ -205,6 +208,7 @@ pub struct UiState { pub export_options: Option, pub data_export: Option, pub table_import_preview: Option, + pub nmr_import: Option, pub settings_dialog: Option, pub command_palette: Option, pub ribbon_tab: WorkflowTab, @@ -444,6 +448,7 @@ impl Default for UiState { export_options: None, data_export: None, table_import_preview: None, + nmr_import: None, settings_dialog: None, command_palette: None, ribbon_tab: WorkflowTab::default(), @@ -637,6 +642,7 @@ pub struct Session { /// Off-thread runner for the heaviest button-triggered DOSY computations. /// Not serialized; rebuilt fresh whenever a `PlotxApp` is constructed. pub compute: ComputeService, + pub data_imports: super::DataImports, /// Background update checker/downloader. Not serialized. pub updates: crate::update::UpdateService, pub line_fit_job: Option, diff --git a/crates/core/src/state/ui_state_nmr_import.rs b/crates/core/src/state/ui_state_nmr_import.rs new file mode 100644 index 00000000..85d68080 --- /dev/null +++ b/crates/core/src/state/ui_state_nmr_import.rs @@ -0,0 +1,56 @@ +use plotx_io::nmr_sampling::{IndexBase, SamplingDeclaration}; +use std::path::PathBuf; + +#[derive(Clone, Debug)] +pub struct NmrImportDraft { + pub path: PathBuf, + pub assertion_id: String, + pub source: String, + pub grid: String, + pub lanes: String, + pub one_based: Option, + pub rows: String, + pub error: Option, +} + +impl NmrImportDraft { + pub fn new(path: PathBuf) -> Self { + Self { + path, + assertion_id: format!("plotx-user-schedule-{}", uuid::Uuid::new_v4()), + source: String::new(), + grid: String::new(), + lanes: String::new(), + one_based: None, + rows: String::new(), + error: None, + } + } + + pub fn declaration(&self) -> Result { + let number = |text: &str, name: &str| { + text.trim() + .parse::() + .map_err(|_| format!("Enter an integer for {name}.")) + }; + let index_base = match self.one_based { + Some(true) => IndexBase::One, + Some(false) => IndexBase::Zero, + None => return Err("Select the sampling table's index base.".into()), + }; + let coordinates = self + .rows + .lines() + .filter(|row| !row.trim().is_empty()) + .map(|row| number(row, "each observation (one per line)").map(|value| vec![value])) + .collect::, _>>()?; + Ok(SamplingDeclaration { + assertion_id: self.assertion_id.clone(), + source: self.source.clone(), + grid_shape: vec![number(&self.grid, "the original indirect grid")?], + component_counts: vec![number(&self.lanes, "lanes per observation")?], + coordinates, + index_base, + }) + } +} diff --git a/crates/core/src/workflow.rs b/crates/core/src/workflow.rs index 5a4b4d20..528545d3 100644 --- a/crates/core/src/workflow.rs +++ b/crates/core/src/workflow.rs @@ -9,7 +9,7 @@ use crate::state::{ PlotObject, PlotxApp, StackMode, StackSpec, default_chart_type, }; use plotx_figure::{Axis, Figure}; -use plotx_io::{Acquisition, DataFormat, Domain, LoadWarning, LoadWarningCode, Provenance}; +use plotx_io::{Acquisition, DataFormat, LoadWarning, LoadWarningCode, Provenance}; use serde::Serialize; use std::path::{Path, PathBuf}; use std::time::Duration; @@ -21,10 +21,13 @@ pub use dataset::{ }; #[path = "workflow/mass_spec_layout.rs"] mod mass_spec_layout; +#[path = "workflow/nmr.rs"] +mod nmr_inspection; #[path = "workflow/trace_collection.rs"] mod trace_collection; #[path = "workflow/xps.rs"] mod xps; +pub use nmr_inspection::{inspect_file, inspect_nmr_dataset}; pub const INSPECTION_SCHEMA: &str = "plotx.inspect.v1"; #[derive(Clone, Debug, Serialize)] pub struct InspectionReport { @@ -151,6 +154,8 @@ pub struct ProcessResult { #[derive(Debug, thiserror::Error)] pub enum WorkflowError { + #[error("NMR dataset: {0}")] + Nmr(String), #[error("input load failed: {0}")] Load(#[from] plotx_io::IoError), #[error("processing scheme failed: {0}")] @@ -169,15 +174,36 @@ pub enum WorkflowError { pub fn load_dataset(path: &Path) -> Result { let loaded = plotx_io::load_path(path)?; - let inspection = inspection_report( + dataset_from_load_result(loaded) +} + +pub fn load_dataset_with_sampling( + path: &Path, + declaration: plotx_io::nmr_sampling::SamplingDeclaration, +) -> Result { + dataset_from_load_result(plotx_io::nmr_sampling::load(path, declaration)?) +} + +fn dataset_from_load_result(loaded: plotx_io::LoadResult) -> Result { + let mut inspection = inspection_report( loaded.format, &loaded.provenance, &loaded.warnings, &loaded.acquisition, ); - let (acquisition, acquisition_identity, _, _, nmr_origin, _) = loaded.into_parts(); + let (acquisition, acquisition_identity, _, _, _) = loaded.into_parts(); let (dataset, source) = - dataset_from_loaded_acquisition(acquisition, acquisition_identity, nmr_origin, true); + dataset_from_loaded_acquisition(acquisition, acquisition_identity, true)?; + if let Some(warning) = dataset + .as_nmr2d() + .and_then(|data| data.reconstruction_warning.as_ref()) + { + inspection.warnings.push(WarningReport { + code: "nmr-reconstruction-failed", + message: warning.clone(), + path: None, + }); + } Ok(LoadedDataset { dataset, inspection, @@ -191,7 +217,15 @@ pub fn process_file( output: &Path, format: ExportFormat, ) -> Result { - let mut loaded = load_dataset(input)?; + process_loaded_dataset(load_dataset(input)?, scheme, output, format) +} + +pub fn process_loaded_dataset( + mut loaded: LoadedDataset, + scheme: &Path, + output: &Path, + format: ExportFormat, +) -> Result { loaded.apply_scheme_file(scheme)?; let mut app = PlotxApp::new_with_settings(crate::settings::Settings::default()); app.session @@ -254,10 +288,12 @@ pub fn build_dataset_figure(dataset: &Dataset, chart: &ChartSpec, size_mm: [f32; fn default_binding(dataset: &Dataset) -> DataBinding { let fields = match dataset { - Dataset::Nmr2D(data) if !data.is_true_2d() => ["nmr.stack", "nmr.dosy_map", "nmr.ilt_map"] - .into_iter() - .filter_map(|key| data.field_catalog.id_for_key(key)) - .collect::>(), + Dataset::Nmr2D(data) if !data.is_true_2d() => { + [data.stack_field_key(), "nmr.dosy_map", "nmr.ilt_map"] + .into_iter() + .filter_map(|key| data.field_catalog.id_for_key(key)) + .collect::>() + } Dataset::Electrophysiology(_) => dataset .field_descriptors() .into_iter() @@ -488,9 +524,45 @@ fn inspection_report( warnings: &[LoadWarning], acquisition: &Acquisition, ) -> InspectionReport { - let (count, shape, domain) = match acquisition { - Acquisition::D1(data) => (1, vec![data.len()], data.domain), - Acquisition::D2(data) => (2, vec![data.rows, data.cols], data.domain), + if let Acquisition::Nmr(source) = acquisition { + let axes = source.axes(); + let domain = if axes + .iter() + .all(|axis| axis.domain == nmr::axis::AxisDomain::Time) + { + "time" + } else if axes + .iter() + .all(|axis| axis.domain == nmr::axis::AxisDomain::Frequency) + { + "frequency" + } else { + "mixed" + }; + return InspectionReport { + schema: INSPECTION_SCHEMA, + format: format.as_str().to_owned(), + provenance: ProvenanceReport { + selected_path: provenance.selected_path.clone(), + data_path: provenance.data_path.clone(), + parameter_paths: provenance.parameter_paths.clone(), + companion_paths: provenance.companion_paths.clone(), + }, + dimension: DimensionReport { + count: axes.len(), + shape: axes.iter().map(|axis| axis.points).collect(), + }, + domain: domain.into(), + warnings: warnings.iter().map(warning_report).collect(), + electrophysiology: None, + afm: None, + mass_spectrometry: None, + xrd: None, + xps: None, + }; + } + match acquisition { + Acquisition::Nmr(_) => unreachable!("native NMR was handled above"), Acquisition::Electrophysiology(data) => { let max_points = data .sweeps @@ -499,7 +571,7 @@ fn inspection_report( .map(Vec::len) .max() .unwrap_or(0); - return InspectionReport { + InspectionReport { schema: INSPECTION_SCHEMA, format: format.as_str().to_owned(), provenance: ProvenanceReport { @@ -534,7 +606,7 @@ fn inspection_report( mass_spectrometry: None, xrd: None, xps: None, - }; + } } Acquisition::Afm(data) => { let force = data.forces.as_ref(); @@ -546,7 +618,7 @@ fn inspection_report( }, |force| vec![force.grid_height, force.grid_width, force.samples_per_curve], ); - return InspectionReport { + InspectionReport { schema: INSPECTION_SCHEMA, format: format.as_str().to_owned(), provenance: ProvenanceReport { @@ -573,7 +645,7 @@ fn inspection_report( mass_spectrometry: None, xrd: None, xps: None, - }; + } } Acquisition::MassSpec(run) => { let ms_scan_count = run @@ -582,7 +654,7 @@ fn inspection_report( .filter(|stream| stream.role == plotx_io::StreamRole::Primary) .map(|stream| stream.spectra.len()) .sum(); - return InspectionReport { + InspectionReport { schema: INSPECTION_SCHEMA, format: format.as_str().to_owned(), provenance: ProvenanceReport { @@ -611,68 +683,47 @@ fn inspection_report( }), xrd: None, xps: None, - }; - } - Acquisition::Xrd(data) => { - return InspectionReport { - schema: INSPECTION_SCHEMA, - format: format.as_str().to_owned(), - provenance: ProvenanceReport { - selected_path: provenance.selected_path.clone(), - data_path: provenance.data_path.clone(), - parameter_paths: provenance.parameter_paths.clone(), - companion_paths: provenance.companion_paths.clone(), - }, - dimension: DimensionReport { - count: 1, - shape: vec![data.len()], - }, - domain: "xrd".to_owned(), - warnings: warnings.iter().map(warning_report).collect(), - electrophysiology: None, - afm: None, - mass_spectrometry: None, - xrd: Some(XrdReport { - instrument: data.instrument.clone(), - target: data.target.clone(), - wavelength_angstrom: data.wavelength_angstrom, - two_theta_range_deg: [ - data.two_theta_deg.first().copied().unwrap_or(0.0), - data.two_theta_deg.last().copied().unwrap_or(0.0), - ], - point_count: data.len(), - }), - xps: None, - }; + } } + Acquisition::Xrd(data) => InspectionReport { + schema: INSPECTION_SCHEMA, + format: format.as_str().to_owned(), + provenance: ProvenanceReport { + selected_path: provenance.selected_path.clone(), + data_path: provenance.data_path.clone(), + parameter_paths: provenance.parameter_paths.clone(), + companion_paths: provenance.companion_paths.clone(), + }, + dimension: DimensionReport { + count: 1, + shape: vec![data.len()], + }, + domain: "xrd".to_owned(), + warnings: warnings.iter().map(warning_report).collect(), + electrophysiology: None, + afm: None, + mass_spectrometry: None, + xrd: Some(XrdReport { + instrument: data.instrument.clone(), + target: data.target.clone(), + wavelength_angstrom: data.wavelength_angstrom, + two_theta_range_deg: [ + data.two_theta_deg.first().copied().unwrap_or(0.0), + data.two_theta_deg.last().copied().unwrap_or(0.0), + ], + point_count: data.len(), + }), + xps: None, + }, Acquisition::Xps(experiment) => { - return xps::inspection_report(format, provenance, warnings, experiment); + xps::inspection_report(format, provenance, warnings, experiment) } - }; - InspectionReport { - schema: INSPECTION_SCHEMA, - format: format.as_str().to_owned(), - provenance: ProvenanceReport { - selected_path: provenance.selected_path.clone(), - data_path: provenance.data_path.clone(), - parameter_paths: provenance.parameter_paths.clone(), - companion_paths: provenance.companion_paths.clone(), - }, - dimension: DimensionReport { count, shape }, - domain: domain_label(domain).to_owned(), - warnings: warnings.iter().map(warning_report).collect(), - electrophysiology: None, - afm: None, - mass_spectrometry: None, - xrd: None, - xps: None, } } pub(super) fn warning_report(warning: &LoadWarning) -> WarningReport { let code = match warning.code { LoadWarningCode::ArchiveEntryFailed => "archive-entry-failed", - LoadWarningCode::OptionalImaginaryMissing => "optional-imaginary-missing", LoadWarningCode::MissingStimulus => "missing-stimulus", LoadWarningCode::InvalidMetadata => "invalid-metadata", LoadWarningCode::MissingCalibration => "missing-calibration", @@ -688,13 +739,6 @@ pub(super) fn warning_report(warning: &LoadWarning) -> WarningReport { } } -fn domain_label(domain: Domain) -> &'static str { - match domain { - Domain::Time => "time", - Domain::Frequency => "frequency", - } -} - fn short_name(source: &str) -> String { Path::new(source) .file_name() diff --git a/crates/core/src/workflow/dataset.rs b/crates/core/src/workflow/dataset.rs index 745e744c..657ad27f 100644 --- a/crates/core/src/workflow/dataset.rs +++ b/crates/core/src/workflow/dataset.rs @@ -4,57 +4,52 @@ use crate::state::{Nmr2DDataset, NmrDataset}; pub fn dataset_from_loaded_acquisition( acquisition: Acquisition, acquisition_identity: plotx_io::AcquisitionIdentity, - nmr_origin: Option, equal_scale_homonuclear_2d_imports: bool, -) -> (Dataset, String) { - let (mut dataset, source) = dataset_from_acquisition_with_origin( - acquisition, - nmr_origin, - equal_scale_homonuclear_2d_imports, - ); +) -> Result<(Dataset, String), WorkflowError> { + let (mut dataset, source) = + convert_acquisition(acquisition, equal_scale_homonuclear_2d_imports)?; dataset.set_acquisition_identity(acquisition_identity); - (dataset, source) + Ok((dataset, source)) } -pub fn dataset_from_acquisition(acquisition: Acquisition) -> (Dataset, String) { +pub fn dataset_from_acquisition( + acquisition: Acquisition, +) -> Result<(Dataset, String), WorkflowError> { dataset_from_acquisition_with_equal_scale_preference(acquisition, true) } pub fn dataset_from_acquisition_with_equal_scale_preference( acquisition: Acquisition, equal_scale_homonuclear_2d_imports: bool, -) -> (Dataset, String) { - dataset_from_acquisition_with_origin(acquisition, None, equal_scale_homonuclear_2d_imports) +) -> Result<(Dataset, String), WorkflowError> { + convert_acquisition(acquisition, equal_scale_homonuclear_2d_imports) } -fn dataset_from_acquisition_with_origin( +fn convert_acquisition( acquisition: Acquisition, - nmr_origin: Option, equal_scale_homonuclear_2d_imports: bool, -) -> (Dataset, String) { - match acquisition { - Acquisition::D1(data) => { - let source = data.source.clone(); - ( - Dataset::Nmr(Box::new(NmrDataset::load_with_origin( - data, - nmr_origin.unwrap_or(plotx_io::NmrOrigin::Derived), - ))), - source, - ) - } - Acquisition::D2(data) => { - let source = data.source.clone(); - ( - Dataset::Nmr2D(Box::new( - Nmr2DDataset::load_with_origin_and_equal_scale_preference( - *data, - nmr_origin.unwrap_or(plotx_io::NmrOrigin::Derived), +) -> Result<(Dataset, String), WorkflowError> { + Ok(match acquisition { + Acquisition::Nmr(data) => { + let source = data.source().to_owned(); + let dataset = match data.axes().len() { + 1 => Dataset::Nmr(Box::new( + NmrDataset::load(data).map_err(WorkflowError::Nmr)?, + )), + 2 => Dataset::Nmr2D(Box::new( + Nmr2DDataset::load_with_equal_scale_preference( + data, equal_scale_homonuclear_2d_imports, - ), + ) + .map_err(WorkflowError::Nmr)?, )), - source, - ) + rank => { + return Err(WorkflowError::Nmr(format!( + "PlotX does not yet display rank-{rank} NMR data" + ))); + } + }; + (dataset, source) } Acquisition::Electrophysiology(data) => { let source = data.source.clone(); @@ -93,7 +88,7 @@ fn dataset_from_acquisition_with_origin( source, ) } - } + }) } pub fn dataset_title(dataset: &Dataset) -> String { @@ -101,7 +96,7 @@ pub fn dataset_title(dataset: &Dataset) -> String { Dataset::Nmr(nmr) => nmr .name .clone() - .unwrap_or_else(|| short_name(&nmr.data.source)), + .unwrap_or_else(|| short_name(nmr.data.source())), Dataset::Nmr2D(nmr) => nmr .name .clone() diff --git a/crates/core/src/workflow/nmr.rs b/crates/core/src/workflow/nmr.rs new file mode 100644 index 00000000..fdba111c --- /dev/null +++ b/crates/core/src/workflow/nmr.rs @@ -0,0 +1,79 @@ +use super::*; +use nmr::{axis::AxisDomain, dataset::DescriptorRef}; + +/// Inspect source data without constructing a processing recipe or running FFT. +/// The NMR branch uses the checked library model, including its logical NUS shape. +pub fn inspect_file(path: &Path) -> Result { + match plotx_io::nmr_bridge::read_options().detect(path) { + Ok(_) => {} + Err(error) if error.kind() == nmr::ReadErrorKind::Unrecognized => { + // A recognized but unsupported NMR input must never fall back to an + // older vendor reader. Detection of other scientific families stays here. + if matches!(plotx_io::detect_format(path)?, DataFormat::Nmr(_)) { + return Err(plotx_io::IoError::Nmr(Box::new(error)).into()); + } + let loaded = plotx_io::load_path(path)?; + return Ok(inspection_report( + loaded.format, + &loaded.provenance, + &loaded.warnings, + &loaded.acquisition, + )); + } + Err(error) => return Err(plotx_io::IoError::Nmr(Box::new(error)).into()), + } + let dataset = plotx_io::nmr_bridge::read(path, &mut nmr::ExecutionContext::default())?; + inspect_nmr_dataset(&dataset) +} + +pub fn inspect_nmr_dataset(dataset: &nmr::Dataset) -> Result { + let domains: Vec<_> = match dataset.descriptor() { + DescriptorRef::Raw(descriptor) => { + descriptor.axes().iter().map(|axis| axis.domain()).collect() + } + DescriptorRef::Processed(descriptor) => { + descriptor.axes().iter().map(|axis| axis.domain()).collect() + } + _ => { + return Err( + plotx_io::IoError::NmrConversion("unsupported NMR descriptor".into()).into(), + ); + } + }; + let domain = if domains.iter().all(|domain| *domain == AxisDomain::Time) { + "time" + } else if domains + .iter() + .all(|domain| *domain == AxisDomain::Frequency) + { + "frequency" + } else { + "mixed" + }; + let provenance = plotx_io::nmr_bridge::provenance(dataset)?; + let shape = plotx_io::nmr_bridge::shape(dataset)?; + Ok(InspectionReport { + schema: INSPECTION_SCHEMA, + format: plotx_io::nmr_bridge::format(dataset)?.as_str().to_owned(), + provenance: ProvenanceReport { + selected_path: provenance.selected_path, + data_path: provenance.data_path, + parameter_paths: provenance.parameter_paths, + companion_paths: provenance.companion_paths, + }, + dimension: DimensionReport { + count: shape.len(), + shape, + }, + domain: domain.to_owned(), + warnings: plotx_io::nmr_bridge::warnings(dataset) + .iter() + .map(warning_report) + .collect(), + electrophysiology: None, + afm: None, + mass_spectrometry: None, + xrd: None, + xps: None, + }) +} diff --git a/crates/core/src/workflow_tests.rs b/crates/core/src/workflow_tests.rs index c0cf6a07..172d5edf 100644 --- a/crates/core/src/workflow_tests.rs +++ b/crates/core/src/workflow_tests.rs @@ -1,17 +1,24 @@ use super::*; use num_complex::Complex64; +use plotx_io::Domain; fn acquisition() -> Acquisition { - Acquisition::D1(plotx_io::NmrData { - points: vec![Complex64::new(1.0, 0.0); 8], - domain: Domain::Frequency, - spectral_width_hz: 4_000.0, - observe_freq_mhz: 400.0, - carrier_ppm: 4.7, - nucleus: "1H".to_owned(), - source: "sample.dx".to_owned(), - group_delay: 0.0, - }) + Acquisition::Nmr( + plotx_io::NmrData { + points: (0..8) + .map(|i| Complex64::new(1.0 / (1.0 + (i as f64 - 3.0).powi(2)), 0.0)) + .collect(), + domain: Domain::Frequency, + spectral_width_hz: 4_000.0, + observe_freq_mhz: 400.0, + carrier_ppm: 4.7, + nucleus: "1H".to_owned(), + source: "sample.dx".to_owned(), + group_delay: 0.0, + } + .try_into() + .unwrap(), + ) } fn homonuclear_2d_acquisition() -> Acquisition { @@ -22,8 +29,10 @@ fn homonuclear_2d_acquisition() -> Acquisition { nucleus: "1H".to_owned(), group_delay: 0.0, }; - Acquisition::D2(Box::new(plotx_io::NmrData2D { - data: vec![Complex64::new(1.0, 0.0); 16], + let source = plotx_io::nmr_series::NmrSeriesSource::try_from(plotx_io::NmrData2D { + data: (0..16) + .map(|i| Complex64::new(1.0 / (1.0 + (i as f64 - 5.0).powi(2)), 0.0)) + .collect(), rows: 4, cols: 4, domain: Domain::Frequency, @@ -36,12 +45,14 @@ fn homonuclear_2d_acquisition() -> Acquisition { diffusion: None, nus: None, source: "cosy".to_owned(), - })) + }) + .unwrap(); + Acquisition::Nmr(source.source_dataset().clone()) } #[test] fn canonical_conversion_and_default_canvas_share_dataset_identity() { - let (dataset, source) = dataset_from_acquisition(acquisition()); + let (dataset, source) = dataset_from_acquisition(acquisition()).unwrap(); assert_eq!(dataset.kind_label(), "NMR 1D"); let canvas = build_default_canvas(&dataset, &source); assert_eq!(canvas.dataset_ids(), vec![dataset.resource_id()]); @@ -68,7 +79,8 @@ fn import_preference_seeds_one_persistent_plot_override() { let (dataset, source) = dataset_from_acquisition_with_equal_scale_preference( homonuclear_2d_acquisition(), preference, - ); + ) + .unwrap(); let canvas = build_default_canvas(&dataset, &source); let plot = canvas.objects[0].plot().expect("default plot"); assert_eq!(plot.axis_overrides.lock_aspect, Some(expected)); diff --git a/crates/core/tests/nmr_auto_nus.rs b/crates/core/tests/nmr_auto_nus.rs new file mode 100644 index 00000000..9001248b --- /dev/null +++ b/crates/core/tests/nmr_auto_nus.rs @@ -0,0 +1,247 @@ +//! Automatic import uses the library estimator and retains its execution evidence. +use nmr::axis::{AxisCoordinates, AxisDomain, AxisUnit}; +use nmr::raw::*; +use nmr::{Complex64, Dataset as NativeDataset}; +use plotx_core::state::{Dataset, Nmr2DDataset, PlotxApp}; +use plotx_io::nmr_view::NmrSource; +use std::sync::Arc; + +fn synthetic() -> NmrSource { + let (grid, observations, points) = (256, 96, 128); + let axis = |kind, points| { + RawAxis::new( + kind, + AxisDomain::Time, + Some(AxisUnit::Second), + points, + AxisCoordinates::Uniform { + start: 0.0, + step: 0.001, + }, + ) + .unwrap() + .with_group_delay(nmr::acquisition::GroupDelayState::NotApplicable) + .unwrap() + }; + let axes = vec![ + axis( + RawAxisKind::Indirect(IndirectComponents::Cartesian( + ComponentEvidence::user_constructed(), + )), + grid, + ), + axis(RawAxisKind::Direct(DirectSamples::Complex), points), + ]; + let coordinates: Vec<_> = (0..observations) + .map(|i| SamplingCoordinate::new(vec![i * 73 % grid])) + .collect(); + let mut rng = 7193_u64; + let mut uniform = || { + rng = rng.wrapping_mul(6364136223846793005).wrapping_add(1); + ((rng >> 11) as f64 + 0.5) / (1u64 << 53) as f64 + }; + let traces = coordinates + .iter() + .enumerate() + .map(|(ordinal, coordinate)| { + let mut samples = Vec::new(); + let angle = + std::f64::consts::TAU * 17.0 * coordinate.as_slice()[0] as f64 / grid as f64; + for lane in [angle.cos(), angle.sin()] { + for j in 0..points { + let signal = Complex64::from_polar( + lane * (-6.0 * j as f64 / points as f64).exp(), + std::f64::consts::TAU * 21.0 * j as f64 / points as f64, + ); + let radius = 0.1 * (-2.0 * uniform().ln()).sqrt(); + samples.push( + signal + Complex64::from_polar(radius, std::f64::consts::TAU * uniform()), + ); + } + } + SparseTrace::new( + ObservationOrdinal::new(ordinal), + coordinate.clone(), + samples, + ) + }) + .collect(); + let raw = RawDatasetBuilder::new(axes, RawMetadata::default()) + .unwrap() + .sparse( + traces, + SamplingSchedule::new(vec![grid], coordinates).unwrap(), + ) + .unwrap(); + NmrSource::new(Arc::new(raw.into())).unwrap() +} + +fn assert_frequency(dataset: &Nmr2DDataset) { + assert!(dataset.is_true_2d()); + assert!(dataset.reconstruction_warning.is_none()); + assert!( + dataset.nus_request.is_none(), + "automatic sigma is result evidence, not an override" + ); + let processed = dataset.native_processed.dataset().as_processed().unwrap(); + assert!( + processed + .descriptor() + .axes() + .iter() + .all(|axis| axis.domain() == AxisDomain::Frequency) + ); + assert!( + dataset + .native_processed + .dataset() + .as_dense_processed() + .unwrap() + .samples() + .iter() + .all(|x| x.is_finite()) + ); +} + +fn assert_auto_evidence(dataset: &NativeDataset) { + let mut bytes = Vec::new(); + nmr::execution_report::write_json( + dataset.as_processed().unwrap(), + &[], + &mut bytes, + 16 * 1024 * 1024, + ) + .unwrap(); + let json = String::from_utf8(bytes).unwrap(); + assert!( + [ + "split-observation-cartesian-rms.v1", + "split-holdout-component-rms.v1", + "jeol-interior-split-rms.v1", + "jeol-interior-holdout-rms.v1", + ] + .iter() + .any(|method| json.contains(method)), + "{json}" + ); +} + +#[test] +fn automatic_import_and_offline_project_reopen_produce_2d() { + let dataset = Nmr2DDataset::load(synthetic()).unwrap(); + assert_frequency(&dataset); + assert_auto_evidence(dataset.native_processed.dataset()); + let mut app = PlotxApp::new(); + app.doc.datasets.push(Dataset::Nmr2D(Box::new(dataset))); + assert!(app.schedule_2d_processing(0, true)); + let started = std::time::Instant::now(); + while app.session.compute.is_busy() { + assert!(started.elapsed() < std::time::Duration::from_secs(30)); + std::thread::sleep(std::time::Duration::from_millis(5)); + app.poll_compute(); + } + app.poll_compute(); + assert_eq!(app.session.status, "Updated 2D processing."); + assert_frequency(app.doc.datasets[0].as_nmr2d().unwrap()); + let path = std::env::temp_dir().join(format!("plotx-auto-nus-{}.plotx", uuid::Uuid::new_v4())); + plotx_core::project::save_project(&app, &path, false).unwrap(); + let reopened = plotx_core::project::load_project(&path).unwrap(); + std::fs::remove_file(path).unwrap(); + let dataset = reopened.doc.datasets[0].as_nmr2d().unwrap(); + assert_frequency(dataset); + assert_auto_evidence(dataset.native_processed.dataset()); +} + +#[test] +fn failed_auto_estimation_keeps_observations_and_reports_the_reason() { + let path = std::path::Path::new(env!("CARGO_MANIFEST_DIR")) + .join("../io/tests/fixtures/nmr/bruker-nus"); + let mut app = PlotxApp::new(); + app.load_from(&path); + assert_eq!(app.doc.datasets.len(), 1); + let data = app.doc.datasets[0].as_nmr2d().unwrap(); + assert!(!data.is_true_2d()); + assert!(data.native_processed.dataset().as_raw().is_some()); + let warning = data.reconstruction_warning.as_ref().unwrap(); + assert!( + warning.contains("automatic NUS noise requires"), + "{warning}" + ); + assert!(app.session.status.contains(warning)); + let loaded = plotx_core::workflow::load_dataset(&path).unwrap(); + assert!( + loaded + .inspection + .warnings + .iter() + .any(|warning| warning.code == "nmr-reconstruction-failed" + && warning.message.contains("automatic NUS noise requires")) + ); +} + +#[test] +fn automatic_noise_analysis_honors_cancellation_and_work_limits() { + use plotx_processing::nmr_bridge::{DelayPolicy, RecipeRange}; + use plotx_processing::nmr_execution::{execute_2d, processing_2d_work_ledger}; + let input = synthetic(); + let params = plotx_processing::Params2D::default_for(plotx_processing::Preset2D::Generic); + let token = nmr::CancellationToken::new(); + let cancel = token.clone(); + let mut saw_noise = false; + let mut progress = |event: nmr::execution::ProgressEvent| { + if event.stage == nmr::execution::ExecutionStage::NoiseEstimation { + saw_noise = true; + cancel.cancel(); + } + }; + let mut work = processing_2d_work_ledger(); + let mut context = nmr::ExecutionContext::new(&mut work) + .with_cancellation(token) + .with_progress(&mut progress); + let error = execute_2d( + &input, + ¶ms, + DelayPolicy::AxisEvidence, + RecipeRange::Base, + None, + &mut context, + ) + .unwrap_err(); + assert!(error.is_cancelled(), "{error}"); + assert!(saw_noise); + + let mut work = nmr::resource::WorkLedger::new(1); + let error = execute_2d( + &input, + ¶ms, + DelayPolicy::AxisEvidence, + RecipeRange::Base, + None, + &mut nmr::ExecutionContext::new(&mut work), + ) + .unwrap_err(); + assert!(!error.is_cancelled()); + assert!(error.to_string().contains("work"), "{error}"); +} + +#[test] +#[ignore = "requires a local NMR acquisition path in PLOTX_NUS_SAMPLE"] +fn local_nus_import_uses_the_default_plotx_recipe() { + let path = std::env::var_os("PLOTX_NUS_SAMPLE").expect("set PLOTX_NUS_SAMPLE"); + let mut app = PlotxApp::new(); + app.load_from(std::path::Path::new(&path)); + assert_eq!(app.doc.datasets.len(), 1, "{}", app.session.status); + let dataset = app.doc.datasets[0].as_nmr2d().unwrap(); + assert_frequency(dataset); + assert_auto_evidence(dataset.native_processed.dataset()); + println!( + "shape={:?}", + dataset + .native_processed + .dataset() + .as_processed() + .unwrap() + .descriptor() + .logical_shape() + ); +} diff --git a/crates/core/tests/slice.rs b/crates/core/tests/slice.rs index e52d61f7..b4ad61f1 100644 --- a/crates/core/tests/slice.rs +++ b/crates/core/tests/slice.rs @@ -4,7 +4,7 @@ use num_complex::Complex64; use plotx_analysis::peaks::{DetectParams, detect_peaks, estimate_noise}; use plotx_core::build_figure; use plotx_io::{Domain, NmrData}; -use plotx_processing::{AxisPipeline, process}; +use plotx_processing::AxisPipeline; use std::f64::consts::TAU; /// An ethanol-like ¹H FID (three singlets at 3:2:1), so the test needs no file. @@ -44,7 +44,16 @@ fn full_slice_load_process_figure_export() { let data = ethanol_fid(); assert_eq!(data.len(), 16_384); - let processed = process(&data, &AxisPipeline::default_1d(), true).unwrap(); + let source = plotx_io::nmr_view::NmrSource::try_from(data.clone()).unwrap(); + let processed = plotx_processing::nmr_execution::execute_1d( + &source, + &AxisPipeline::default_1d(), + plotx_processing::nmr_bridge::DelayPolicy::AxisEvidence, + plotx_processing::nmr_bridge::RecipeRange::All, + &mut nmr::ExecutionContext::default(), + ) + .unwrap() + .view; let spec = processed.as_frequency().unwrap(); assert_eq!(spec.len(), data.len()); @@ -65,7 +74,7 @@ fn full_slice_load_process_figure_export() { assert!(has(2.61), "missing OH peak; got {peaks:?}"); assert!(has(3.70), "missing CH2 peak; got {peaks:?}"); - let fig = build_figure(&data, spec, &[]); + let fig = build_figure(&data.try_into().unwrap(), spec, &[]); let svg = plotx_render::svg::export(&fig); assert!(svg.starts_with(" Dim { } } -/// A phase-modulated 2D FID with a single cross peak at `(f2_ppm, f1_ppm)`. +/// A Cartesian (States) 2D FID with a single cross peak at `(f2_ppm, f1_ppm)`. fn synthetic_hsqc(f2_ppm: f64, f1_ppm: f64, experiment: &str) -> NmrData2D { let (cols, rows) = (256usize, 128usize); let direct = dim(4000.0, 400.0, "1H"); @@ -41,26 +41,31 @@ fn synthetic_hsqc(f2_ppm: f64, f1_ppm: f64, experiment: &str) -> NmrData2D { let dt1 = 1.0 / indirect.spectral_width_hz; let f2_hz = f2_ppm * direct.observe_freq_mhz; let f1_hz = f1_ppm * indirect.observe_freq_mhz; - let mut data = Vec::with_capacity(rows * cols); + let mut data = Vec::with_capacity(2 * rows * cols); for k in 0..rows { let t1 = k as f64 * dt1; - for j in 0..cols { - let t2 = j as f64 * dt2; - let decay = (-t2 / 0.3 - t1 / 0.3).exp(); - data.push(Complex64::from_polar( - decay, - TAU * (f2_hz * t2 + f1_hz * t1), - )); + for component in 0..2 { + let indirect = if component == 0 { + (TAU * f1_hz * t1).cos() + } else { + (TAU * f1_hz * t1).sin() + }; + for j in 0..cols { + let t2 = j as f64 * dt2; + let decay = (-t2 / 0.3 - t1 / 0.3).exp(); + data.push(Complex64::from_polar(decay, TAU * f2_hz * t2) * indirect); + } } } + NmrData2D { data, - rows, + rows: 2 * rows, cols, domain: Domain::Time, direct, indirect, - quad: QuadMode::Complex, + quad: QuadMode::States, indirect_conjugate: false, experiment: Some(experiment.to_owned()), pseudo_axis: None, @@ -74,7 +79,7 @@ fn synthetic_hsqc(f2_ppm: f64, f1_ppm: f64, experiment: &str) -> NmrData2D { fn contour_slice_places_peak_and_exports_svg() { // Shifts stay inside the ±SW/2 Nyquist range (F1: 10 ppm × 100 MHz = 1 kHz). let data = synthetic_hsqc(3.0, 10.0, "hsqcetgpsisp"); - let preset = recommend_preset(&data); + let preset = recommend_preset(&data.clone().try_into().unwrap()); assert_eq!(preset, Preset2D::Hsqc); assert_eq!(preset.layout(), Layout2D::Ft); @@ -108,7 +113,7 @@ fn contour_slice_places_peak_and_exports_svg() { let mut app = PlotxApp::new(); app.doc .datasets - .push(Dataset::Nmr2D(Box::new(Nmr2DDataset::load(data)))); + .push(Dataset::Nmr2D(Box::new(Nmr2DDataset::load(data).unwrap()))); let mut canvas = CanvasDocument::new("contour".to_owned(), [120.0, 80.0]); let [width, height] = canvas.size_pt(); let object = app.build_plot_object( @@ -167,11 +172,9 @@ fn settle(app: &mut PlotxApp) { /// a genuine noise estimate rather than a hand-written grid. fn contour_page() -> (PlotxApp, ObjectId) { let mut app = PlotxApp::new(); - app.doc - .datasets - .push(Dataset::Nmr2D(Box::new(Nmr2DDataset::load( - synthetic_hsqc(3.0, 10.0, "hsqcetgpsisp"), - )))); + app.doc.datasets.push(Dataset::Nmr2D(Box::new( + Nmr2DDataset::load(synthetic_hsqc(3.0, 10.0, "hsqcetgpsisp")).unwrap(), + ))); let mut canvas = CanvasDocument::new("contour".to_owned(), [120.0, 80.0]); let [width, height] = canvas.size_pt(); let id = canvas.allocate_object_id(); @@ -639,7 +642,10 @@ fn stack_slice_exports_waterfall() { let mut data = synthetic_hsqc(3.0, 40.0, "ledbpgp2s"); // A DOSY-style hint should recommend the stacked (pseudo-2D) layout. data.experiment = Some("ledbpgp2s".into()); - assert_eq!(recommend_preset(&data).layout(), Layout2D::Stack); + assert_eq!( + recommend_preset(&data.clone().try_into().unwrap()).layout(), + Layout2D::Stack + ); let stack = match process_2d( &data, @@ -651,10 +657,25 @@ fn stack_slice_exports_waterfall() { Processed2D::Stack(s) => s, Processed2D::Ft(_) => panic!("expected Stack"), }; - assert_eq!(stack.increments(), data.rows); + // States stores two component rows for each logical increment. + assert_eq!(stack.increments(), data.rows / 2); let fig = build_stack_figure(&stack); assert!(!fig.series.is_empty()); let svg = plotx_render::svg::export(&fig); assert!(svg.contains(" Processed2D { + let source = plotx_io::nmr_series::NmrSeriesSource::try_from(data.clone()).unwrap(); + plotx_processing::nmr_execution::execute_2d( + source.source_dataset(), + params, + plotx_processing::nmr_bridge::DelayPolicy::AxisEvidence, + plotx_processing::nmr_bridge::RecipeRange::Base, + None, + &mut nmr::ExecutionContext::default(), + ) + .unwrap() + .view +} diff --git a/crates/io/Cargo.toml b/crates/io/Cargo.toml index bcdb7856..efcad4c9 100644 --- a/crates/io/Cargo.toml +++ b/crates/io/Cargo.toml @@ -11,6 +11,7 @@ name = "plotx_io" path = "src/lib.rs" [dependencies] +nmr.workspace = true num-complex.workspace = true thiserror.workspace = true zip.workspace = true diff --git a/crates/io/src/archive.rs b/crates/io/src/archive.rs index 1a403bd8..099a7fa1 100644 --- a/crates/io/src/archive.rs +++ b/crates/io/src/archive.rs @@ -76,10 +76,7 @@ fn scratch_dir() -> PathBuf { // is loaded as a unit and not descended into; any other directory is recursed; // loose JEOL and JCAMP-DX files are read individually. fn collect_acquisitions(dir: &Path, out: &mut ArchiveLoadResult) { - if crate::bruker::detect_processed(dir).is_some() - || crate::bruker::is_bruker_dir(dir) - || crate::varian::is_varian(dir) - { + if crate::nmr_bridge::is_candidate(dir) { match crate::load_path(dir) { Ok(result) => out.items.push(result), Err(error) => out.warnings.push(entry_warning(dir, error)), @@ -115,14 +112,6 @@ fn entry_warning(path: &Path, error: IoError) -> LoadWarning { } } -fn is_jdf(path: &Path) -> bool { - path.extension() - .and_then(|e| e.to_str()) - .map(|e| e.eq_ignore_ascii_case("jdf")) - .unwrap_or(false) - || crate::jeol::is_jdf(path) -} - fn is_supported_spectrum(path: &Path) -> bool { - is_jdf(path) || crate::jcamp_dx::has_jcamp_extension(path) + crate::nmr_bridge::is_candidate(path) } diff --git a/crates/io/src/bruker.rs b/crates/io/src/bruker.rs deleted file mode 100644 index 8843844e..00000000 --- a/crates/io/src/bruker.rs +++ /dev/null @@ -1,776 +0,0 @@ -//! Bruker TopSpin acquisition reader — a directory of a binary `fid` plus a -//! text `acqus` parameter file. - -use crate::{ - Acquisition, DataFormat, Dim, Domain, IoError, LoadResult, LoadWarning, LoadWarningCode, - NmrData, NmrData2D, NmrFormat, NmrInstrumentOrigin, NmrOrigin, NmrPortableMetadata, - NmrSourceFormat, NmrSourceParameters, Provenance, QuadMode, -}; -use num_complex::Complex64; -use sha2::{Digest, Sha256}; -use std::collections::HashMap; -use std::path::{Path, PathBuf}; - -// Each 1D FID within a `ser` file is padded so its byte length is a multiple of -// this block size (256 four-byte words). -const SER_BLOCK_BYTES: usize = 1024; - -mod processed; - -pub use processed::{detect_processed, load_processed}; - -/// A directory holding a Bruker acquisition: an `acqus` parameter file next to a -/// binary `fid` (1D) or `ser` (nD) data file. -pub fn is_bruker_dir(path: &Path) -> bool { - path.is_dir() - && path.join("acqus").is_file() - && (path.join("fid").is_file() || path.join("ser").is_file()) -} - -/// A Bruker acquisition selected either as its directory or directly as the -/// `fid`/`ser` data file inside it (whose parent holds the `acqus`). -pub fn is_bruker(path: &Path) -> bool { - if path.is_dir() { - return is_bruker_dir(path); - } - matches!( - path.file_name().and_then(|s| s.to_str()), - Some("fid" | "ser") - ) && path - .parent() - .map(|d| d.join("acqus").is_file()) - .unwrap_or(false) -} - -// Resolve a user-selected path to (acquisition dir, binary data file). A -// directory prefers `fid` over `ser`; a file is taken as-is with its parent as -// the acquisition dir. -fn resolve_bruker(path: &Path) -> (PathBuf, PathBuf) { - if path.is_dir() { - let fid = path.join("fid"); - let data = if fid.is_file() { fid } else { path.join("ser") }; - (path.to_path_buf(), data) - } else { - let dir = path - .parent() - .map(Path::to_path_buf) - .unwrap_or_else(|| PathBuf::from(".")); - (dir, path.to_path_buf()) - } -} - -// A readable dataset label for an acquisition dir `/`: the -// processed-data title (or the sample folder as a fallback), always tagged with -// the numeric expno since one sample folder holds many experiments. -fn source_prefix(dir: &Path) -> String { - let expno = dir.file_name().and_then(|s| s.to_str()); - let name = pdata_title(dir).or_else(|| { - dir.parent() - .and_then(Path::file_name) - .and_then(|s| s.to_str()) - .map(str::to_owned) - }); - match (name, expno) { - (Some(name), Some(expno)) => format!("{name} (expno {expno})"), - (Some(name), None) => name, - (None, Some(expno)) => expno.to_owned(), - (None, None) => "".to_owned(), - } -} - -fn acquisition_identity(dir: &Path, params: Option<&JcampParams>) -> crate::AcquisitionIdentity { - let source_label = dir - .file_name() - .and_then(|value| value.to_str()) - .unwrap_or("Untitled NMR") - .to_owned(); - let subject = dir - .parent() - .and_then(Path::file_name) - .and_then(|value| value.to_str()) - .map(str::trim) - .filter(|value| !value.is_empty()) - .map(str::to_owned); - let acquisition = params - .and_then(|params| params.string("EXP").or_else(|| params.string("PULPROG"))) - .map(|value| { - value - .trim_matches(|c| c == '<' || c == '>') - .trim() - .to_owned() - }) - .filter(|value| !value.is_empty()); - crate::AcquisitionIdentity { - subject, - acquisition, - source_label, - } -} - -// The first non-empty line of a processed-data `title` file, preferring proc no. -// 1 and otherwise the lowest-numbered proc dir carrying a non-empty title. -fn pdata_title(dir: &Path) -> Option { - let mut procs: Vec = std::fs::read_dir(dir.join("pdata")) - .ok()? - .flatten() - .map(|e| e.path()) - .filter(|p| p.is_dir()) - .collect(); - procs.sort_by_key(|p| { - p.file_name() - .and_then(|s| s.to_str()) - .and_then(|s| s.parse::().ok()) - .unwrap_or(u64::MAX) - }); - procs.iter().find_map(|proc| { - let text = std::fs::read_to_string(proc.join("title")).ok()?; - text.lines() - .map(str::trim) - .find(|l| !l.is_empty()) - .map(str::to_owned) - }) -} - -pub fn read_bruker(path: &Path) -> Result { - let (dir, data_path) = resolve_bruker(path); - let acqus_path = dir.join("acqus"); - let params = JcampParams::parse(&std::fs::read_to_string(&acqus_path)?); - - // A `ser` file alongside an `acqu2s` is a 2D (or nD) acquisition. - let acqu2s_path = dir.join("acqu2s"); - let is_ser = data_path.file_name().and_then(|s| s.to_str()) == Some("ser"); - if is_ser && acqu2s_path.is_file() { - return read_bruker_2d(&dir, &data_path, ¶ms, &acqu2s_path) - .map(|d| Acquisition::D2(Box::new(d))); - } - - read_bruker_1d(&dir, &data_path, ¶ms).map(Acquisition::D1) -} - -pub fn load_raw(path: &Path) -> Result { - let (dir, data_path) = resolve_bruker(path); - let acqus = std::fs::read_to_string(dir.join("acqus"))?; - let params = JcampParams::parse(&acqus); - let mut parameter_paths = vec![dir.join("acqus")]; - if data_path.file_name().and_then(|s| s.to_str()) == Some("ser") && dir.join("acqu2s").is_file() - { - parameter_paths.push(dir.join("acqu2s")); - } - let acquisition = read_bruker(path)?; - let data_bytes = std::fs::read(&data_path)?; - let mut digest = Sha256::new(); - digest.update(b"fid\0"); - digest.update(&data_bytes); - digest.update(b"acqus\0"); - digest.update(acqus.as_bytes()); - let title = pdata_title(&dir); - if let Some(title) = &title { - digest.update(b"title\0"); - digest.update(title.as_bytes()); - } - let pulse_program = params - .string("PULPROG") - .map(|value| value.trim_matches(['<', '>']).to_owned()); - Ok(LoadResult::new( - acquisition, - acquisition_identity(&dir, Some(¶ms)), - DataFormat::Nmr(NmrFormat::BrukerRaw), - Provenance { - selected_path: path.to_path_buf(), - data_path, - parameter_paths, - companion_paths: Vec::new(), - }, - Vec::new(), - ) - .with_nmr_origin(NmrOrigin::Instrument { - instrument: NmrInstrumentOrigin { - format: NmrSourceFormat::BrukerRaw, - source_sha256: digest.finalize().into(), - portable: NmrPortableMetadata { - solvent: params - .string("SOLVENT") - .map(|value| value.trim_matches(['<', '>']).to_owned()), - temperature_k: params.f64("TE").filter(|value| value.is_finite()), - transients: params - .usize("NS") - .and_then(|value| u64::try_from(value).ok()), - pulse_sequence: pulse_program.clone(), - }, - parameters: NmrSourceParameters::Bruker { - acqus, - title, - pulse_program, - }, - }, - })) -} - -fn read_bruker_1d(dir: &Path, fid_path: &Path, params: &JcampParams) -> Result { - // TD counts individual real values, so complex points = TD/2. - let td = params.usize("TD").unwrap_or(0); - if td < 2 { - return Err(IoError::Unsupported(format!( - "acqus reports TD={td} (need at least one complex point)" - ))); - } - let n_complex = td / 2; - - let byte_order = match params.i64("BYTORDA").unwrap_or(0) { - 1 => Endian::Big, - _ => Endian::Little, - }; - let sample = match params.i64("DTYPA").unwrap_or(0) { - 2 => SampleFmt::F64, - _ => SampleFmt::I32, - }; - let stride = sample.size(); - - let bytes = std::fs::read(fid_path)?; - let need = n_complex - .checked_mul(2 * stride) - .ok_or_else(|| IoError::Unsupported("TD overflow".into()))?; - if bytes.len() < need { - return Err(IoError::Truncated { - offset: 0, - needed: need, - have: bytes.len(), - }); - } - - // De-interleave (re, im, re, im, …) into complex points. - let r = Reader { - bytes: &bytes, - endian: byte_order, - }; - let points: Vec = (0..n_complex) - .map(|i| { - let base = i * 2 * stride; - Complex64::new(r.real(base, sample), r.real(base + stride, sample)) - }) - .collect(); - - let spectral_width_hz = params - .f64("SW_h") - .filter(|v| v.is_finite() && *v > 0.0) - .unwrap_or(0.0); - // SFO1 is the observed (Larmor) frequency; BF1 is the 0-ppm reference. - let observe_freq_mhz = params - .f64("SFO1") - .or_else(|| params.f64("BF1")) - .filter(|v| v.is_finite() && *v > 1.0) - .unwrap_or(400.0); - let bf1 = params - .f64("BF1") - .filter(|v| v.is_finite() && *v > 1.0) - .unwrap_or(observe_freq_mhz); - let carrier_ppm = params.f64("O1").map(|o1| o1 / bf1).unwrap_or(0.0); - - let nucleus = params - .string("NUC1") - .map(|s| s.trim_matches(|c| c == '<' || c == '>').to_string()) - .filter(|s| !s.is_empty() && s != "off") - .unwrap_or_else(|| guess_nucleus(observe_freq_mhz)); - - let group_delay = group_delay(params); - - let source = format!( - "{} (Bruker TopSpin, {sample:?}, {n_complex} pts)", - source_prefix(dir) - ); - - Ok(NmrData { - points, - domain: Domain::Time, - spectral_width_hz: if spectral_width_hz > 0.0 { - spectral_width_hz - } else { - observe_freq_mhz * 20.0 - }, - observe_freq_mhz, - carrier_ppm, - nucleus, - source, - group_delay, - }) -} - -fn read_bruker_2d( - dir: &Path, - ser_path: &Path, - f2: &JcampParams, - acqu2s_path: &Path, -) -> Result { - let f1 = JcampParams::parse(&std::fs::read_to_string(acqu2s_path)?); - - let td2 = f2.usize("TD").unwrap_or(0); - let rows = f1.usize("TD").unwrap_or(0); - if td2 < 2 || rows == 0 { - return Err(IoError::Unsupported(format!( - "acqus/acqu2s report TD={td2}, TD1={rows} (need a non-empty 2D)" - ))); - } - let cols = td2 / 2; - - let byte_order = match f2.i64("BYTORDA").unwrap_or(0) { - 1 => Endian::Big, - _ => Endian::Little, - }; - let sample = match f2.i64("DTYPA").unwrap_or(0) { - 2 => SampleFmt::F64, - _ => SampleFmt::I32, - }; - let stride = sample.size(); - - // Each stored row is `td2` reals padded up to a whole number of blocks. - let row_bytes = td2 - .checked_mul(stride) - .map(|b| b.div_ceil(SER_BLOCK_BYTES) * SER_BLOCK_BYTES) - .ok_or_else(|| IoError::Unsupported("TD overflow".into()))?; - let bytes = std::fs::read(ser_path)?; - let need = rows - .checked_mul(row_bytes) - .ok_or_else(|| IoError::Unsupported("ser size overflow".into()))?; - if bytes.len() < need { - return Err(IoError::Truncated { - offset: 0, - needed: need, - have: bytes.len(), - }); - } - - let r = Reader { - bytes: &bytes, - endian: byte_order, - }; - let mut data = Vec::with_capacity(rows * cols); - for row in 0..rows { - let base = row * row_bytes; - for i in 0..cols { - let off = base + i * 2 * stride; - data.push(Complex64::new( - r.real(off, sample), - r.real(off + stride, sample), - )); - } - } - - let quad = match f1.i64("FnMODE").unwrap_or(0) { - 4 => QuadMode::States, - 5 => QuadMode::StatesTppi, - 6 => QuadMode::EchoAntiecho, - _ => QuadMode::Complex, - }; - - let direct = dim_from(f2, group_delay(f2)); - let indirect = dim_from(&f1, 0.0); - - let experiment = acquisition_identity(dir, Some(f2)).acquisition; - - let source = format!( - "{} (Bruker TopSpin 2D, {sample:?}, {cols}×{rows})", - source_prefix(dir) - ); - - Ok(NmrData2D { - data, - rows, - cols, - domain: Domain::Time, - direct, - indirect, - quad, - indirect_conjugate: false, - experiment, - pseudo_axis: None, - diffusion: None, - nus: None, - source, - }) -} - -fn dim_from(p: &JcampParams, group_delay: f64) -> Dim { - let observe_freq_mhz = p - .f64("SFO1") - .or_else(|| p.f64("BF1")) - .filter(|v| v.is_finite() && *v > 1.0) - .unwrap_or(400.0); - let bf1 = p - .f64("BF1") - .filter(|v| v.is_finite() && *v > 1.0) - .unwrap_or(observe_freq_mhz); - let spectral_width_hz = p - .f64("SW_h") - .filter(|v| v.is_finite() && *v > 0.0) - .unwrap_or(observe_freq_mhz * 20.0); - let nucleus = p - .string("NUC1") - .map(|s| s.trim_matches(|c| c == '<' || c == '>').to_string()) - .filter(|s| !s.is_empty() && s != "off") - .unwrap_or_else(|| guess_nucleus(observe_freq_mhz)); - Dim { - spectral_width_hz, - observe_freq_mhz, - carrier_ppm: p.f64("O1").map(|o1| o1 / bf1).unwrap_or(0.0), - nucleus, - group_delay, - } -} - -// Group delay in points: an explicit `GRPDLY` when present, else a lookup from -// the (`DSPFVS`, `DECIM`) table for older data. -fn group_delay(params: &JcampParams) -> f64 { - if let Some(g) = params.f64("GRPDLY") - && g.is_finite() - && g >= 0.0 - { - return g; - } - let dspfvs = params.i64("DSPFVS").unwrap_or(-1); - let decim = params.i64("DECIM").unwrap_or(-1); - grpdly_from_table(dspfvs, decim).unwrap_or(0.0) -} - -fn guess_nucleus(mhz: f64) -> String { - if mhz > 300.0 { - "1H".into() - } else if mhz > 90.0 { - "13C".into() - } else { - "X".into() - } -} - -// Parsed JCAMP-DX `acqus`: scalar `##$KEY= value` entries. Array-valued -// parameters (`##$KEY= (0..N)` then value lines) are skipped. -struct JcampParams { - map: HashMap, -} - -impl JcampParams { - fn parse(text: &str) -> Self { - let mut map = HashMap::new(); - for line in text.lines() { - let Some(rest) = line.strip_prefix("##$") else { - continue; - }; - let Some((key, val)) = rest.split_once('=') else { - continue; - }; - let val = val.trim(); - // Array declarations like "(0..15)" carry their payload on following - // lines, which are not consumed. - if val.starts_with('(') { - continue; - } - map.insert(key.trim().to_string(), val.to_string()); - } - Self { map } - } - - fn string(&self, key: &str) -> Option { - self.map.get(key).cloned() - } - - fn f64(&self, key: &str) -> Option { - self.map.get(key)?.parse().ok() - } - - fn i64(&self, key: &str) -> Option { - self.map.get(key)?.parse().ok() - } - - fn usize(&self, key: &str) -> Option { - self.map.get(key)?.parse().ok() - } -} - -// Standard Bruker group-delay table for older data (`DSPFVS` 10–13), keyed by -// `DECIM`. -#[allow(clippy::excessive_precision)] -fn grpdly_from_table(dspfvs: i64, decim: i64) -> Option { - let row: &[(i64, f64)] = match dspfvs { - 10 => &[ - (2, 44.75), - (3, 33.5), - (4, 66.625), - (6, 59.083333333333333), - (8, 68.5625), - (12, 60.375), - (16, 69.53125), - (24, 61.020833333333333), - (32, 70.015625), - (48, 61.34375), - (64, 70.2578125), - (96, 61.505208333333333), - (128, 70.37890625), - (192, 61.5859375), - (256, 70.439453125), - (384, 61.626302083333333), - (512, 70.4697265625), - (768, 61.646484375), - (1024, 70.48486328125), - (1536, 61.656575520833333), - (2048, 70.4924316406250), - ], - 11 => &[ - (2, 46.0), - (3, 36.5), - (4, 48.0), - (6, 50.166666666666667), - (8, 53.25), - (12, 69.5), - (16, 72.25), - (24, 70.166666666666667), - (32, 72.75), - (48, 70.5), - (64, 73.0), - (96, 70.666666666666667), - (128, 72.5), - (192, 71.333333333333333), - (256, 72.25), - (384, 71.666666666666667), - (512, 72.125), - (768, 71.833333333333333), - (1024, 72.0625), - (1536, 71.916666666666667), - (2048, 72.03125), - ], - 12 => &[ - (2, 46.0), - (3, 36.5), - (4, 48.0), - (6, 50.166666666666667), - (8, 53.25), - (12, 69.5), - (16, 71.625), - (24, 70.166666666666667), - (32, 72.125), - (48, 70.5), - (64, 72.375), - (96, 70.666666666666667), - (128, 72.5), - (192, 71.333333333333333), - (256, 72.25), - (384, 71.666666666666667), - (512, 72.125), - (768, 71.833333333333333), - (1024, 72.0625), - (1536, 71.916666666666667), - (2048, 72.03125), - ], - 13 => &[ - (2, 2.75), - (3, 2.8333333333333333), - (4, 2.875), - (6, 2.9166666666666667), - (8, 2.9375), - (12, 2.9583333333333333), - (16, 2.96875), - (24, 2.9791666666666667), - (32, 2.984375), - (48, 2.9895833333333333), - (64, 2.9921875), - (96, 2.9947916666666667), - ], - _ => return None, - }; - row.iter().find(|(d, _)| *d == decim).map(|(_, g)| *g) -} - -#[derive(Debug, Clone, Copy)] -enum SampleFmt { - I32, - F64, -} - -impl SampleFmt { - #[inline] - fn size(self) -> usize { - match self { - SampleFmt::I32 => 4, - SampleFmt::F64 => 8, - } - } -} - -#[derive(Debug, Clone, Copy)] -enum Endian { - Big, - Little, -} - -struct Reader<'a> { - bytes: &'a [u8], - endian: Endian, -} - -impl Reader<'_> { - fn real(&self, at: usize, fmt: SampleFmt) -> f64 { - match fmt { - SampleFmt::I32 => { - let b: [u8; 4] = self.bytes[at..at + 4].try_into().unwrap(); - let v = match self.endian { - Endian::Big => i32::from_be_bytes(b), - Endian::Little => i32::from_le_bytes(b), - }; - v as f64 - } - SampleFmt::F64 => { - let b: [u8; 8] = self.bytes[at..at + 8].try_into().unwrap(); - match self.endian { - Endian::Big => f64::from_be_bytes(b), - Endian::Little => f64::from_le_bytes(b), - } - } - } - } -} - -#[cfg(test)] -#[path = "bruker/parser_tests.rs"] -mod parser_tests; - -#[cfg(test)] -mod tests { - use super::*; - - #[test] - fn accepts_dir_or_fid_file() { - let dir = std::env::temp_dir().join(format!("plotx_bruker_{}", std::process::id())); - let _ = std::fs::remove_dir_all(&dir); - std::fs::create_dir_all(&dir).unwrap(); - std::fs::write( - dir.join("acqus"), - "##$TD= 4\n##$DTYPA= 2\n##$BYTORDA= 0\n##$SW_h= 1000\n##$SFO1= 400\n##$BF1= 400\n", - ) - .unwrap(); - let mut fid = Vec::new(); - for v in [1.0f64, 2.0, 3.0, 4.0] { - fid.extend_from_slice(&v.to_le_bytes()); - } - std::fs::write(dir.join("fid"), &fid).unwrap(); - - assert!(is_bruker(&dir)); - assert!(is_bruker(&dir.join("fid"))); - - let unwrap1d = |a: Acquisition| match a { - Acquisition::D1(d) => d, - Acquisition::D2(_) => panic!("expected 1D"), - Acquisition::Electrophysiology(_) => panic!("expected NMR"), - Acquisition::Afm(_) => panic!("expected NMR"), - Acquisition::MassSpec(_) | Acquisition::Xrd(_) => panic!("expected NMR"), - Acquisition::Xps(_) => panic!("expected NMR"), - }; - let from_dir = unwrap1d(read_bruker(&dir).unwrap()); - let from_file = unwrap1d(read_bruker(&dir.join("fid")).unwrap()); - assert_eq!( - from_dir.points, - vec![Complex64::new(1.0, 2.0), Complex64::new(3.0, 4.0)] - ); - assert_eq!(from_dir.points, from_file.points); - - let mut loaded = load_raw(&dir).unwrap(); - let origin = loaded.take_nmr_origin().unwrap(); - let instrument = origin.instrument().expect("raw Bruker origin"); - assert_eq!(instrument.format, NmrSourceFormat::BrukerRaw); - assert!(matches!( - instrument.parameters, - NmrSourceParameters::Bruker { .. } - )); - - std::fs::remove_dir_all(&dir).unwrap(); - } - - #[test] - fn reads_a_hand_built_2d_ser() { - let dir = std::env::temp_dir().join(format!("plotx_bruker2d_{}", std::process::id())); - let _ = std::fs::remove_dir_all(&dir); - std::fs::create_dir_all(&dir).unwrap(); - std::fs::write( - dir.join("acqus"), - "##$TD= 4\n##$DTYPA= 2\n##$BYTORDA= 0\n##$SW_h= 1000\n##$SFO1= 600\n##$BF1= 600\n\ - ##$O1= 1200\n##$NUC1= <1H>\n##$GRPDLY= 0\n##$PULPROG= \n", - ) - .unwrap(); - std::fs::write( - dir.join("acqu2s"), - "##$TD= 2\n##$SW_h= 1000\n##$SFO1= 600\n##$BF1= 600\n##$O1= 1200\n##$NUC1= <1H>\n\ - ##$FnMODE= 4\n", - ) - .unwrap(); - - // Two rows; each 1D FID (4 reals) padded to a 1024-byte block. - let mut ser = vec![0u8; 2 * SER_BLOCK_BYTES]; - for (row, vals) in [[1.0f64, 2.0, 3.0, 4.0], [5.0, 6.0, 7.0, 8.0]] - .iter() - .enumerate() - { - for (i, v) in vals.iter().enumerate() { - let off = row * SER_BLOCK_BYTES + i * 8; - ser[off..off + 8].copy_from_slice(&v.to_le_bytes()); - } - } - std::fs::write(dir.join("ser"), &ser).unwrap(); - - let two = match read_bruker(&dir).unwrap() { - Acquisition::D2(d) => *d, - Acquisition::D1(_) => panic!("expected 2D"), - Acquisition::Electrophysiology(_) => panic!("expected NMR"), - Acquisition::Afm(_) => panic!("expected NMR"), - Acquisition::MassSpec(_) | Acquisition::Xrd(_) => panic!("expected NMR"), - Acquisition::Xps(_) => panic!("expected NMR"), - }; - assert_eq!((two.cols, two.rows), (2, 2)); - assert_eq!( - two.data, - vec![ - Complex64::new(1.0, 2.0), - Complex64::new(3.0, 4.0), - Complex64::new(5.0, 6.0), - Complex64::new(7.0, 8.0), - ] - ); - assert_eq!(two.quad, QuadMode::States); - assert!(!two.indirect_conjugate); - assert_eq!(two.experiment.as_deref(), Some("cosygpppqf")); - assert!((two.direct.carrier_ppm - 2.0).abs() < 1e-9); - - std::fs::remove_dir_all(&dir).unwrap(); - } - - #[test] - fn source_prefix_prefers_title_then_sample_folder() { - let base = std::env::temp_dir().join(format!("plotx_bruker_name_{}", std::process::id())); - let _ = std::fs::remove_dir_all(&base); - let expno = base.join("Sucrose").join("3"); - std::fs::create_dir_all(expno.join("pdata").join("1")).unwrap(); - std::fs::create_dir_all(expno.join("pdata").join("2")).unwrap(); - - // An empty proc-1 title falls through to the next-lowest proc. - std::fs::write(expno.join("pdata").join("1").join("title"), " \n").unwrap(); - std::fs::write( - expno.join("pdata").join("2").join("title"), - "\nProton in CDCl3\n", - ) - .unwrap(); - assert_eq!(source_prefix(&expno), "Proton in CDCl3 (expno 3)"); - - // A non-empty proc-1 title wins. - std::fs::write(expno.join("pdata").join("1").join("title"), "Sucrose 1H\n").unwrap(); - assert_eq!(source_prefix(&expno), "Sucrose 1H (expno 3)"); - - // With no titles at all, the sample folder names the dataset. - std::fs::remove_file(expno.join("pdata").join("1").join("title")).unwrap(); - std::fs::remove_file(expno.join("pdata").join("2").join("title")).unwrap(); - assert_eq!(source_prefix(&expno), "Sucrose (expno 3)"); - - let params = JcampParams::parse("##$EXP= \n##$PULPROG= \n"); - let identity = acquisition_identity(&expno, Some(¶ms)); - assert_eq!(identity.subject.as_deref(), Some("Sucrose")); - assert_eq!(identity.acquisition.as_deref(), Some("COSY")); - assert_eq!(identity.source_label, "3"); - - std::fs::remove_dir_all(&base).unwrap(); - } -} diff --git a/crates/io/src/bruker/parser_tests.rs b/crates/io/src/bruker/parser_tests.rs deleted file mode 100644 index 99813649..00000000 --- a/crates/io/src/bruker/parser_tests.rs +++ /dev/null @@ -1,56 +0,0 @@ -use super::*; - -#[test] -fn parses_scalar_and_skips_arrays() { - let text = "\ -##TITLE= params -##$TD= 16384 -##$NUC1= <1H> -##$SW_h= 9615.38461538464 -##$GRPDLY= 76 -##$XGF= (0..3) -0 0 0 0 -##$O1= 2820.61 -"; - let params = JcampParams::parse(text); - assert_eq!(params.usize("TD"), Some(16384)); - assert_eq!(params.string("NUC1").as_deref(), Some("<1H>")); - assert_eq!(params.f64("GRPDLY"), Some(76.0)); - assert_eq!(params.f64("O1"), Some(2820.61)); - assert_eq!(params.string("XGF"), None); -} - -#[test] -fn group_delay_prefers_explicit_grpdly() { - let params = JcampParams::parse("##$GRPDLY= 67.98\n##$DSPFVS= 21\n##$DECIM= 2080\n"); - assert!((group_delay(¶ms) - 67.98).abs() < 1e-9); -} - -#[test] -fn group_delay_falls_back_to_table() { - let params = JcampParams::parse("##$GRPDLY= -1\n##$DSPFVS= 12\n##$DECIM= 16\n"); - assert!((group_delay(¶ms) - 71.625).abs() < 1e-9); -} - -#[test] -fn deinterleaves_complex_f64() { - // TD = 4 real values → 2 complex points: (1+2i), (3+4i). - let mut buffer = Vec::new(); - for value in [1.0f64, 2.0, 3.0, 4.0] { - buffer.extend_from_slice(&value.to_le_bytes()); - } - let reader = Reader { - bytes: &buffer, - endian: Endian::Little, - }; - let first = Complex64::new( - reader.real(0, SampleFmt::F64), - reader.real(8, SampleFmt::F64), - ); - let second = Complex64::new( - reader.real(16, SampleFmt::F64), - reader.real(24, SampleFmt::F64), - ); - assert_eq!(first, Complex64::new(1.0, 2.0)); - assert_eq!(second, Complex64::new(3.0, 4.0)); -} diff --git a/crates/io/src/bruker/processed.rs b/crates/io/src/bruker/processed.rs deleted file mode 100644 index 7d3e669a..00000000 --- a/crates/io/src/bruker/processed.rs +++ /dev/null @@ -1,334 +0,0 @@ -//! Bruker processed-data reader (`1r`/`1i`/`2rr`) built from the `procs` -//! parameter files under a `pdata` directory. - -use super::*; -use crate::NmrFormat; - -/// Identify a processed payload from a proc directory, a `1r`/`1i`/`2rr` -/// file, a `pdata` directory, or an experiment directory containing `pdata`. -pub fn detect_processed(path: &Path) -> Option { - let resolved = resolve_processed(path)?; - Some(if resolved.two_d { - DataFormat::Nmr(NmrFormat::BrukerProcessed2D) - } else { - DataFormat::Nmr(NmrFormat::BrukerProcessed1D) - }) -} - -struct ProcessedPaths { - proc_dir: PathBuf, - data_path: PathBuf, - two_d: bool, -} - -fn resolve_processed(path: &Path) -> Option { - if path.is_file() { - let name = path.file_name()?.to_str()?; - if matches!(name, "1r" | "1i" | "2rr") { - let proc_dir = path.parent()?.to_path_buf(); - let two_d = name == "2rr" || proc_dir.join("2rr").is_file(); - let data_path = if two_d { - proc_dir.join("2rr") - } else { - proc_dir.join("1r") - }; - return (proc_dir.join("procs").is_file() && data_path.is_file()).then_some( - ProcessedPaths { - proc_dir, - data_path, - two_d, - }, - ); - } - return None; - } - if !path.is_dir() { - return None; - } - - if path.join("procs").is_file() { - return processed_in_proc_dir(path); - } - let pdata = if path.file_name().and_then(|s| s.to_str()) == Some("pdata") { - path.to_path_buf() - } else { - path.join("pdata") - }; - let mut proc_dirs: Vec = std::fs::read_dir(pdata) - .ok()? - .filter_map(|e| e.ok().map(|e| e.path())) - .filter(|p| p.is_dir()) - .collect(); - proc_dirs.sort_by_key(|p| { - let procno = p - .file_name() - .and_then(|s| s.to_str()) - .and_then(|s| s.parse::().ok()) - .unwrap_or(u64::MAX); - (procno != 1, procno, p.clone()) - }); - proc_dirs.iter().find_map(|p| processed_in_proc_dir(p)) -} - -fn processed_in_proc_dir(proc_dir: &Path) -> Option { - if !proc_dir.join("procs").is_file() { - return None; - } - let (data_path, two_d) = if proc_dir.join("2rr").is_file() && proc_dir.join("proc2s").is_file() - { - (proc_dir.join("2rr"), true) - } else if proc_dir.join("1r").is_file() { - (proc_dir.join("1r"), false) - } else { - return None; - }; - Some(ProcessedPaths { - proc_dir: proc_dir.to_path_buf(), - data_path, - two_d, - }) -} - -pub fn load_processed(path: &Path) -> Result { - let resolved = resolve_processed(path).ok_or_else(|| { - IoError::Unsupported(format!( - "no complete Bruker processed dataset at {}", - path.display() - )) - })?; - let procs_path = resolved.proc_dir.join("procs"); - let procs = JcampParams::parse(&std::fs::read_to_string(&procs_path)?); - let mut warnings = Vec::new(); - let (mut acquisition, format, mut parameter_paths) = if resolved.two_d { - let proc2s_path = resolved.proc_dir.join("proc2s"); - let proc2s = JcampParams::parse(&std::fs::read_to_string(&proc2s_path)?); - ( - Acquisition::D2(Box::new(read_processed_2d( - &resolved.proc_dir, - &resolved.data_path, - &procs, - &proc2s, - )?)), - DataFormat::Nmr(NmrFormat::BrukerProcessed2D), - vec![procs_path, proc2s_path], - ) - } else { - let imag_path = resolved.proc_dir.join("1i"); - if !imag_path.is_file() { - warnings.push(LoadWarning { - code: LoadWarningCode::OptionalImaginaryMissing, - message: "Bruker 1i is absent; phase correction is limited to the real channel" - .into(), - path: Some(imag_path.clone()), - }); - } - ( - Acquisition::D1(read_processed_1d( - &resolved.proc_dir, - &resolved.data_path, - imag_path.is_file().then_some(imag_path.as_path()), - &procs, - )?), - DataFormat::Nmr(NmrFormat::BrukerProcessed1D), - vec![procs_path], - ) - }; - let experiment_dir = resolved.proc_dir.parent().and_then(Path::parent); - let acquisition_identity = experiment_dir - .map(|dir| { - let params = std::fs::read_to_string(dir.join("acqus")) - .ok() - .map(|text| JcampParams::parse(&text)); - super::acquisition_identity(dir, params.as_ref()) - }) - .unwrap_or_else(|| crate::AcquisitionIdentity::from_path(path)); - if let Acquisition::D2(data) = &mut acquisition { - data.experiment - .clone_from(&acquisition_identity.acquisition); - } - if let Some(acqus) = acquisition_params_for(&resolved.proc_dir, "acqus") { - parameter_paths.push(acqus); - } - if resolved.two_d - && let Some(acqu2s) = acquisition_params_for(&resolved.proc_dir, "acqu2s") - { - parameter_paths.push(acqu2s); - } - Ok(LoadResult::new( - acquisition, - acquisition_identity, - format, - Provenance { - selected_path: path.to_path_buf(), - data_path: resolved.data_path, - parameter_paths, - companion_paths: Vec::new(), - }, - warnings, - )) -} - -fn acquisition_params_for(proc_dir: &Path, name: &str) -> Option { - proc_dir - .parent()? - .parent() - .map(|experiment| experiment.join(name)) - .filter(|p| p.is_file()) -} - -fn processed_sample(params: &JcampParams) -> SampleFmt { - match params.i64("DTYPP").unwrap_or(0) { - 2 => SampleFmt::F64, - _ => SampleFmt::I32, - } -} - -fn processed_endian(params: &JcampParams) -> Endian { - match params.i64("BYTORDP").unwrap_or(0) { - 1 => Endian::Big, - _ => Endian::Little, - } -} - -fn read_processed_values( - path: &Path, - count: usize, - params: &JcampParams, -) -> Result, IoError> { - let sample = processed_sample(params); - let bytes = std::fs::read(path)?; - let need = count - .checked_mul(sample.size()) - .ok_or_else(|| IoError::Unsupported("processed SI overflow".into()))?; - if bytes.len() < need { - return Err(IoError::Truncated { - offset: 0, - needed: need, - have: bytes.len(), - }); - } - let reader = Reader { - bytes: &bytes, - endian: processed_endian(params), - }; - let scale = 2.0f64.powi( - params - .i64("NC_proc") - .unwrap_or(0) - .clamp(i32::MIN as i64, i32::MAX as i64) as i32, - ); - Ok((0..count) - .map(|i| reader.real(i * sample.size(), sample) * scale) - .collect()) -} - -fn processed_dim(params: &JcampParams) -> Dim { - let observe_freq_mhz = params - .f64("SF") - .filter(|v| v.is_finite() && *v > 1.0) - .unwrap_or(400.0); - let spectral_width_hz = params - .f64("SW_p") - .filter(|v| v.is_finite() && *v > 0.0) - .unwrap_or(observe_freq_mhz * 20.0); - let offset = params.f64("OFFSET").unwrap_or(0.0); - let nucleus = params - .string("AXNUC") - .map(|s| s.trim_matches(|c| c == '<' || c == '>').to_string()) - .filter(|s| !s.is_empty() && s != "off") - .unwrap_or_else(|| guess_nucleus(observe_freq_mhz)); - Dim { - spectral_width_hz, - observe_freq_mhz, - carrier_ppm: offset - spectral_width_hz / (2.0 * observe_freq_mhz), - nucleus, - group_delay: 0.0, - } -} - -fn read_processed_1d( - proc_dir: &Path, - real_path: &Path, - imag_path: Option<&Path>, - params: &JcampParams, -) -> Result { - let si = params - .usize("SI") - .filter(|n| *n > 0) - .ok_or_else(|| IoError::Unsupported("Bruker procs has no positive SI".into()))?; - let real = read_processed_values(real_path, si, params)?; - let imag = imag_path - .map(|p| read_processed_values(p, si, params)) - .transpose()?; - let mut points: Vec = (0..si) - .map(|i| Complex64::new(real[i], imag.as_ref().map_or(0.0, |v| v[i]))) - .collect(); - points.reverse(); - let dim = processed_dim(params); - Ok(NmrData { - points, - domain: Domain::Frequency, - spectral_width_hz: dim.spectral_width_hz, - observe_freq_mhz: dim.observe_freq_mhz, - carrier_ppm: dim.carrier_ppm, - nucleus: dim.nucleus, - source: format!( - "{} (Bruker TopSpin processed 1D, {si} pts)", - processed_source_prefix(proc_dir) - ), - group_delay: 0.0, - }) -} - -fn read_processed_2d( - proc_dir: &Path, - data_path: &Path, - f2: &JcampParams, - f1: &JcampParams, -) -> Result { - let cols = f2 - .usize("SI") - .filter(|n| *n > 0) - .ok_or_else(|| IoError::Unsupported("Bruker procs has no positive SI".into()))?; - let rows = f1 - .usize("SI") - .filter(|n| *n > 0) - .ok_or_else(|| IoError::Unsupported("Bruker proc2s has no positive SI".into()))?; - let stored = read_processed_values( - data_path, - rows.checked_mul(cols) - .ok_or_else(|| IoError::Unsupported("processed 2D SI overflow".into()))?, - f2, - )?; - let mut data = Vec::with_capacity(stored.len()); - for r in (0..rows).rev() { - for c in (0..cols).rev() { - data.push(Complex64::new(stored[r * cols + c], 0.0)); - } - } - Ok(NmrData2D { - data, - rows, - cols, - domain: Domain::Frequency, - direct: processed_dim(f2), - indirect: processed_dim(f1), - quad: QuadMode::Complex, - indirect_conjugate: false, - experiment: None, - pseudo_axis: None, - diffusion: None, - nus: None, - source: format!( - "{} (Bruker TopSpin processed 2D, {cols}x{rows})", - processed_source_prefix(proc_dir) - ), - }) -} - -fn processed_source_prefix(proc_dir: &Path) -> String { - let experiment = proc_dir.parent().and_then(Path::parent); - experiment - .map(source_prefix) - .unwrap_or_else(|| proc_dir.display().to_string()) -} diff --git a/crates/io/src/jcamp_dx.rs b/crates/io/src/jcamp_dx.rs deleted file mode 100644 index 1005bc08..00000000 --- a/crates/io/src/jcamp_dx.rs +++ /dev/null @@ -1,746 +0,0 @@ -//! Strict JCAMP-DX reader for one-dimensional, frequency-domain NMR spectra. -//! -//! This module deliberately owns the JCAMP label-record and ASDF semantics. It -//! is not related to Bruker's similarly shaped parameter files. - -use crate::{Acquisition, DataFormat, Domain, IoError, LoadResult, NmrData, NmrFormat, Provenance}; -use num_complex::Complex64; -use std::collections::HashMap; -use std::path::Path; - -#[derive(Debug, thiserror::Error)] -pub enum JcampDxError { - #[error("JCAMP-DX input is not valid UTF-8/ASCII text")] - InvalidTextEncoding, - #[error("malformed JCAMP-DX label record on line {line}: {detail}")] - MalformedRecord { line: usize, detail: String }, - #[error("required JCAMP-DX label ##{0}= is missing")] - MissingLabel(&'static str), - #[error("duplicate JCAMP-DX label ##{label}= is not valid for a single spectrum")] - DuplicateLabel { label: String }, - #[error("JCAMP-DX LINK/compound files are not supported")] - LinkDataset, - #[error("JCAMP-DX NTUPLES data are not supported")] - NtuplesDataset, - #[error("JCAMP-DX file contains more than one spectrum")] - MultipleSpectra, - #[error("unsupported JCAMP-DX DATA TYPE: {0}")] - UnsupportedDataType(String), - #[error("unsupported JCAMP-DX table declaration: {0}")] - UnsupportedTable(String), - #[error("unsupported JCAMP-DX {axis} unit: {unit}")] - UnsupportedUnit { axis: &'static str, unit: String }, - #[error("invalid value for JCAMP-DX ##{label}=: {value}")] - InvalidMetadata { label: &'static str, value: String }, - #[error("malformed JCAMP-DX XYDATA on line {line}: {detail}")] - MalformedData { line: usize, detail: String }, - #[error("JCAMP-DX invalid-data ordinate '?' is unsupported (line {line})")] - InvalidOrdinate { line: usize }, - #[error("JCAMP-DX X-sequence check failed on line {line}: expected {expected}, found {found}")] - XSequence { - line: usize, - expected: f64, - found: f64, - }, - #[error( - "JCAMP-DX DIF checkpoint failed on line {line}: expected ordinate {expected}, found {found}" - )] - Checkpoint { - line: usize, - expected: f64, - found: f64, - }, - #[error("JCAMP-DX decoded {actual} points, but ##NPOINTS= declares {declared}")] - PointCount { declared: usize, actual: usize }, -} - -#[derive(Debug)] -struct Document { - fields: HashMap, - xy_declaration: String, - data_lines: Vec<(usize, String)>, -} - -#[derive(Debug, Clone, Copy)] -enum XUnit { - Ppm, - Hz, -} - -#[derive(Debug, Clone, Copy)] -enum EncodedValue { - Actual(f64), - Difference(f64), - Duplicate(usize), - Invalid, -} - -#[derive(Debug, Clone, Copy)] -enum RepeatBasis { - Actual(f64), - Difference(f64), -} - -/// True for the registered JCAMP-DX filename extensions. -pub fn has_jcamp_extension(path: &Path) -> bool { - path.extension() - .and_then(|extension| extension.to_str()) - .map(|extension| { - matches!( - extension.to_ascii_lowercase().as_str(), - "dx" | "jdx" | "jcamp" - ) - }) - .unwrap_or(false) -} - -/// Load one standard JCAMP-DX 1D NMR spectrum with complete provenance. -pub fn load(path: &Path) -> Result { - let bytes = std::fs::read(path)?; - let acquisition = parse_bytes(&bytes, path.to_string_lossy().as_ref())?; - Ok(LoadResult::new( - acquisition, - crate::AcquisitionIdentity::from_path(path), - DataFormat::Nmr(NmrFormat::JcampDx1D), - Provenance { - selected_path: path.to_path_buf(), - data_path: path.to_path_buf(), - parameter_paths: Vec::new(), - companion_paths: Vec::new(), - }, - Vec::new(), - )) -} - -fn parse_bytes(bytes: &[u8], source: &str) -> Result { - let text = std::str::from_utf8(bytes).map_err(|_| JcampDxError::InvalidTextEncoding)?; - if !text.is_ascii() { - return Err(JcampDxError::InvalidTextEncoding); - } - let document = parse_document(text)?; - parse_spectrum(document, source).map(Acquisition::D1) -} - -fn parse_document(text: &str) -> Result { - let mut fields = HashMap::new(); - let mut xy_declaration = None; - let mut data_lines = Vec::new(); - let mut in_xydata = false; - let mut title_count = 0usize; - - for (index, original) in text.lines().enumerate() { - let line_number = index + 1; - let uncommented = original.split("$$").next().unwrap_or("").trim_end(); - let trimmed = uncommented.trim_start(); - if let Some(record) = trimmed.strip_prefix("##") { - in_xydata = false; - let (raw_label, raw_value) = - record - .split_once('=') - .ok_or_else(|| JcampDxError::MalformedRecord { - line: line_number, - detail: "label record has no '=' delimiter".to_owned(), - })?; - let label = normalize_label(raw_label); - let value = raw_value.trim().to_owned(); - if label.is_empty() { - return Err(JcampDxError::MalformedRecord { - line: line_number, - detail: "empty label".to_owned(), - }); - } - - match label.as_str() { - "TITLE" => { - title_count += 1; - if title_count > 1 { - return Err(JcampDxError::MultipleSpectra); - } - insert_unique(&mut fields, label, value)?; - } - "XYDATA" => { - if xy_declaration.replace(value).is_some() { - return Err(JcampDxError::DuplicateLabel { - label: "XYDATA".to_owned(), - }); - } - in_xydata = true; - } - "NTUPLES" | "VARNAME" | "SYMBOL" | "DATATABLE" | "PAGE" => { - return Err(JcampDxError::NtuplesDataset); - } - "XYPOINTS" | "PEAKTABLE" | "PEAKASSIGNMENTS" => { - return Err(JcampDxError::UnsupportedTable(raw_label.trim().to_owned())); - } - "END" => {} - _ if is_core_label(&label) => insert_unique(&mut fields, label, value)?, - _ => {} - } - } else if in_xydata && !trimmed.is_empty() { - data_lines.push((line_number, uncommented.trim().to_owned())); - } - } - - let xy_declaration = xy_declaration.ok_or(JcampDxError::MissingLabel("XYDATA"))?; - Ok(Document { - fields, - xy_declaration, - data_lines, - }) -} - -fn insert_unique( - fields: &mut HashMap, - label: String, - value: String, -) -> Result<(), JcampDxError> { - if fields.insert(label.clone(), value).is_some() { - return Err(JcampDxError::DuplicateLabel { label }); - } - Ok(()) -} - -fn normalize_label(label: &str) -> String { - label - .trim() - .trim_start_matches(['.', '$']) - .chars() - .filter(|character| !character.is_ascii_whitespace() && !matches!(character, '-' | '_')) - .map(|character| character.to_ascii_uppercase()) - .collect() -} - -fn is_core_label(label: &str) -> bool { - matches!( - label, - "DATATYPE" - | "XUNITS" - | "YUNITS" - | "FIRSTX" - | "LASTX" - | "NPOINTS" - | "XFACTOR" - | "YFACTOR" - | "OBSERVEFREQUENCY" - | "OBSERVENUCLEUS" - | "BLOCKS" - ) -} - -fn parse_spectrum(document: Document, source: &str) -> Result { - required(&document.fields, "TITLE", "TITLE")?; - let data_type = required(&document.fields, "DATATYPE", "DATA TYPE")?; - let normalized_data_type = data_type.trim().to_ascii_uppercase(); - if normalized_data_type.contains("LINK") || document.fields.contains_key("BLOCKS") { - return Err(JcampDxError::LinkDataset); - } - if !normalized_data_type.contains("NMR") || !normalized_data_type.contains("SPECTRUM") { - return Err(JcampDxError::UnsupportedDataType(data_type.to_owned())); - } - - let declaration: String = document - .xy_declaration - .chars() - .filter(|character| !character.is_ascii_whitespace()) - .map(|character| character.to_ascii_uppercase()) - .collect(); - if declaration != "(X++(Y..Y))" { - return Err(JcampDxError::UnsupportedTable(document.xy_declaration)); - } - - let x_unit_raw = required(&document.fields, "XUNITS", "XUNITS")?; - let x_unit = match normalized_unit(x_unit_raw).as_str() { - "PPM" => XUnit::Ppm, - "HZ" | "HERTZ" => XUnit::Hz, - _ => { - return Err(JcampDxError::UnsupportedUnit { - axis: "X", - unit: x_unit_raw.to_owned(), - }); - } - }; - let y_unit_raw = required(&document.fields, "YUNITS", "YUNITS")?; - if !matches!( - normalized_unit(y_unit_raw).as_str(), - "ARBITRARYUNITS" | "RELATIVEINTENSITY" | "INTENSITY" | "COUNTS" - ) { - return Err(JcampDxError::UnsupportedUnit { - axis: "Y", - unit: y_unit_raw.to_owned(), - }); - } - - let first_x = field_f64(&document.fields, "FIRSTX", "FIRSTX")?; - let last_x = field_f64(&document.fields, "LASTX", "LASTX")?; - let npoints = field_usize(&document.fields, "NPOINTS", "NPOINTS")?; - if npoints < 2 { - return Err(JcampDxError::InvalidMetadata { - label: "NPOINTS", - value: npoints.to_string(), - }); - } - if first_x == last_x { - return Err(JcampDxError::InvalidMetadata { - label: "FIRSTX/LASTX", - value: first_x.to_string(), - }); - } - - let x_factor = optional_field_f64(&document.fields, "XFACTOR", "XFACTOR")?.unwrap_or(1.0); - let y_factor = optional_field_f64(&document.fields, "YFACTOR", "YFACTOR")?.unwrap_or(1.0); - if x_factor == 0.0 || y_factor == 0.0 { - return Err(JcampDxError::InvalidMetadata { - label: if x_factor == 0.0 { - "XFACTOR" - } else { - "YFACTOR" - }, - value: "0".to_owned(), - }); - } - - let observe_freq_mhz = field_f64(&document.fields, "OBSERVEFREQUENCY", "OBSERVE FREQUENCY")?; - if observe_freq_mhz <= 0.0 { - return Err(JcampDxError::InvalidMetadata { - label: "OBSERVE FREQUENCY", - value: observe_freq_mhz.to_string(), - }); - } - let nucleus_raw = required(&document.fields, "OBSERVENUCLEUS", "OBSERVE NUCLEUS")?; - let nucleus = normalize_nucleus(nucleus_raw); - if nucleus.is_empty() { - return Err(JcampDxError::InvalidMetadata { - label: "OBSERVE NUCLEUS", - value: nucleus_raw.to_owned(), - }); - } - - let mut ordinates = decode_xydata(&document.data_lines, first_x, last_x, npoints, x_factor)?; - for ordinate in &mut ordinates { - *ordinate *= y_factor; - if !ordinate.is_finite() { - return Err(JcampDxError::InvalidMetadata { - label: "YFACTOR", - value: y_factor.to_string(), - }); - } - } - - let to_ppm = |x: f64| match x_unit { - XUnit::Ppm => x, - XUnit::Hz => x / observe_freq_mhz, - }; - let first_ppm = to_ppm(first_x); - let last_ppm = to_ppm(last_x); - let (low_ppm, high_ppm) = if first_ppm <= last_ppm { - (first_ppm, last_ppm) - } else { - ordinates.reverse(); - (last_ppm, first_ppm) - }; - let step_ppm = (high_ppm - low_ppm) / (npoints - 1) as f64; - let spectral_width_hz = step_ppm * observe_freq_mhz * npoints as f64; - let carrier_ppm = low_ppm + npoints as f64 * step_ppm / 2.0; - let points = ordinates - .into_iter() - .map(|ordinate| Complex64::new(ordinate, 0.0)) - .collect(); - - Ok(NmrData { - points, - domain: Domain::Frequency, - spectral_width_hz, - observe_freq_mhz, - carrier_ppm, - nucleus, - source: format!("{source} (JCAMP-DX 1D NMR, {npoints} pts)"), - group_delay: 0.0, - }) -} - -fn normalized_unit(unit: &str) -> String { - unit.chars() - .filter(|character| !character.is_ascii_whitespace() && !matches!(character, '-' | '_')) - .map(|character| character.to_ascii_uppercase()) - .collect() -} - -fn normalize_nucleus(nucleus: &str) -> String { - nucleus - .trim() - .trim_matches(|character| matches!(character, '<' | '>' | '"' | '\'')) - .chars() - .filter(|character| !character.is_ascii_whitespace() && *character != '^') - .collect() -} - -fn required<'a>( - fields: &'a HashMap, - key: &str, - label: &'static str, -) -> Result<&'a str, JcampDxError> { - fields - .get(key) - .map(String::as_str) - .filter(|value| !value.trim().is_empty()) - .ok_or(JcampDxError::MissingLabel(label)) -} - -fn field_f64( - fields: &HashMap, - key: &str, - label: &'static str, -) -> Result { - let raw = required(fields, key, label)?; - parse_finite(raw).ok_or_else(|| JcampDxError::InvalidMetadata { - label, - value: raw.to_owned(), - }) -} - -fn optional_field_f64( - fields: &HashMap, - key: &str, - label: &'static str, -) -> Result, JcampDxError> { - let Some(raw) = fields.get(key) else { - return Ok(None); - }; - parse_finite(raw) - .map(Some) - .ok_or_else(|| JcampDxError::InvalidMetadata { - label, - value: raw.to_owned(), - }) -} - -fn field_usize( - fields: &HashMap, - key: &str, - label: &'static str, -) -> Result { - let raw = required(fields, key, label)?; - raw.trim() - .parse::() - .map_err(|_| JcampDxError::InvalidMetadata { - label, - value: raw.to_owned(), - }) -} - -fn parse_finite(raw: &str) -> Option { - raw.trim() - .parse::() - .ok() - .filter(|value| value.is_finite()) -} - -fn decode_xydata( - lines: &[(usize, String)], - first_x: f64, - last_x: f64, - npoints: usize, - x_factor: f64, -) -> Result, JcampDxError> { - let increment = (last_x - first_x) / (npoints - 1) as f64; - let mut values = Vec::with_capacity(npoints); - let mut previous_y = None; - let mut previous_line_ended_in_difference = false; - - for (line_number, line) in lines { - let (encoded_x, remainder) = split_line_x(line, *line_number)?; - let x = encoded_x * x_factor; - let checkpoint = previous_line_ended_in_difference && !values.is_empty(); - let index = if checkpoint { - values.len() - 1 - } else { - values.len() - }; - if index >= npoints { - return Err(JcampDxError::PointCount { - declared: npoints, - actual: values.len() + 1, - }); - } - let expected_x = first_x + index as f64 * increment; - if !axis_close(x, expected_x, increment) { - return Err(JcampDxError::XSequence { - line: *line_number, - expected: expected_x, - found: x, - }); - } - - let tokens = tokenize_ordinates(remainder, *line_number)?; - if tokens.is_empty() { - return Err(JcampDxError::MalformedData { - line: *line_number, - detail: "data line has no ordinates".to_owned(), - }); - } - let first = tokens[0]; - let first_actual = match first { - EncodedValue::Actual(value) => value, - EncodedValue::Invalid => { - return Err(JcampDxError::InvalidOrdinate { line: *line_number }); - } - _ => { - return Err(JcampDxError::MalformedData { - line: *line_number, - detail: "the first ordinate of a line must be an absolute AFFN/SQZ value" - .to_owned(), - }); - } - }; - - let mut basis = RepeatBasis::Actual(first_actual); - if checkpoint { - let expected_y = previous_y.expect("checkpoint requires a preceding ordinate"); - if !ordinate_close(first_actual, expected_y) { - return Err(JcampDxError::Checkpoint { - line: *line_number, - expected: expected_y, - found: first_actual, - }); - } - } else { - append_value(&mut values, first_actual, npoints)?; - previous_y = Some(first_actual); - } - - for token in tokens.into_iter().skip(1) { - match token { - EncodedValue::Actual(value) => { - append_value(&mut values, value, npoints)?; - previous_y = Some(value); - basis = RepeatBasis::Actual(value); - } - EncodedValue::Difference(difference) => { - let value = previous_y.ok_or_else(|| JcampDxError::MalformedData { - line: *line_number, - detail: "DIF value has no preceding ordinate".to_owned(), - })? + difference; - append_value(&mut values, value, npoints)?; - previous_y = Some(value); - basis = RepeatBasis::Difference(difference); - } - EncodedValue::Duplicate(count) => { - if count < 2 { - return Err(JcampDxError::MalformedData { - line: *line_number, - detail: "DUP count must include at least two values".to_owned(), - }); - } - for _ in 1..count { - let value = match basis { - RepeatBasis::Actual(value) => value, - RepeatBasis::Difference(difference) => { - previous_y.expect("DIF duplicate requires a preceding ordinate") - + difference - } - }; - append_value(&mut values, value, npoints)?; - previous_y = Some(value); - } - } - EncodedValue::Invalid => { - return Err(JcampDxError::InvalidOrdinate { line: *line_number }); - } - } - } - previous_line_ended_in_difference = matches!(basis, RepeatBasis::Difference(_)); - } - - if values.len() != npoints { - return Err(JcampDxError::PointCount { - declared: npoints, - actual: values.len(), - }); - } - Ok(values) -} - -fn append_value(values: &mut Vec, value: f64, declared: usize) -> Result<(), JcampDxError> { - if !value.is_finite() || values.len() >= declared { - return Err(JcampDxError::PointCount { - declared, - actual: values.len() + 1, - }); - } - values.push(value); - Ok(()) -} - -fn axis_close(actual: f64, expected: f64, increment: f64) -> bool { - let tolerance = (expected.abs().max(1.0) * 1.0e-9).max(increment.abs() * 1.0e-5); - (actual - expected).abs() <= tolerance -} - -fn ordinate_close(actual: f64, expected: f64) -> bool { - (actual - expected).abs() <= expected.abs().max(1.0) * 1.0e-10 -} - -fn split_line_x(line: &str, line_number: usize) -> Result<(f64, &str), JcampDxError> { - let text = line.trim_start(); - let length = numeric_prefix_len(text); - if length == 0 { - return Err(JcampDxError::MalformedData { - line: line_number, - detail: "line does not start with an AFFN X value".to_owned(), - }); - } - let x = parse_finite(&text[..length]).ok_or_else(|| JcampDxError::MalformedData { - line: line_number, - detail: "invalid AFFN X value".to_owned(), - })?; - Ok((x, &text[length..])) -} - -fn tokenize_ordinates(text: &str, line_number: usize) -> Result, JcampDxError> { - let bytes = text.as_bytes(); - let mut tokens = Vec::new(); - let mut offset = 0usize; - while offset < bytes.len() { - let byte = bytes[offset]; - if byte.is_ascii_whitespace() || matches!(byte, b',' | b';') { - offset += 1; - continue; - } - if byte == b'?' { - tokens.push(EncodedValue::Invalid); - offset += 1; - continue; - } - if let Some((kind, leading, sign)) = pseudo_digit(byte) { - let mut end = offset + 1; - while end < bytes.len() && bytes[end].is_ascii_digit() { - end += 1; - } - match kind { - PseudoKind::Squeezed => { - tokens.push(EncodedValue::Actual(pseudo_number( - leading, - sign, - &bytes[offset + 1..end], - ))); - } - PseudoKind::Difference => { - tokens.push(EncodedValue::Difference(pseudo_number( - leading, - sign, - &bytes[offset + 1..end], - ))); - } - PseudoKind::Duplicate => { - tokens.push(EncodedValue::Duplicate(pseudo_count( - leading, - &bytes[offset + 1..end], - line_number, - )?)); - } - } - offset = end; - continue; - } - - let length = numeric_prefix_len(&text[offset..]); - if length == 0 { - return Err(JcampDxError::MalformedData { - line: line_number, - detail: format!("unexpected character {:?}", byte as char), - }); - } - let raw = &text[offset..offset + length]; - let value = parse_finite(raw).ok_or_else(|| JcampDxError::MalformedData { - line: line_number, - detail: format!("invalid AFFN/PAC ordinate {raw:?}"), - })?; - tokens.push(EncodedValue::Actual(value)); - offset += length; - } - Ok(tokens) -} - -#[derive(Debug, Clone, Copy)] -enum PseudoKind { - Squeezed, - Difference, - Duplicate, -} - -fn pseudo_digit(byte: u8) -> Option<(PseudoKind, u8, f64)> { - match byte { - b'@' => Some((PseudoKind::Squeezed, 0, 1.0)), - b'A'..=b'I' => Some((PseudoKind::Squeezed, byte - b'A' + 1, 1.0)), - b'a'..=b'i' => Some((PseudoKind::Squeezed, byte - b'a' + 1, -1.0)), - b'%' => Some((PseudoKind::Difference, 0, 1.0)), - b'J'..=b'R' => Some((PseudoKind::Difference, byte - b'J' + 1, 1.0)), - b'j'..=b'r' => Some((PseudoKind::Difference, byte - b'j' + 1, -1.0)), - b'S'..=b'Z' => Some((PseudoKind::Duplicate, byte - b'S' + 1, 1.0)), - b's' => Some((PseudoKind::Duplicate, 9, 1.0)), - _ => None, - } -} - -fn pseudo_number(leading: u8, sign: f64, tail: &[u8]) -> f64 { - let mut value = leading as f64; - for digit in tail { - value = value * 10.0 + (digit - b'0') as f64; - } - sign * value -} - -fn pseudo_count(leading: u8, tail: &[u8], line_number: usize) -> Result { - let mut value = leading as usize; - for digit in tail { - value = value - .checked_mul(10) - .and_then(|value| value.checked_add((digit - b'0') as usize)) - .ok_or_else(|| JcampDxError::MalformedData { - line: line_number, - detail: "DUP count overflows usize".to_owned(), - })?; - } - Ok(value) -} - -fn numeric_prefix_len(text: &str) -> usize { - let bytes = text.as_bytes(); - let mut index = 0usize; - if matches!(bytes.first(), Some(b'+') | Some(b'-')) { - index += 1; - } - let mut digits = 0usize; - while index < bytes.len() && bytes[index].is_ascii_digit() { - index += 1; - digits += 1; - } - if index < bytes.len() && bytes[index] == b'.' { - index += 1; - while index < bytes.len() && bytes[index].is_ascii_digit() { - index += 1; - digits += 1; - } - } - if digits == 0 { - return 0; - } - if index < bytes.len() && matches!(bytes[index], b'E' | b'e') { - let exponent_start = index; - index += 1; - if index < bytes.len() && matches!(bytes[index], b'+' | b'-') { - index += 1; - } - let exponent_digits = index; - while index < bytes.len() && bytes[index].is_ascii_digit() { - index += 1; - } - if index == exponent_digits { - return exponent_start; - } - } - index -} - -#[cfg(test)] -mod tests; diff --git a/crates/io/src/jcamp_dx/tests.rs b/crates/io/src/jcamp_dx/tests.rs deleted file mode 100644 index 419f01d2..00000000 --- a/crates/io/src/jcamp_dx/tests.rs +++ /dev/null @@ -1,123 +0,0 @@ -use super::*; - -fn header(extra: &str, body: &str) -> String { - format!( - "##TITLE=fixture\n\ - ##JCAMP-DX=5.01\n\ - ##DATA TYPE=NMR SPECTRUM\n\ - ##XUNITS=PPM\n\ - ##YUNITS=ARBITRARY UNITS\n\ - ##XFACTOR=1\n\ - ##YFACTOR=1\n\ - ##FIRSTX=0\n\ - ##LASTX=3\n\ - ##NPOINTS=4\n\ - ##.OBSERVE FREQUENCY=400\n\ - ##.OBSERVE NUCLEUS=^1H\n\ - {extra}\ - ##XYDATA=(X++(Y..Y))\n\ - {body}\n\ - ##END=\n" - ) -} - -fn data(text: &str) -> NmrData { - match parse_bytes(text.as_bytes(), "fixture.jdx").unwrap() { - Acquisition::D1(data) => data, - Acquisition::D2(_) => panic!("expected 1D data"), - Acquisition::Electrophysiology(_) => panic!("expected NMR"), - Acquisition::Afm(_) => panic!("expected NMR"), - Acquisition::MassSpec(_) | Acquisition::Xrd(_) => panic!("expected NMR"), - Acquisition::Xps(_) => panic!("expected NMR"), - } -} - -#[test] -fn decodes_affn_and_pac() { - let spectrum = data(&header("", "0 1+2-3+4")); - let values: Vec = spectrum.points.iter().map(|point| point.re).collect(); - assert_eq!(values, vec![1.0, 2.0, -3.0, 4.0]); -} - -#[test] -fn decodes_sqz_dif_dup_and_checkpoint_continuity() { - let fixture = "##TITLE=compressed\n\ - ##DATA TYPE=NMR SPECTRUM\n\ - ##XUNITS=PPM\n\ - ##YUNITS=RELATIVE INTENSITY\n\ - ##XFACTOR=1\n\ - ##YFACTOR=0.5\n\ - ##FIRSTX=0\n\ - ##LASTX=7\n\ - ##NPOINTS=8\n\ - ##.OBSERVE FREQUENCY=400\n\ - ##.OBSERVE NUCLEUS=<1H>\n\ - ##XYDATA=(X++(Y..Y))\n\ - 0A0KU\n\ - 3A6%TjN\n\ - 7B0 $$ final DIF checkpoint\n\ - ##END=\n"; - let spectrum = data(fixture); - let values: Vec = spectrum.points.iter().map(|point| point.re).collect(); - assert_eq!(values, vec![5.0, 6.0, 7.0, 8.0, 8.0, 8.0, 7.5, 10.0]); -} - -#[test] -fn applies_factors_and_canonicalizes_a_descending_axis() { - let fixture = "##TITLE=reverse\n\ - ##DATA TYPE=NMR SPECTRUM\n\ - ##XUNITS=PPM\n\ - ##YUNITS=ARBITRARY UNITS\n\ - ##XFACTOR=0.5\n\ - ##YFACTOR=0.25\n\ - ##FIRSTX=10\n\ - ##LASTX=7\n\ - ##NPOINTS=4\n\ - ##.OBSERVE FREQUENCY=400\n\ - ##.OBSERVE NUCLEUS=1H\n\ - ##XYDATA=(X++(Y..Y))\n\ - 20 2 4\n\ - 16 6 8\n\ - ##END=\n"; - let spectrum = data(fixture); - let values: Vec = spectrum.points.iter().map(|point| point.re).collect(); - assert_eq!(values, vec![2.0, 1.5, 1.0, 0.5]); - assert!((spectrum.spectral_width_hz - 1600.0).abs() < 1.0e-12); - assert!((spectrum.carrier_ppm - 9.0).abs() < 1.0e-12); -} - -#[test] -fn rejects_compound_and_ntuples_documents() { - let link = header("##BLOCKS=2\n##DATA TYPE=LINK\n", "0 1 2 3 4"); - assert!(matches!( - parse_bytes(link.as_bytes(), "link.jdx"), - Err(JcampDxError::DuplicateLabel { .. }) | Err(JcampDxError::LinkDataset) - )); - - let ntuples = header("##NTUPLES=NMR SPECTRUM\n", "0 1 2 3 4"); - assert!(matches!( - parse_bytes(ntuples.as_bytes(), "ntuples.jdx"), - Err(JcampDxError::NtuplesDataset) - )); -} - -#[test] -fn rejects_missing_metadata_unsupported_units_and_bad_checkpoints() { - let missing = header("", "0 1 2 3 4").replace("##.OBSERVE NUCLEUS=^1H\n", ""); - assert!(matches!( - parse_bytes(missing.as_bytes(), "missing.jdx"), - Err(JcampDxError::MissingLabel("OBSERVE NUCLEUS")) - )); - - let unit = header("", "0 1 2 3 4").replace("##XUNITS=PPM", "##XUNITS=SECONDS"); - assert!(matches!( - parse_bytes(unit.as_bytes(), "unit.jdx"), - Err(JcampDxError::UnsupportedUnit { axis: "X", .. }) - )); - - let checkpoint = header("", "0A0K\n1A3KK\n3A6"); - assert!(matches!( - parse_bytes(checkpoint.as_bytes(), "checkpoint.jdx"), - Err(JcampDxError::Checkpoint { .. }) - )); -} diff --git a/crates/io/src/jeol.rs b/crates/io/src/jeol.rs deleted file mode 100644 index e24870de..00000000 --- a/crates/io/src/jeol.rs +++ /dev/null @@ -1,746 +0,0 @@ -//! JEOL Delta `.jdf` reader. - -use crate::{ - Acquisition, AcquisitionIdentity, DataFormat, DiffusionMeta, Dim, Domain, IoError, LoadResult, - NmrData, NmrData2D, NmrFormat, NmrInstrumentOrigin, NmrOrigin, NmrPortableMetadata, - NmrSourceFormat, NmrSourceParameters, Provenance, PseudoAxis, PseudoKind, QuadMode, - gradient_shape_factor, gyromagnetic_ratio, -}; -use base64::{Engine as _, engine::general_purpose::STANDARD}; -use num_complex::Complex64; -use sha2::{Digest, Sha256}; -use std::path::Path; - -mod filter; -mod nus; -mod params; -mod ruler; -use filter::group_delay; -use nus::detect_nus; -#[cfg(test)] -use nus::extract_nuslist; -use params::Params; -use ruler::{kind_for_unit, prefix_exponent, scan_embedded_axis}; - -const MAGIC: &[u8; 8] = b"JEOL.NMR"; -const HEADER_LEN: usize = 1360; - -// Byte offsets into the fixed big-endian header. Array fields hold one slot per -// possible dimension; slot 0 is read for 1D data. -#[allow(dead_code)] -mod off { - pub const ENDIAN: usize = 8; // u8: body endianness, 0 = big, 1 = little - pub const MAJOR_VERSION: usize = 9; // u8 - pub const DATA_DIMENSION_NUMBER: usize = 12; // u8 - pub const DATA_AXIS_TYPE: usize = 24; // 8 × u8 (0 None, 1 Real, 3 Complex, ...) - pub const DATA_AXIS_UNITS: usize = 32; // 8 × (unit prefix/power u8, base unit u8) - pub const DATA_POINTS: usize = 176; // 8 × u32 (per axis, padded to a tile edge) - pub const DATA_OFFSET_STOP: usize = 240; // 8 × u32 (per axis, last real index) - pub const DATA_AXIS_START: usize = 272; // 8 × f64 (axis low end) - pub const DATA_AXIS_STOP: usize = 336; // 8 × f64 (axis high end; for a FID = acq time, s) - pub const BASE_FREQ: usize = 1064; // 8 × f64 (MHz) - pub const PARAM_LIST: usize = 1360; // parameter-list header, right after the fixed header - pub const DATA_START: usize = 1284; // u32: byte offset of the data section - pub const DATA_LENGTH: usize = 1288; // u64: length of the data section in bytes -} - -const AXIS_COMPLEX: u8 = 3; -const AXIS_REAL_COMPLEX: u8 = 4; -const UNIT_HERTZ: u8 = 13; -const UNIT_PPM: u8 = 26; -const UNIT_SECOND: u8 = 28; - -#[derive(Clone, Copy, Debug, PartialEq, Eq)] -enum AxisUnit { - Hertz, - Ppm, - Second, -} - -impl AxisUnit { - fn decode(bytes: &[u8], axis: usize) -> Result<(Self, f64), IoError> { - let at = off::DATA_AXIS_UNITS + axis * 2; - let unit = match bytes[at + 1] { - UNIT_HERTZ => Self::Hertz, - UNIT_PPM => Self::Ppm, - UNIT_SECOND => Self::Second, - unit => { - return Err(IoError::Unsupported(format!( - "unknown JEOL unit {unit} for axis {}", - axis + 1 - ))); - } - }; - let scale = 10f64.powi(prefix_exponent(bytes[at])); - Ok((unit, scale)) - } -} - -// Edge of the square submatrix tiles nD data is stored in. Data_Points are -// padded up to a multiple of this along every axis. True-2D data uses the 32 -// edge; pseudo-2D arrays with few increments use the 4 edge. -const TILE: usize = 32; -const SMALL_TILE: usize = 4; - -/// True if the file begins with the JEOL Delta magic, regardless of extension. -pub fn is_jdf(path: &Path) -> bool { - use std::io::Read; - let mut magic = [0u8; MAGIC.len()]; - std::fs::File::open(path) - .and_then(|mut f| f.read_exact(&mut magic)) - .map(|()| &magic == MAGIC) - .unwrap_or(false) -} - -pub fn read_jdf_path(path: &Path) -> Result { - let bytes = std::fs::read(path)?; - let source = path - .file_name() - .and_then(|s| s.to_str()) - .unwrap_or("") - .to_string(); - read_jdf_bytes(&bytes, source) -} - -pub fn load_jdf_path(path: &Path) -> Result { - let bytes = std::fs::read(path)?; - let source = path - .file_name() - .and_then(|s| s.to_str()) - .unwrap_or("") - .to_owned(); - let acquisition = read_jdf_bytes(&bytes, source)?; - let endian = if bytes[off::ENDIAN] == 0 { - Endian::Big - } else { - Endian::Little - }; - let params = Params::parse(&bytes, off::PARAM_LIST, endian); - let mut acquisition_identity = AcquisitionIdentity::from_path(path); - acquisition_identity.acquisition = experiment_name(¶ms); - let header = Reader { - bytes: &bytes, - endian: Endian::Big, - }; - let data_start = (header.u32(off::DATA_START) as usize).clamp(HEADER_LEN, bytes.len()); - Ok(LoadResult::new( - acquisition, - acquisition_identity, - DataFormat::Nmr(NmrFormat::JeolDelta), - Provenance { - selected_path: path.to_path_buf(), - data_path: path.to_path_buf(), - parameter_paths: Vec::new(), - companion_paths: Vec::new(), - }, - Vec::new(), - ) - .with_nmr_origin(NmrOrigin::Instrument { - instrument: NmrInstrumentOrigin { - format: NmrSourceFormat::JeolDelta, - source_sha256: Sha256::digest(&bytes).into(), - portable: NmrPortableMetadata { - solvent: params.string_ci("solvent"), - temperature_k: None, - transients: params - .f64("scans") - .filter(|value| *value > 0.0 && value.fract() == 0.0) - .map(|value| value as u64), - pulse_sequence: experiment_name(¶ms), - }, - parameters: NmrSourceParameters::Jeol { - metadata_base64: STANDARD.encode(&bytes[..data_start]), - }, - }, - })) -} - -pub fn read_jdf_bytes(bytes: &[u8], source: String) -> Result { - if bytes.len() < HEADER_LEN { - return Err(IoError::Truncated { - offset: 0, - needed: HEADER_LEN, - have: bytes.len(), - }); - } - if &bytes[..8] != MAGIC { - return Err(IoError::BadMagic); - } - - let body_endian = match bytes[off::ENDIAN] { - 0 => Endian::Big, - 1 => Endian::Little, - other => { - return Err(IoError::Unsupported(format!( - "unknown endian marker {other} at byte 8" - ))); - } - }; - - match bytes[off::DATA_DIMENSION_NUMBER] { - 1 => read_jdf_1d(bytes, source, body_endian).map(Acquisition::D1), - 2 => read_jdf_2d(bytes, source, body_endian).map(|d| Acquisition::D2(Box::new(d))), - ndim => Err(IoError::Unsupported(format!( - "{ndim}-dimensional data (only 1D and 2D are implemented)" - ))), - } -} - -fn read_jdf_1d(bytes: &[u8], source: String, body_endian: Endian) -> Result { - // The fixed header is always big-endian; the body follows the Endian byte. - let h = Reader { - bytes, - endian: Endian::Big, - }; - - let axis_type = bytes[off::DATA_AXIS_TYPE]; - let components = match axis_type { - AXIS_COMPLEX | AXIS_REAL_COMPLEX => 2, - _ => 1, - }; - - let npoints = h.u32(off::DATA_POINTS) as usize; - if npoints == 0 { - return Err(IoError::Unsupported( - "header reports zero data points".into(), - )); - } - let raw_axis_start = h.f64(off::DATA_AXIS_START); - let raw_axis_stop = h.f64(off::DATA_AXIS_STOP); - let (axis_unit, axis_scale) = AxisUnit::decode(bytes, 0)?; - let axis_start = raw_axis_start * axis_scale; - let axis_stop = raw_axis_stop * axis_scale; - let domain = match axis_unit { - AxisUnit::Hertz | AxisUnit::Ppm => Domain::Frequency, - AxisUnit::Second => Domain::Time, - }; - // DATA_POINTS is padded to a tile edge; DATA_OFFSET_STOP is the last real - // time-domain index, so real count = stop+1. Processed spectra use the full - // DATA_POINTS array; their offset-stop describes the processing window, not - // trailing storage padding, and truncating there shifts every ppm coordinate. - let real_n = if domain == Domain::Time { - let stop = h.u32(off::DATA_OFFSET_STOP) as usize; - if stop > 0 { - (stop + 1).min(npoints) - } else { - npoints - } - } else { - npoints - }; - - let base_freq_mhz = h.f64(off::BASE_FREQ); - let axis_span = (axis_stop - axis_start).abs(); - - let params = Params::parse(bytes, off::PARAM_LIST, body_endian); - - let data_start = { - let ds = h.u32(off::DATA_START) as usize; - if ds >= HEADER_LEN && ds < bytes.len() { - ds - } else { - HEADER_LEN - } - }; - let data_length = h.u64(off::DATA_LENGTH) as usize; - - // Sample width (f32/f64) comes from the data-section byte budget, requiring an - // exact f32 or f64 fit rather than an ambiguous type nibble or a size guess. - let total_samples = npoints - .checked_mul(components) - .ok_or_else(|| IoError::Unsupported("point count overflow".into()))?; - let avail = bytes.len().saturating_sub(data_start); - let budget = if data_length > 0 && data_length <= avail { - data_length - } else { - avail - }; - let sample = sample_format(budget, total_samples)?; - let stride = sample.size(); - - let need = data_start - .checked_add(total_samples * stride) - .ok_or_else(|| IoError::Unsupported("data section size overflow".into()))?; - if bytes.len() < need { - return Err(IoError::Truncated { - offset: data_start, - needed: total_samples * stride, - have: avail, - }); - } - - let d = Reader { - bytes, - endian: body_endian, - }; - // Real then imaginary channel, each padded to `npoints`; read only the real - // extent, but the imaginary channel still starts after the full padded block. - let real = d.read_reals(data_start, real_n, sample); - let imag = if components == 2 { - d.read_reals(data_start + npoints * stride, real_n, sample) - } else { - vec![0.0; real_n] - }; - // JEOL stores the FID with the opposite quadrature sense to a naive forward - // FFT; conjugating it here (negating the imaginary channel) makes a plain - // forward FFT downstream yield the correct ppm ordering. - let mut points: Vec = real - .into_iter() - .zip(imag) - .map(|(re, im)| Complex64::new(re, -im)) - .collect(); - - if !base_freq_mhz.is_finite() || base_freq_mhz <= 1.0 { - return Err(IoError::Unsupported(format!( - "invalid JEOL observe frequency {base_freq_mhz} MHz" - ))); - } - let observe_freq_mhz = base_freq_mhz; - // Sweep width = 1/dwell; the last FID point sits at (N-1)·dwell = acq_time. - let spectral_width_hz = match axis_unit { - AxisUnit::Second if axis_span.is_finite() && axis_span > 0.0 && real_n > 1 => { - (real_n as f64 - 1.0) / axis_span - } - AxisUnit::Hertz if axis_span.is_finite() && axis_span > 0.0 => axis_span, - AxisUnit::Ppm if axis_span.is_finite() && axis_span > 0.0 => axis_span * observe_freq_mhz, - _ => { - return Err(IoError::Unsupported(format!( - "invalid JEOL axis span from {raw_axis_start} to {raw_axis_stop}" - ))); - } - }; - let axis_midpoint = (axis_start + axis_stop) / 2.0; - let carrier_ppm = match axis_unit { - AxisUnit::Hertz => axis_midpoint / observe_freq_mhz, - AxisUnit::Ppm => axis_midpoint, - AxisUnit::Second => params.f64("X_OFFSET").unwrap_or(0.0), - }; - - // Stored processed spectra run from high to low ppm. PlotX's imported - // frequency representation is low to high, matching its generated axis. - if domain == Domain::Frequency && axis_stop < axis_start { - points.reverse(); - } - - let nucleus = params - .string("X_DOMAIN") - .map(|s| normalize_nucleus(&s)) - .unwrap_or_else(|| guess_nucleus(observe_freq_mhz)); - let solvent = params.string("SOLVENT").unwrap_or_default(); - let provenance = if solvent.is_empty() { - format!("{source} (JEOL Delta, {sample:?}, {real_n} pts)") - } else { - format!("{source} (JEOL Delta, {solvent}, {real_n} pts)") - }; - - Ok(NmrData { - points, - domain, - spectral_width_hz, - observe_freq_mhz, - carrier_ppm, - nucleus, - source: provenance, - group_delay: group_delay(¶ms), - }) -} - -// nD data is stored as square submatrix tiles of edge `TILE`. For a 2D dataset -// with a complex direct (F2) axis and a real indirect (F1) axis there are two -// planes — all F2-real tiles, then all F2-imag tiles. Within a plane the F1 -// tile-block is the outer loop and the F2 tile-block the inner, and each tile is -// row-major. The imaginary plane is negated relative to a forward-FFT -// convention, so the complex value is `re - i·im` (the same conjugation the 1D -// reader applies). -fn read_jdf_2d(bytes: &[u8], source: String, body_endian: Endian) -> Result { - let h = Reader { - bytes, - endian: Endian::Big, - }; - let axis_u32 = |base: usize, i: usize| h.u32(base + i * 4) as usize; - - let cols_pad = axis_u32(off::DATA_POINTS, 0); - let rows_pad = axis_u32(off::DATA_POINTS, 1); - if cols_pad == 0 || rows_pad == 0 { - return Err(IoError::Unsupported( - "header reports zero data points".into(), - )); - } - // Real (non-padding) extent; the offset-stop is the last valid index. - let cols_real = (axis_u32(off::DATA_OFFSET_STOP, 0) + 1).min(cols_pad); - let rows_real = (axis_u32(off::DATA_OFFSET_STOP, 1) + 1).min(rows_pad); - - // JEOL stores nD data in square submatrix tiles. True-2D acquisitions use a - // 32-point tile edge; pseudo-2D arrays with few increments (DOSY, T1/T2) use - // a 4-point tile. Both loop the indirect (F1) tile-block outer and the direct - // (F2) tile-block inner, row-major within each tile. - let tile = if cols_pad % TILE == 0 && rows_pad % TILE == 0 { - TILE - } else { - SMALL_TILE - }; - if cols_pad % tile != 0 || rows_pad % tile != 0 { - return Err(IoError::Unsupported(format!( - "2D data points ({cols_pad}×{rows_pad}) are not a multiple of the {tile}-point tile edge" - ))); - } - - let axis_kind = |i: usize| bytes[off::DATA_AXIS_TYPE + i]; - let f2_complex = matches!(axis_kind(0), AXIS_COMPLEX | AXIS_REAL_COMPLEX); - // A `Complex` (type 3) indirect axis is States-style hypercomplex: the F1 - // cosine and sine modulations are acquired separately and stored as their own - // sample planes, so the plane count doubles and the indirect FFT needs States - // recombination. A `Real_Complex` (type 4) indirect axis is already a single - // phase-modulated interferogram (one plane pair) recombined as plain Complex. - let f1_hypercomplex = axis_kind(1) == AXIS_COMPLEX; - let f2_planes = if f2_complex { 2 } else { 1 }; - let f1_planes = if f1_hypercomplex { 2 } else { 1 }; - let planes = f2_planes * f1_planes; - - let data_start = { - let ds = h.u32(off::DATA_START) as usize; - if ds >= HEADER_LEN && ds < bytes.len() { - ds - } else { - HEADER_LEN - } - }; - let data_length = h.u64(off::DATA_LENGTH) as usize; - - let total_samples = cols_pad - .checked_mul(rows_pad) - .and_then(|v| v.checked_mul(planes)) - .ok_or_else(|| IoError::Unsupported("2D point count overflow".into()))?; - let avail = bytes.len().saturating_sub(data_start); - let budget = if data_length > 0 && data_length <= avail { - data_length - } else { - avail - }; - let sample = sample_format(budget, total_samples)?; - let stride = sample.size(); - let need = data_start - .checked_add(total_samples * stride) - .ok_or_else(|| IoError::Unsupported("2D data section size overflow".into()))?; - if bytes.len() < need { - return Err(IoError::Truncated { - offset: data_start, - needed: total_samples * stride, - have: avail, - }); - } - - let d = Reader { - bytes, - endian: body_endian, - }; - let n_f2_blocks = cols_pad / tile; - let plane_len = rows_pad * cols_pad; - let sample_at = |plane: usize, row: usize, col: usize| -> f64 { - let block = (row / tile) * n_f2_blocks + (col / tile); - let idx = plane * plane_len + block * tile * tile + (row % tile) * tile + (col % tile); - d.sample(data_start + idx * stride, sample) - }; - // Section plane index for (F1 imaginary?, F2 imaginary?); F2 toggles fastest, - // matching the 2-plane (F2-only-complex) layout the 1D reader shares. JEOL's - // imaginary plane is negated relative to a forward-FFT convention. - let complex_at = |f1_imag: usize, row: usize, col: usize| -> Complex64 { - let re = sample_at(f1_imag * f2_planes, row, col); - let im = f2_complex.then(|| sample_at(f1_imag * f2_planes + 1, row, col)); - Complex64::new(re, -im.unwrap_or(0.0)) - }; - - // For a hypercomplex indirect axis, interleave each increment's cosine - // (F1-real) and sine (F1-imag) channel as consecutive rows so the indirect - // FFT's States recombination pairs the 2k / 2k+1 rows into one t1 point. - let f1_channels = if f1_hypercomplex { 2 } else { 1 }; - let mut data = Vec::with_capacity(f1_channels * rows_real * cols_real); - for row in 0..rows_real { - for f1_imag in 0..f1_channels { - for col in 0..cols_real { - data.push(complex_at(f1_imag, row, col)); - } - } - } - let stored_rows = f1_channels * rows_real; - let quad = if f1_hypercomplex { - QuadMode::States - } else { - QuadMode::Complex - }; - - let params = Params::parse(bytes, off::PARAM_LIST, body_endian); - let acq_time = - |i: usize| (h.f64(off::DATA_AXIS_STOP + i * 8) - h.f64(off::DATA_AXIS_START + i * 8)).abs(); - let base_freq = |i: usize| { - let f = h.f64(off::BASE_FREQ + i * 8); - if f.is_finite() && f > 1.0 { f } else { 400.0 } - }; - let sweep = |i: usize, real_n: usize| { - let acq = acq_time(i); - if acq.is_finite() && acq > 0.0 && real_n > 1 { - (real_n as f64 - 1.0) / acq - } else { - base_freq(i) * 20.0 - } - }; - // The indirect axis stores no usable t1 acquisition time (its `Data_Axis_Stop` - // is not the increment span), so the acq-time estimate collapses the F1 sweep. - // The `Y_SWEEP` parameter (SI Hz, scaler folded) is authoritative; fall back to - // the acq-time estimate only when it is absent. - let indirect_sweep = params - .si("Y_SWEEP") - .filter(|v| v.is_finite() && *v > 1.0) - .unwrap_or_else(|| sweep(1, rows_real)); - let direct = Dim { - spectral_width_hz: sweep(0, cols_real), - observe_freq_mhz: base_freq(0), - carrier_ppm: params.f64("X_OFFSET").unwrap_or(0.0), - nucleus: params - .string("X_DOMAIN") - .map(|s| normalize_nucleus(&s)) - .unwrap_or_else(|| guess_nucleus(base_freq(0))), - group_delay: group_delay(¶ms), - }; - let indirect = Dim { - spectral_width_hz: indirect_sweep, - observe_freq_mhz: base_freq(1), - carrier_ppm: params.f64("Y_OFFSET").unwrap_or(0.0), - nucleus: params - .string("Y_DOMAIN") - .map(|s| normalize_nucleus(&s)) - .unwrap_or_else(|| guess_nucleus(base_freq(1))), - group_delay: 0.0, - }; - - let experiment = experiment_name(¶ms).map(|name| name.to_ascii_lowercase()); - - let (pseudo_axis, diffusion) = extract_pseudo(bytes, ¶ms, &experiment, &direct, rows_real); - let nus = detect_nus(bytes, ¶ms, rows_real); - - Ok(NmrData2D { - data, - rows: stored_rows, - cols: cols_real, - domain: Domain::Time, - direct, - indirect, - quad, - indirect_conjugate: true, - experiment, - pseudo_axis, - diffusion, - nus, - source: format!("{source} (JEOL Delta 2D, {sample:?}, {cols_real}×{rows_real})"), - }) -} - -fn experiment_value(params: &Params) -> Option { - params - .string_ci("experiment") - .or_else(|| params.string_ci("content")) - .map(|value| value.trim().to_owned()) - .filter(|value| !value.is_empty()) -} - -fn experiment_name(params: &Params) -> Option { - let value = experiment_value(params)?; - let file_name = value.rsplit(['/', '\\']).next().unwrap_or(&value).trim(); - let name = file_name - .rsplit_once('.') - .filter(|(_, extension)| extension.eq_ignore_ascii_case("jxp")) - .map_or(file_name, |(stem, _)| stem) - .trim(); - (!name.is_empty()).then(|| name.to_owned()) -} - -/// Recover the pseudo-2D indirect ruler and (for DOSY) the diffusion-encoding -/// parameters. The ruler comes from the embedded experiment text; diffusion -/// scalars come from the SI-normalized parameter list. -fn extract_pseudo( - bytes: &[u8], - params: &Params, - experiment: &Option, - direct: &Dim, - rows: usize, -) -> (Option, Option) { - let axis = scan_embedded_axis(bytes).map(|(name, mut values, unit, source)| { - // Trust the stored row count over a ramp that rounded to a different length. - if values.len() > rows && rows > 0 { - values.truncate(rows); - } - PseudoAxis { - kind: kind_for_unit(&unit), - name, - values, - unit, - source, - } - }); - - let hint = experiment.as_deref().unwrap_or(""); - let looks_dosy = axis - .as_ref() - .map(|a| a.kind == PseudoKind::Gradient) - .unwrap_or(false) - || ["dosy", "diffusion", "bpp", "ste", "led", "oneshot"] - .iter() - .any(|k| hint.contains(k)); - - let diffusion = if looks_dosy { - let gamma = gyromagnetic_ratio(&direct.nucleus).unwrap_or(2.675_222_005e8); - let delta = params.si("delta").unwrap_or(0.0); - let big_delta = params - .si("diffusion_time") - .or_else(|| params.si("delta_large")) - .unwrap_or(0.0); - let tau = params.si("tau").unwrap_or(0.0); - let shape_factor = gradient_shape_factor( - params - .string_ci("grad_shape") - .as_deref() - .unwrap_or("SQUARE"), - ); - (delta > 0.0 && big_delta > 0.0).then_some(DiffusionMeta { - gamma, - delta, - big_delta, - tau, - shape_factor, - }) - } else { - None - }; - - (axis, diffusion) -} - -/// Recover non-uniform-sampling metadata from the parameter list. Present only -/// when `sampling` reports a NUS scheme; the acquired increment count is the -/// stored real row count and the nominal grid is inferred from the sampling -/// rate. Recent Delta files also serialize `Y_NUSLIST` as a big-endian integer -/// array near the file tail; use it when its size and bounds agree with the -/// acquisition, otherwise leave the schedule for the user to supply. -fn normalize_nucleus(domain: &str) -> String { - match domain.trim().to_ascii_lowercase().as_str() { - "proton" => "1H".into(), - "carbon13" | "carbon" => "13C".into(), - "phosphorus31" | "phosphorus" => "31P".into(), - "fluorine19" | "fluorine" => "19F".into(), - "nitrogen15" | "nitrogen" => "15N".into(), - other if !other.is_empty() => domain.trim().to_string(), - _ => "X".into(), - } -} - -fn guess_nucleus(mhz: f64) -> String { - if mhz > 300.0 { - "1H".into() - } else if mhz > 90.0 { - "13C".into() - } else { - "X".into() - } -} - -/// Stored sample width from the data-section byte budget, requiring an exact f32/f64 -/// fit — a size matching neither is reported, not guessed (else silent garbage). -fn sample_format(budget: usize, total_samples: usize) -> Result { - if total_samples == 0 { - return Err(IoError::Unsupported( - "header reports zero data points".into(), - )); - } - if budget == 8 * total_samples { - Ok(SampleFmt::F64) - } else if budget == 4 * total_samples { - Ok(SampleFmt::F32) - } else { - Err(IoError::Unsupported(format!( - "data section of {budget} bytes fits neither f32 ({}) nor f64 ({}) for {total_samples} samples", - 4 * total_samples, - 8 * total_samples - ))) - } -} - -#[derive(Debug, Clone, Copy)] -enum SampleFmt { - F32, - F64, -} - -impl SampleFmt { - #[inline] - fn size(self) -> usize { - match self { - SampleFmt::F32 => 4, - SampleFmt::F64 => 8, - } - } -} - -#[derive(Debug, Clone, Copy)] -enum Endian { - Big, - Little, -} - -struct Reader<'a> { - bytes: &'a [u8], - endian: Endian, -} - -impl Reader<'_> { - fn u32(&self, at: usize) -> u32 { - let b: [u8; 4] = self.bytes[at..at + 4].try_into().unwrap(); - match self.endian { - Endian::Big => u32::from_be_bytes(b), - Endian::Little => u32::from_le_bytes(b), - } - } - - fn u64(&self, at: usize) -> u64 { - let b: [u8; 8] = self.bytes[at..at + 8].try_into().unwrap(); - match self.endian { - Endian::Big => u64::from_be_bytes(b), - Endian::Little => u64::from_le_bytes(b), - } - } - - fn f64(&self, at: usize) -> f64 { - let b: [u8; 8] = self.bytes[at..at + 8].try_into().unwrap(); - match self.endian { - Endian::Big => f64::from_be_bytes(b), - Endian::Little => f64::from_le_bytes(b), - } - } - - fn f32(&self, at: usize) -> f32 { - let b: [u8; 4] = self.bytes[at..at + 4].try_into().unwrap(); - match self.endian { - Endian::Big => f32::from_be_bytes(b), - Endian::Little => f32::from_le_bytes(b), - } - } - - #[inline] - fn sample(&self, at: usize, fmt: SampleFmt) -> f64 { - match fmt { - SampleFmt::F32 => self.f32(at) as f64, - SampleFmt::F64 => self.f64(at), - } - } - - fn read_reals(&self, at: usize, n: usize, fmt: SampleFmt) -> Vec { - (0..n) - .map(|i| match fmt { - SampleFmt::F32 => self.f32(at + i * 4) as f64, - SampleFmt::F64 => self.f64(at + i * 8), - }) - .collect() - } -} - -#[cfg(test)] -mod tests; diff --git a/crates/io/src/jeol/filter.rs b/crates/io/src/jeol/filter.rs deleted file mode 100644 index 06172c0e..00000000 --- a/crates/io/src/jeol/filter.rs +++ /dev/null @@ -1,51 +0,0 @@ -//! JEOL Delta digital-filter (oversampling FIR decimation) group delay. - -use super::Params; - -// Group delay in final sample points, present on the direct axis whenever -// `DIGITAL_FILTER = TRUE`. The filter is a cascade of symmetric FIR stages: -// `orders` is the stage count followed by each stage's tap count (`"2 41 74"` → -// two stages of 41 and 74 taps) and `factors` the matching per-stage decimation -// (`"6 2"`). A symmetric FIR of `M` taps delays by `(M-1)/2` samples at its own -// input rate; referred to the fully-decimated output rate that is scaled by the -// decimation accumulated before the stage, so the total is -// `Σ (M_k-1)/2 · D_{k-1} / D_total`. Returns 0.0 when the filter is off or the -// parameters are missing/unparsable, leaving the FID untouched. -pub(super) fn group_delay(params: &Params) -> f64 { - let enabled = params - .string_ci("DIGITAL_FILTER") - .map(|s| s.trim().eq_ignore_ascii_case("true")) - .unwrap_or(false); - if !enabled { - return 0.0; - } - let ints = |name: &str| -> Vec { - params - .string_ci(name) - .map(|s| { - s.split_whitespace() - .filter_map(|t| t.parse().ok()) - .collect() - }) - .unwrap_or_default() - }; - let orders = ints("orders"); - let factors = ints("factors"); - let taps = orders.split_first().map(|(_, rest)| rest).unwrap_or(&[]); - let stages = taps.len().min(factors.len()); - if stages == 0 { - return 0.0; - } - let total_decim: f64 = factors[..stages].iter().product(); - if total_decim <= 0.0 { - return 0.0; - } - let mut delay = 0.0; - let mut cumulative = 1.0; // decimation accumulated before the current stage - for k in 0..stages { - delay += (taps[k] - 1.0) / 2.0 * cumulative; - cumulative *= factors[k]; - } - let g = delay / total_decim; - if g.is_finite() && g >= 0.0 { g } else { 0.0 } -} diff --git a/crates/io/src/jeol/nus.rs b/crates/io/src/jeol/nus.rs deleted file mode 100644 index bc9fdf0e..00000000 --- a/crates/io/src/jeol/nus.rs +++ /dev/null @@ -1,120 +0,0 @@ -//! JEOL non-uniform-sampling metadata and serialized schedule extraction. - -use super::Params; -use crate::NusMeta; - -pub(super) fn detect_nus(bytes: &[u8], params: &Params, acquired: usize) -> Option { - let sampling = params.string_ci("sampling")?; - if !sampling.trim().to_ascii_uppercase().starts_with("NUS") { - return None; - } - // `sampling_rate` is a percentage (25 → 0.25). Fall back to a 1:1 grid. - let rate = params - .f64("sampling_rate") - .or_else(|| params.si("sampling_rate")) - .filter(|r| *r > 0.0 && *r <= 100.0) - .map(|r| r / 100.0) - .unwrap_or(1.0); - let grid = params - .f64("Y_ORIG_POINTS") - .or_else(|| params.si("Y_ORIG_POINTS")) - .filter(|v| v.is_finite() && *v >= acquired as f64) - .map(|v| v.round() as usize) - .unwrap_or_else(|| ((acquired as f64 / rate).round() as usize).max(acquired)); - let idx_base = params - .f64("nuslist_idx_base") - .or_else(|| params.si("nuslist_idx_base")) - .map(|v| v as usize) - .unwrap_or(1); - let mode = params - .string_ci("nus_mode") - .or_else(|| params.string_ci("auto_nus_mode")) - .unwrap_or_else(|| "unknown".to_string()); - let echo_antiecho = params - .string_ci("pn_type") - .map(|s| s.trim().eq_ignore_ascii_case("y")) - .unwrap_or(false); - let schedule = extract_nuslist(bytes, b"Y_NUSLIST").and_then(|raw| { - if raw.len() != acquired { - return None; - } - raw.into_iter() - .map(|value| value.checked_sub(idx_base).filter(|value| *value < grid)) - .collect::>>() - .filter(|values| { - let mut sorted = values.clone(); - sorted.sort_unstable(); - sorted.dedup(); - sorted.len() == values.len() - }) - }); - Some(NusMeta { - grid, - acquired, - idx_base, - mode, - echo_antiecho, - schedule, - }) -} - -/// Find a named integer-array parameter in Delta's serialized parameter tail. -/// The fixed parameter table contains the name too, so candidates are accepted -/// only when the preceding string header and following typed array both match. -pub(super) fn extract_nuslist(bytes: &[u8], wanted: &[u8]) -> Option> { - const STRING_TAG: u32 = 0x271d; - const INTEGER_TAG: u32 = 0x271a; - const CONTAINER_TAG: u32 = 0x2b2a; - - let be_u32 = |at: usize| -> Option { - let chunk: [u8; 4] = bytes.get(at..at.checked_add(4)?)?.try_into().ok()?; - Some(u32::from_be_bytes(chunk)) - }; - - for (name_at, name) in bytes.windows(wanted.len()).enumerate() { - if !name.eq_ignore_ascii_case(wanted) || name_at < 8 { - continue; - } - if be_u32(name_at - 8) != Some(STRING_TAG) - || be_u32(name_at - 4) != u32::try_from(wanted.len()).ok() - { - continue; - } - - let after_name = name_at + wanted.len(); - let container_at = (after_name..after_name.saturating_add(4)) - .take_while(|at| { - bytes - .get(after_name..*at) - .is_some_and(|pad| pad.iter().all(|b| *b == 0)) - }) - .find(|at| be_u32(*at) == Some(CONTAINER_TAG)); - let Some(container_at) = container_at else { - continue; - }; - let count = be_u32(container_at + 4).and_then(|v| usize::try_from(v).ok())?; - let array_bytes = count.checked_mul(12)?; - let mut pos = container_at.checked_add(8)?; - if pos - .checked_add(array_bytes) - .is_none_or(|end| end > bytes.len()) - { - continue; - } - - let mut values = Vec::with_capacity(count); - for _ in 0..count { - if be_u32(pos) != Some(INTEGER_TAG) || be_u32(pos + 4) != Some(1) { - values.clear(); - break; - } - let value = be_u32(pos + 8).and_then(|v| usize::try_from(v).ok())?; - values.push(value); - pos += 12; - } - if values.len() == count { - return Some(values); - } - } - None -} diff --git a/crates/io/src/jeol/params.rs b/crates/io/src/jeol/params.rs deleted file mode 100644 index cd5d846d..00000000 --- a/crates/io/src/jeol/params.rs +++ /dev/null @@ -1,106 +0,0 @@ -use super::{Endian, Reader}; -use crate::jeol::ruler::{ascii_trim, prefix_exponent}; -use std::collections::HashMap; - -pub(super) struct Params { - pub(super) f64s: HashMap, - /// SI-normalized numeric values (raw value with its scaler prefix folded in). - si: HashMap, - pub(super) strings: HashMap, -} - -impl Params { - const SCALER: usize = 0x06; - const VALUE: usize = 0x10; - const VALUE_TYPE: usize = 0x20; - const NAME: usize = 0x24; - const NAME_LEN: usize = 28; - - pub(super) fn empty() -> Self { - Self { - f64s: HashMap::new(), - si: HashMap::new(), - strings: HashMap::new(), - } - } - - // List header at `at` (body endianness): record_size u32, low_index u32, - // high_index u32, total_size u32; then fixed-size records. - pub(super) fn parse(bytes: &[u8], at: usize, endian: Endian) -> Self { - let r = Reader { bytes, endian }; - if at + 16 > bytes.len() { - return Self::empty(); - } - let rec_size = r.u32(at) as usize; - let high = r.u32(at + 8) as usize; - if !(Self::NAME + Self::NAME_LEN..=4096).contains(&rec_size) { - return Self::empty(); - } - let count = high.saturating_add(1).min(4096); - let base = at + 16; - let mut out = Self::empty(); - for i in 0..count { - let rec = base + i * rec_size; - if rec + rec_size > bytes.len() { - break; - } - let name = ascii_trim(&bytes[rec + Self::NAME..rec + Self::NAME + Self::NAME_LEN]); - if name.is_empty() { - continue; - } - match r.u32(rec + Self::VALUE_TYPE) { - 2 => { - let raw = r.f64(rec + Self::VALUE); - let si = raw * 10f64.powi(prefix_exponent(bytes[rec + Self::SCALER])); - out.si.insert(name.clone(), si); - out.f64s.insert(name, raw); - } - 0 => { - let value = ascii_trim(&bytes[rec + Self::VALUE..rec + Self::VALUE + 16]); - if !value.is_empty() { - out.strings.insert(name, value); - } - } - _ => {} - } - } - out - } - - pub(super) fn f64(&self, name: &str) -> Option { - self.f64s - .get(name) - .copied() - .filter(|value| value.is_finite()) - } - - pub(super) fn string(&self, name: &str) -> Option { - self.strings.get(name).cloned() - } - - pub(super) fn si(&self, name: &str) -> Option { - self.numeric_ci(name, &self.si) - } - - fn numeric_ci(&self, name: &str, values: &HashMap) -> Option { - values - .get(name) - .copied() - .or_else(|| { - values - .iter() - .find(|(key, _)| key.eq_ignore_ascii_case(name)) - .map(|(_, value)| *value) - }) - .filter(|value| value.is_finite()) - } - - pub(super) fn string_ci(&self, name: &str) -> Option { - self.strings.get(name).cloned().or_else(|| { - self.strings - .iter() - .find(|(key, _)| key.eq_ignore_ascii_case(name)) - .map(|(_, value)| value.clone()) - }) - } -} diff --git a/crates/io/src/jeol/ruler.rs b/crates/io/src/jeol/ruler.rs deleted file mode 100644 index 26e3b29f..00000000 --- a/crates/io/src/jeol/ruler.rs +++ /dev/null @@ -1,127 +0,0 @@ -//! Decoding of JEOL scaler bytes, display units, and the embedded arrayed-axis -//! (pseudo-2D ruler) text. - -use crate::{AxisSource, PseudoKind}; - -/// Decode a JEOL scaler byte to the base-10 exponent it applies to a stored -/// value. The high nibble is a signed SI-prefix index: 0→10⁰, 1→milli, 2→micro, -/// 3→nano, …, and 0xF→kilo, 0xE→mega for the positive prefixes. -pub(super) fn prefix_exponent(scaler: u8) -> i32 { - let n = (scaler >> 4) as i32; - if n < 8 { -3 * n } else { -3 * (n - 16) } -} - -/// Convert a `value[unit]` display unit to an SI multiplier, e.g. `ms → 1e-3`, -/// `mT/m → 1e-3`, `G/cm → 1e-2`. Unrecognised units map to 1.0. -fn unit_to_si(unit: &str) -> f64 { - match unit.trim() { - "s" => 1.0, - "ms" => 1e-3, - "us" | "µs" => 1e-6, - "ns" => 1e-9, - "T/m" => 1.0, - "mT/m" => 1e-3, - "G/cm" => 1e-2, // 1 gauss/cm = 1e-4 T / 1e-2 m = 1e-2 T/m - "G/mm" => 0.1, - _ => 1.0, - } -} - -pub(super) fn kind_for_unit(unit: &str) -> PseudoKind { - match unit.trim() { - "s" | "ms" | "us" | "µs" | "ns" => PseudoKind::Delay, - "T/m" | "mT/m" | "G/cm" | "G/mm" => PseudoKind::Gradient, - _ => PseudoKind::Generic, - } -} - -/// Parse a single `123.4[unit]` token into `(value, unit)`; the value is left in -/// its display unit (the caller applies `unit_to_si`). -fn parse_quantity_token(tok: &str) -> Option<(f64, String)> { - let tok = tok.trim(); - let open = tok.find('[')?; - let close = tok.find(']')?; - if close < open { - return None; - } - let value: f64 = tok[..open].trim().parse().ok()?; - let unit = tok[open + 1..close].trim().to_string(); - Some((value, unit)) -} - -/// Scan the embedded experiment text for the arrayed indirect axis. JEOL writes -/// it as `name => y_acq {v1[u], v2[u], …}` (explicit list) or -/// `name => y_acq start[u]..stop[u] : step[u]` (linear ramp). Returns the SI -/// values, the (display) unit, and which form was found. -pub(super) fn scan_embedded_axis(bytes: &[u8]) -> Option<(String, Vec, String, AxisSource)> { - // Work over a lossy-ASCII view; the experiment text is plain ASCII. - let text = String::from_utf8_lossy(bytes); - let marker = "y_acq"; - let mut search_from = 0; - while let Some(rel) = text[search_from..].find(marker) { - let at = search_from + rel; - search_from = at + marker.len(); - - // Recover the parameter name: the identifier just before "=>"/"=?". - let name = text[..at] - .rfind(['>', '?']) - .map(|arrow| text[..arrow].trim_end_matches(['=', ' ']).to_string()) - .and_then(|s| s.rsplit([' ', '\n', '\t', ';']).next().map(str::to_string)) - .filter(|s| !s.is_empty()) - .unwrap_or_else(|| "increment".to_string()); - - let rest = text[at + marker.len()..].trim_start(); - - // Explicit list form: { … }. - if let Some(stripped) = rest.strip_prefix('{') - && let Some(end) = stripped.find('}') - { - let mut unit = String::new(); - let values: Vec = stripped[..end] - .split(',') - .filter_map(|tok| { - let (v, u) = parse_quantity_token(tok)?; - if unit.is_empty() { - unit = u.clone(); - } - Some(v * unit_to_si(&u)) - }) - .collect(); - if values.len() >= 2 { - return Some((name, values, unit, AxisSource::EmbeddedList)); - } - } - - // Ramp form: start[u]..stop[u] : step[u]. Each token carries its own - // unit (start may be mT/m while stop is T/m), so convert independently. - let ramp = rest.split(['\n', ',']).next().unwrap_or(rest); - if let Some((lo_s, hi_step)) = ramp.split_once("..") { - let (hi_s, step_s) = hi_step.split_once(':').unwrap_or((hi_step, "")); - if let (Some((lo, lu)), Some((hi, hu)), Some((step, su))) = ( - parse_quantity_token(lo_s), - parse_quantity_token(hi_s), - parse_quantity_token(step_s), - ) { - let lo_si = lo * unit_to_si(&lu); - let hi_si = hi * unit_to_si(&hu); - let step_si = (step * unit_to_si(&su)).abs(); - if step_si > 0.0 && hi_si.is_finite() && lo_si.is_finite() { - let mut values = Vec::new(); - let n = ((hi_si - lo_si) / step_si).round() as i64; - for i in 0..=n.max(0) { - values.push(lo_si + step_si * i as f64); - } - if values.len() >= 2 { - return Some((name, values, lu, AxisSource::EmbeddedRamp)); - } - } - } - } - } - None -} - -pub(super) fn ascii_trim(raw: &[u8]) -> String { - let end = raw.iter().position(|&b| b == 0).unwrap_or(raw.len()); - String::from_utf8_lossy(&raw[..end]).trim().to_string() -} diff --git a/crates/io/src/jeol/tests.rs b/crates/io/src/jeol/tests.rs deleted file mode 100644 index 02e6a8a3..00000000 --- a/crates/io/src/jeol/tests.rs +++ /dev/null @@ -1,718 +0,0 @@ -use super::*; -use crate::AxisSource; -use crate::jeol::ruler::prefix_exponent; - -#[test] -fn prefix_exponent_ladder() { - assert_eq!(prefix_exponent(0x01), 0); // none - assert_eq!(prefix_exponent(0x11), -3); // milli - assert_eq!(prefix_exponent(0x21), -6); // micro - assert_eq!(prefix_exponent(0x31), -9); // nano - assert_eq!(prefix_exponent(0xF1), 3); // kilo - assert_eq!(prefix_exponent(0xE1), 6); // mega -} - -#[test] -fn scans_embedded_list_ruler() { - let text = b"comment_7 => \"*** Pulse Delay ***\";\n tau_interval \ - => y_acq {1[ms], 1.7644[ms], 3.11312[ms], 5[s]}, help \"arrayed list\";"; - let (name, values, unit, source) = scan_embedded_axis(text).expect("axis"); - assert_eq!(name, "tau_interval"); - assert_eq!(unit, "ms"); - assert_eq!(source, AxisSource::EmbeddedList); - assert!((values[0] - 0.001).abs() < 1e-12); - assert!((values[1] - 0.0017644).abs() < 1e-12); - assert!((values[3] - 5.0).abs() < 1e-12); // 5[s] converted from seconds - assert_eq!(kind_for_unit(&unit), PseudoKind::Delay); -} - -#[test] -fn scans_embedded_ramp_ruler() { - let text = - b" g => y_acq 20[mT/m]..0.28[T/m] : 17.33333[mT/m], help \"g\";"; - let (name, values, unit, source) = scan_embedded_axis(text).expect("axis"); - assert_eq!(name, "g"); - assert_eq!(source, AxisSource::EmbeddedRamp); - assert_eq!(kind_for_unit(&unit), PseudoKind::Gradient); - assert_eq!(values.len(), 16); - assert!((values[0] - 0.02).abs() < 1e-9); - assert!((values[15] - 0.28).abs() < 1e-6); -} - -fn params_with(strings: &[(&str, &str)]) -> Params { - let mut p = Params::empty(); - for (k, v) in strings { - p.strings.insert((*k).to_string(), (*v).to_string()); - } - p -} - -#[test] -fn experiment_name_is_cleaned_for_acquisition_identity() { - let params = params_with(&[( - "experiment", - r"C:\Program Files\JEOL\experiments\13c_eb_sn.jxp", - )]); - assert_eq!(experiment_name(¶ms).as_deref(), Some("13c_eb_sn")); - - let params = params_with(&[("CONTENT", "cosy.JXP")]); - assert_eq!(experiment_name(¶ms).as_deref(), Some("cosy")); -} - -#[test] -fn group_delay_from_fir_cascade() { - // orders = " ", factors = per-stage decimation. - // Delay (final points) = Σ (taps_k-1)/2 · D_{k-1} / D_total. - // "6 2" / "41 74": (20·1 + 36.5·6)/12 = 19.9166… - let g = group_delay(¶ms_with(&[ - ("DIGITAL_FILTER", "TRUE"), - ("orders", "2 41 74"), - ("factors", "6 2"), - ])); - assert!((g - 239.0 / 12.0).abs() < 1e-9, "got {g}"); - - // "2 2" / "15 73": (7·1 + 36·2)/4 = 19.75. - let g = group_delay(¶ms_with(&[ - ("DIGITAL_FILTER", "TRUE"), - ("orders", "2 15 73"), - ("factors", "2 2"), - ])); - assert!((g - 19.75).abs() < 1e-9, "got {g}"); -} - -#[test] -fn group_delay_gated_and_guarded() { - // Filter off → no correction even with orders/factors present. - let g = group_delay(¶ms_with(&[ - ("DIGITAL_FILTER", "FALSE"), - ("orders", "2 41 74"), - ("factors", "6 2"), - ])); - assert_eq!(g, 0.0); - - // Filter flag absent → no correction. - assert_eq!(group_delay(¶ms_with(&[("orders", "2 41 74")])), 0.0); - - // Filter on but parameters missing → no correction, no panic. - assert_eq!( - group_delay(¶ms_with(&[("DIGITAL_FILTER", "TRUE")])), - 0.0 - ); -} - -#[test] -fn rejects_bad_magic() { - let buf = vec![0u8; HEADER_LEN + 16]; - let err = read_jdf_bytes(&buf, "x".into()).unwrap_err(); - assert!(matches!(err, IoError::BadMagic)); -} - -#[test] -fn rejects_truncated_header() { - let buf = vec![0u8; 100]; - let err = read_jdf_bytes(&buf, "x".into()).unwrap_err(); - assert!(matches!(err, IoError::Truncated { .. })); -} - -#[test] -fn round_trips_a_hand_built_1d_le_file() { - let npoints = 4usize; - let rec_size = 64usize; - let param_hdr = HEADER_LEN; - let param_recs = param_hdr + 16; - let data_start = param_recs + rec_size; - let data_len = npoints * 8 * 2; - let mut buf = vec![0u8; data_start + data_len]; - - buf[..8].copy_from_slice(MAGIC); - buf[off::ENDIAN] = 1; // little-endian body - buf[off::DATA_DIMENSION_NUMBER] = 1; - buf[off::DATA_AXIS_TYPE] = AXIS_COMPLEX; - buf[off::DATA_AXIS_UNITS + 1] = UNIT_SECOND; - buf[off::DATA_POINTS..off::DATA_POINTS + 4].copy_from_slice(&(npoints as u32).to_be_bytes()); - buf[off::BASE_FREQ..off::BASE_FREQ + 8].copy_from_slice(&600.17f64.to_be_bytes()); - // SW = (npoints-1)/acq = 1000 Hz. - let acq = (npoints as f64 - 1.0) / 1000.0; - buf[off::DATA_AXIS_START..off::DATA_AXIS_START + 8].copy_from_slice(&0.0f64.to_be_bytes()); - buf[off::DATA_AXIS_STOP..off::DATA_AXIS_STOP + 8].copy_from_slice(&acq.to_be_bytes()); - buf[off::DATA_START..off::DATA_START + 4].copy_from_slice(&(data_start as u32).to_be_bytes()); - buf[off::DATA_LENGTH..off::DATA_LENGTH + 8].copy_from_slice(&(data_len as u64).to_be_bytes()); - - buf[param_hdr..param_hdr + 4].copy_from_slice(&(rec_size as u32).to_le_bytes()); - buf[param_hdr + 8..param_hdr + 12].copy_from_slice(&0u32.to_le_bytes()); - buf[param_recs + 0x10..param_recs + 0x18].copy_from_slice(&4.7f64.to_le_bytes()); - buf[param_recs + 0x20..param_recs + 0x24].copy_from_slice(&2u32.to_le_bytes()); - let name = b"X_OFFSET"; - buf[param_recs + 0x24..param_recs + 0x24 + name.len()].copy_from_slice(name); - - for i in 0..npoints { - let ro = data_start + i * 8; - let io = data_start + npoints * 8 + i * 8; - buf[ro..ro + 8].copy_from_slice(&((i as f64) + 1.0).to_le_bytes()); - buf[io..io + 8].copy_from_slice(&((i as f64) + 5.0).to_le_bytes()); - } - - let data = match read_jdf_bytes(&buf, "test.jdf".into()).unwrap() { - Acquisition::D1(d) => d, - Acquisition::D2(_) => panic!("expected 1D"), - Acquisition::Electrophysiology(_) => panic!("expected NMR"), - Acquisition::Afm(_) => panic!("expected NMR"), - Acquisition::MassSpec(_) | Acquisition::Xrd(_) => panic!("expected NMR"), - Acquisition::Xps(_) => panic!("expected NMR"), - }; - assert_eq!(data.len(), 4); - // FID conjugated on read (imaginary channel negated). - assert_eq!(data.points[0], Complex64::new(1.0, -5.0)); - assert_eq!(data.points[3], Complex64::new(4.0, -8.0)); - assert!((data.observe_freq_mhz - 600.17).abs() < 1e-6); - assert!((data.spectral_width_hz - 1000.0).abs() < 1e-6); - assert!( - (data.carrier_ppm - 4.7).abs() < 1e-9, - "carrier from X_OFFSET" - ); - assert_eq!(data.nucleus, "1H"); -} - -#[test] -fn recognizes_a_processed_1d_axis_without_applying_fft() { - let data = processed_1d(0x01, UNIT_PPM, -10.0, 10.0, 100.0); - - assert_eq!(data.domain, Domain::Frequency); - assert_eq!(data.spectral_width_hz, 2000.0); - assert_eq!(data.carrier_ppm, 0.0); - assert_eq!( - data.points.iter().map(|point| point.re).collect::>(), - vec![1.0, 2.0, 3.0, 4.0] - ); -} - -#[test] -fn normalizes_a_descending_processed_axis_and_its_samples() { - let data = processed_1d(0x01, UNIT_PPM, 10.0, -10.0, 100.0); - - assert_eq!(data.spectral_width_hz, 2000.0); - assert_eq!(data.carrier_ppm, 0.0); - assert_eq!( - data.points.iter().map(|point| point.re).collect::>(), - vec![4.0, 3.0, 2.0, 1.0] - ); -} - -#[test] -fn processed_axis_uses_all_stored_points_not_the_time_domain_offset_stop() { - let mut buf = processed_1d_bytes(0x01, UNIT_PPM, -10.0, 10.0, 100.0); - buf[off::DATA_OFFSET_STOP..off::DATA_OFFSET_STOP + 4].copy_from_slice(&2u32.to_be_bytes()); - - let Acquisition::D1(data) = read_jdf_bytes(&buf, "processed-window.jdf".into()).unwrap() else { - panic!("expected 1D NMR"); - }; - assert_eq!(data.len(), 4); - assert_eq!(data.spectral_width_hz, 2000.0); -} - -#[test] -fn converts_hertz_axis_to_frequency_metadata() { - let data = processed_1d(0x01, UNIT_HERTZ, 1000.0, 3000.0, 400.0); - - assert_eq!(data.domain, Domain::Frequency); - assert_eq!(data.spectral_width_hz, 2000.0); - assert_eq!(data.carrier_ppm, 5.0); -} - -#[test] -fn applies_axis_prefix_before_converting_frequency_metadata() { - let data = processed_1d(0xF1, UNIT_HERTZ, -1.0, 3.0, 400.0); - - assert_eq!(data.spectral_width_hz, 4000.0); - assert_eq!(data.carrier_ppm, 2.5); -} - -#[test] -fn applies_axis_prefix_to_fid_acquisition_time() { - let data = processed_1d(0x11, UNIT_SECOND, 0.0, 3.0, 400.0); - - assert_eq!(data.domain, Domain::Time); - assert!((data.spectral_width_hz - 1000.0).abs() < 1e-9); -} - -#[test] -fn rejects_an_unknown_axis_unit() { - let mut buf = vec![0u8; HEADER_LEN + 16]; - buf[..8].copy_from_slice(MAGIC); - buf[off::DATA_DIMENSION_NUMBER] = 1; - buf[off::DATA_AXIS_TYPE] = AXIS_COMPLEX; - buf[off::DATA_POINTS..off::DATA_POINTS + 4].copy_from_slice(&1u32.to_be_bytes()); - - let err = read_jdf_bytes(&buf, "unknown-unit.jdf".into()).unwrap_err(); - assert!( - matches!(err, IoError::Unsupported(ref message) if message.contains("unknown JEOL unit")), - "got {err:?}" - ); -} - -#[test] -fn rejects_invalid_frequency_metadata_instead_of_inventing_defaults() { - let mut buf = processed_1d_bytes(0x01, UNIT_HERTZ, 1.0, 1.0, 0.0); - let err = read_jdf_bytes(&buf, "invalid-frequency.jdf".into()).unwrap_err(); - assert!( - matches!(err, IoError::Unsupported(ref message) if message.contains("observe frequency")), - "got {err:?}" - ); - - buf[off::BASE_FREQ..off::BASE_FREQ + 8].copy_from_slice(&400.0f64.to_be_bytes()); - let err = read_jdf_bytes(&buf, "invalid-span.jdf".into()).unwrap_err(); - assert!( - matches!(err, IoError::Unsupported(ref message) if message.contains("axis span")), - "got {err:?}" - ); -} - -fn processed_1d( - unit_scaler: u8, - unit: u8, - axis_start: f64, - axis_stop: f64, - observe_freq_mhz: f64, -) -> NmrData { - let buf = processed_1d_bytes(unit_scaler, unit, axis_start, axis_stop, observe_freq_mhz); - let Acquisition::D1(data) = read_jdf_bytes(&buf, "axis.jdf".into()).unwrap() else { - panic!("expected 1D NMR"); - }; - data -} - -fn processed_1d_bytes( - unit_scaler: u8, - unit: u8, - axis_start: f64, - axis_stop: f64, - observe_freq_mhz: f64, -) -> Vec { - let npoints = 4usize; - let data_start = HEADER_LEN + 16; - let data_len = npoints * 8 * 2; - let mut buf = vec![0u8; data_start + data_len]; - - buf[..8].copy_from_slice(MAGIC); - buf[off::ENDIAN] = 1; - buf[off::DATA_DIMENSION_NUMBER] = 1; - buf[off::DATA_AXIS_TYPE] = AXIS_COMPLEX; - buf[off::DATA_AXIS_UNITS] = unit_scaler; - buf[off::DATA_AXIS_UNITS + 1] = unit; - buf[off::DATA_POINTS..off::DATA_POINTS + 4].copy_from_slice(&(npoints as u32).to_be_bytes()); - buf[off::DATA_OFFSET_STOP..off::DATA_OFFSET_STOP + 4] - .copy_from_slice(&((npoints - 1) as u32).to_be_bytes()); - buf[off::BASE_FREQ..off::BASE_FREQ + 8].copy_from_slice(&observe_freq_mhz.to_be_bytes()); - buf[off::DATA_AXIS_START..off::DATA_AXIS_START + 8].copy_from_slice(&axis_start.to_be_bytes()); - buf[off::DATA_AXIS_STOP..off::DATA_AXIS_STOP + 8].copy_from_slice(&axis_stop.to_be_bytes()); - buf[off::DATA_START..off::DATA_START + 4].copy_from_slice(&(data_start as u32).to_be_bytes()); - buf[off::DATA_LENGTH..off::DATA_LENGTH + 8].copy_from_slice(&(data_len as u64).to_be_bytes()); - - for i in 0..npoints { - let real = data_start + i * 8; - let imag = data_start + npoints * 8 + i * 8; - buf[real..real + 8].copy_from_slice(&(i as f64 + 1.0).to_le_bytes()); - buf[imag..imag + 8].copy_from_slice(&0.0f64.to_le_bytes()); - } - - buf -} - -#[test] -fn uses_real_point_count_over_padded_count_for_1d() { - // 8 padded points, only 4 real (DATA_OFFSET_STOP = 3). The FID must be - // truncated to 4 and the sweep width computed from the real count, not the - // padded one — otherwise every ppm is scaled by (8-1)/(4-1). - let npad = 8usize; - let nreal = 4usize; - let rec_size = 64usize; - let param_hdr = HEADER_LEN; - let param_recs = param_hdr + 16; - let data_start = param_recs + rec_size; - let data_len = npad * 8 * 2; // padded reals then padded imags, f64 - let mut buf = vec![0u8; data_start + data_len]; - - buf[..8].copy_from_slice(MAGIC); - buf[off::ENDIAN] = 1; - buf[off::DATA_DIMENSION_NUMBER] = 1; - buf[off::DATA_AXIS_TYPE] = AXIS_COMPLEX; - buf[off::DATA_AXIS_UNITS + 1] = UNIT_SECOND; - buf[off::DATA_POINTS..off::DATA_POINTS + 4].copy_from_slice(&(npad as u32).to_be_bytes()); - buf[off::DATA_OFFSET_STOP..off::DATA_OFFSET_STOP + 4] - .copy_from_slice(&((nreal - 1) as u32).to_be_bytes()); - buf[off::BASE_FREQ..off::BASE_FREQ + 8].copy_from_slice(&600.0f64.to_be_bytes()); - // acq time over the real count → SW = (nreal-1)/acq = 1000 Hz. - let acq = (nreal as f64 - 1.0) / 1000.0; - buf[off::DATA_AXIS_START..off::DATA_AXIS_START + 8].copy_from_slice(&0.0f64.to_be_bytes()); - buf[off::DATA_AXIS_STOP..off::DATA_AXIS_STOP + 8].copy_from_slice(&acq.to_be_bytes()); - buf[off::DATA_START..off::DATA_START + 4].copy_from_slice(&(data_start as u32).to_be_bytes()); - buf[off::DATA_LENGTH..off::DATA_LENGTH + 8].copy_from_slice(&(data_len as u64).to_be_bytes()); - - buf[param_hdr..param_hdr + 4].copy_from_slice(&(rec_size as u32).to_le_bytes()); - buf[param_hdr + 8..param_hdr + 12].copy_from_slice(&0u32.to_le_bytes()); - - for i in 0..npad { - // Real channel: 1..=4 real, then padding sentinels that must be dropped. - let ro = data_start + i * 8; - let io = data_start + npad * 8 + i * 8; - let re = if i < nreal { i as f64 + 1.0 } else { 999.0 }; - let im = if i < nreal { i as f64 + 5.0 } else { -999.0 }; - buf[ro..ro + 8].copy_from_slice(&re.to_le_bytes()); - buf[io..io + 8].copy_from_slice(&im.to_le_bytes()); - } - - let data = match read_jdf_bytes(&buf, "padded.jdf".into()).unwrap() { - Acquisition::D1(d) => d, - Acquisition::D2(_) => panic!("expected 1D"), - Acquisition::Electrophysiology(_) => panic!("expected NMR"), - Acquisition::Afm(_) => panic!("expected NMR"), - Acquisition::MassSpec(_) | Acquisition::Xrd(_) => panic!("expected NMR"), - Acquisition::Xps(_) => panic!("expected NMR"), - }; - assert_eq!(data.len(), nreal, "FID truncated to the real point count"); - assert_eq!(data.points[0], Complex64::new(1.0, -5.0)); - assert_eq!(data.points[3], Complex64::new(4.0, -8.0)); - assert!( - (data.spectral_width_hz - 1000.0).abs() < 1e-6, - "sweep width uses the real count, got {}", - data.spectral_width_hz - ); -} - -#[test] -fn rejects_ambiguous_sample_width() { - // A data section that is neither 4× nor 8× the sample count must error rather - // than silently pick a width and splice unrelated samples into garbage. - let npoints = 4usize; - let rec_size = 64usize; - let param_hdr = HEADER_LEN; - let param_recs = param_hdr + 16; - let data_start = param_recs + rec_size; - // components = 2 → total 8 samples; f32 wants 32 bytes, f64 wants 64. Give 48. - let data_len = 48usize; - let mut buf = vec![0u8; data_start + data_len]; - - buf[..8].copy_from_slice(MAGIC); - buf[off::ENDIAN] = 1; - buf[off::DATA_DIMENSION_NUMBER] = 1; - buf[off::DATA_AXIS_TYPE] = AXIS_COMPLEX; - buf[off::DATA_AXIS_UNITS + 1] = UNIT_SECOND; - buf[off::DATA_POINTS..off::DATA_POINTS + 4].copy_from_slice(&(npoints as u32).to_be_bytes()); - buf[off::DATA_START..off::DATA_START + 4].copy_from_slice(&(data_start as u32).to_be_bytes()); - buf[off::DATA_LENGTH..off::DATA_LENGTH + 8].copy_from_slice(&(data_len as u64).to_be_bytes()); - buf[param_hdr..param_hdr + 4].copy_from_slice(&(rec_size as u32).to_le_bytes()); - buf[param_hdr + 8..param_hdr + 12].copy_from_slice(&0u32.to_le_bytes()); - - let err = read_jdf_bytes(&buf, "ambiguous.jdf".into()).unwrap_err(); - assert!(matches!(err, IoError::Unsupported(_)), "got {err:?}"); -} - -fn write_param_record(buf: &mut [u8], rec: usize, name: &[u8], is_f64: bool, f: f64, s: &[u8]) { - if is_f64 { - buf[rec + 0x10..rec + 0x18].copy_from_slice(&f.to_le_bytes()); - buf[rec + 0x20..rec + 0x24].copy_from_slice(&2u32.to_le_bytes()); - } else { - buf[rec + 0x10..rec + 0x10 + s.len()].copy_from_slice(s); - buf[rec + 0x20..rec + 0x24].copy_from_slice(&0u32.to_le_bytes()); - } - buf[rec + 0x24..rec + 0x24 + name.len()].copy_from_slice(name); -} - -#[test] -fn de_tiles_a_hand_built_2d_across_tile_blocks() { - // 64×32 padded (two F2 tile-blocks), 34×2 real, complex X / real Y. - let (cols_pad, rows_pad) = (64usize, 32usize); - let (cols_real, rows_real) = (34usize, 2usize); - let planes = 2usize; - let rec_size = 64usize; - let param_hdr = HEADER_LEN; - let param_recs = param_hdr + 16; - let n_records = 4usize; - let data_start = param_recs + n_records * rec_size; - let data_len = cols_pad * rows_pad * planes * 8; - let mut buf = vec![0u8; data_start + data_len]; - - buf[..8].copy_from_slice(MAGIC); - buf[off::ENDIAN] = 1; // little-endian body - buf[off::DATA_DIMENSION_NUMBER] = 2; - buf[off::DATA_AXIS_TYPE] = AXIS_REAL_COMPLEX; - buf[off::DATA_AXIS_TYPE + 1] = AXIS_REAL_COMPLEX; - buf[off::DATA_POINTS..off::DATA_POINTS + 4].copy_from_slice(&(cols_pad as u32).to_be_bytes()); - buf[off::DATA_POINTS + 4..off::DATA_POINTS + 8] - .copy_from_slice(&(rows_pad as u32).to_be_bytes()); - buf[off::DATA_OFFSET_STOP..off::DATA_OFFSET_STOP + 4] - .copy_from_slice(&((cols_real - 1) as u32).to_be_bytes()); - buf[off::DATA_OFFSET_STOP + 4..off::DATA_OFFSET_STOP + 8] - .copy_from_slice(&((rows_real - 1) as u32).to_be_bytes()); - buf[off::BASE_FREQ..off::BASE_FREQ + 8].copy_from_slice(&600.0f64.to_be_bytes()); - buf[off::BASE_FREQ + 8..off::BASE_FREQ + 16].copy_from_slice(&150.0f64.to_be_bytes()); - buf[off::DATA_AXIS_STOP..off::DATA_AXIS_STOP + 8].copy_from_slice(&1e-3f64.to_be_bytes()); - buf[off::DATA_AXIS_STOP + 8..off::DATA_AXIS_STOP + 16].copy_from_slice(&2e-3f64.to_be_bytes()); - buf[off::DATA_START..off::DATA_START + 4].copy_from_slice(&(data_start as u32).to_be_bytes()); - buf[off::DATA_LENGTH..off::DATA_LENGTH + 8].copy_from_slice(&(data_len as u64).to_be_bytes()); - - buf[param_hdr..param_hdr + 4].copy_from_slice(&(rec_size as u32).to_le_bytes()); - buf[param_hdr + 8..param_hdr + 12].copy_from_slice(&((n_records - 1) as u32).to_le_bytes()); - write_param_record(&mut buf, param_recs, b"X_OFFSET", true, 1.5, b""); - write_param_record( - &mut buf, - param_recs + rec_size, - b"Y_OFFSET", - true, - 75.0, - b"", - ); - write_param_record( - &mut buf, - param_recs + 2 * rec_size, - b"X_DOMAIN", - false, - 0.0, - b"Proton", - ); - write_param_record( - &mut buf, - param_recs + 3 * rec_size, - b"Y_DOMAIN", - false, - 0.0, - b"Carbon13", - ); - - // Fill the data section in JEOL tiled order (plane, F1-block, F2-block, - // row-in-tile, col-in-tile) with a distinctive value per cell. - let re_val = |row: usize, col: usize| 100.0 + row as f64 * 10.0 + col as f64; - let im_val = |row: usize, col: usize| 1.0 + row as f64 + col as f64 * 0.5; - let n_f2b = cols_pad / TILE; - let n_f1b = rows_pad / TILE; - let mut w = data_start; - for plane in 0..planes { - for fb in 0..n_f1b { - for cb in 0..n_f2b { - for r in 0..TILE { - for c in 0..TILE { - let (row, col) = (fb * TILE + r, cb * TILE + c); - let v = if row < rows_real && col < cols_real { - if plane == 0 { - re_val(row, col) - } else { - im_val(row, col) - } - } else { - 0.0 - }; - buf[w..w + 8].copy_from_slice(&v.to_le_bytes()); - w += 8; - } - } - } - } - } - - let two = match read_jdf_bytes(&buf, "t2d.jdf".into()).unwrap() { - Acquisition::D2(d) => *d, - Acquisition::D1(_) => panic!("expected 2D"), - Acquisition::Electrophysiology(_) => panic!("expected NMR"), - Acquisition::Afm(_) => panic!("expected NMR"), - Acquisition::MassSpec(_) | Acquisition::Xrd(_) => panic!("expected NMR"), - Acquisition::Xps(_) => panic!("expected NMR"), - }; - assert_eq!((two.cols, two.rows), (cols_real, rows_real)); - assert_eq!(two.data.len(), cols_real * rows_real); - for row in 0..rows_real { - for col in 0..cols_real { - let got = two.data[row * cols_real + col]; - // Imaginary channel negated (conjugated on read). - assert_eq!( - got, - Complex64::new(re_val(row, col), -im_val(row, col)), - "cell ({row},{col}) mismatch (col {col} is in F2 block {})", - col / TILE - ); - } - } - assert_eq!(two.direct.nucleus, "1H"); - assert_eq!(two.indirect.nucleus, "13C"); - assert!((two.direct.carrier_ppm - 1.5).abs() < 1e-9); - assert!((two.indirect.carrier_ppm - 75.0).abs() < 1e-9); - assert_eq!(two.quad, QuadMode::Complex); - assert!(two.indirect_conjugate); -} - -#[test] -fn de_tiles_a_hand_built_hypercomplex_2d() { - // Both axes `Complex` (States hypercomplex): four sample planes ordered - // (F1-imag?, F2-imag?) with F2 toggling fastest — RR, RI, IR, II. Each t1 - // increment's cosine (F1-real) and sine (F1-imag) channel must be interleaved - // as consecutive stored rows and tagged QuadMode::States. - let (cols_pad, rows_pad) = (32usize, 32usize); - let (cols_real, rows_real) = (3usize, 2usize); - let planes = 4usize; - let rec_size = 64usize; - let param_hdr = HEADER_LEN; - let param_recs = param_hdr + 16; - let data_start = param_recs + rec_size; - let data_len = cols_pad * rows_pad * planes * 8; - let mut buf = vec![0u8; data_start + data_len]; - - buf[..8].copy_from_slice(MAGIC); - buf[off::ENDIAN] = 1; // little-endian body - buf[off::DATA_DIMENSION_NUMBER] = 2; - buf[off::DATA_AXIS_TYPE] = AXIS_COMPLEX; - buf[off::DATA_AXIS_TYPE + 1] = AXIS_COMPLEX; - buf[off::DATA_POINTS..off::DATA_POINTS + 4].copy_from_slice(&(cols_pad as u32).to_be_bytes()); - buf[off::DATA_POINTS + 4..off::DATA_POINTS + 8] - .copy_from_slice(&(rows_pad as u32).to_be_bytes()); - buf[off::DATA_OFFSET_STOP..off::DATA_OFFSET_STOP + 4] - .copy_from_slice(&((cols_real - 1) as u32).to_be_bytes()); - buf[off::DATA_OFFSET_STOP + 4..off::DATA_OFFSET_STOP + 8] - .copy_from_slice(&((rows_real - 1) as u32).to_be_bytes()); - buf[off::DATA_START..off::DATA_START + 4].copy_from_slice(&(data_start as u32).to_be_bytes()); - buf[off::DATA_LENGTH..off::DATA_LENGTH + 8].copy_from_slice(&(data_len as u64).to_be_bytes()); - buf[param_hdr..param_hdr + 4].copy_from_slice(&(rec_size as u32).to_le_bytes()); - buf[param_hdr + 8..param_hdr + 12].copy_from_slice(&0u32.to_le_bytes()); - - // Distinctive value per (plane, row, col); a single 32-tile so tiling is a - // plain row-major fill within each plane. - let val = |plane: usize, row: usize, col: usize| { - 1000.0 * plane as f64 + 10.0 * row as f64 + col as f64 - }; - let mut w = data_start; - for plane in 0..planes { - for row in 0..rows_pad { - for col in 0..cols_pad { - let v = if row < rows_real && col < cols_real { - val(plane, row, col) - } else { - 0.0 - }; - buf[w..w + 8].copy_from_slice(&v.to_le_bytes()); - w += 8; - } - } - } - - let two = match read_jdf_bytes(&buf, "hc2d.jdf".into()).unwrap() { - Acquisition::D2(d) => *d, - Acquisition::D1(_) => panic!("expected 2D"), - Acquisition::Electrophysiology(_) => panic!("expected NMR"), - Acquisition::Afm(_) => panic!("expected NMR"), - Acquisition::MassSpec(_) | Acquisition::Xrd(_) => panic!("expected NMR"), - Acquisition::Xps(_) => panic!("expected NMR"), - }; - assert_eq!(two.quad, QuadMode::States); - assert_eq!(two.cols, cols_real); - assert_eq!( - two.rows, - 2 * rows_real, - "cos/sin channels interleaved as rows" - ); - assert_eq!(two.data.len(), 2 * rows_real * cols_real); - for row in 0..rows_real { - for col in 0..cols_real { - // Cosine channel (F1-real): planes RR (0) and RI (1). - let cos = two.data[(2 * row) * cols_real + col]; - assert_eq!(cos, Complex64::new(val(0, row, col), -val(1, row, col))); - // Sine channel (F1-imag): planes IR (2) and II (3). - let sin = two.data[(2 * row + 1) * cols_real + col]; - assert_eq!(sin, Complex64::new(val(2, row, col), -val(3, row, col))); - } - } - assert!(two.indirect_conjugate); -} - -#[test] -fn detects_nus_echo_antiecho_grid_from_rate() { - let mut p = Params::empty(); - p.strings.insert("sampling".into(), "NUS (Auto)".into()); - p.strings.insert("pn_type".into(), "y".into()); - p.strings.insert("nus_mode".into(), "poisson gap".into()); - p.f64s.insert("sampling_rate".into(), 25.0); - let nus = detect_nus(&[], &p, 32).expect("nus detected"); - assert_eq!(nus.acquired, 32); - assert_eq!(nus.grid, 128, "grid = round(M / rate)"); - assert!(nus.echo_antiecho, "pn_type = y is echo/anti-echo"); - assert!(nus.schedule.is_none(), "schedule withheld until entered"); - assert_eq!(nus.mode, "poisson gap"); -} - -#[test] -fn detects_nus_phase_modulated_and_skips_linear() { - // Type-4 NUS (HMBC): NUS but no P/N conversion. - let mut p = Params::empty(); - p.strings.insert("sampling".into(), "NUS (Auto)".into()); - p.f64s.insert("sampling_rate".into(), 25.0); - let nus = detect_nus(&[], &p, 64).expect("nus detected"); - assert_eq!(nus.grid, 256); - assert!(!nus.echo_antiecho); - - // Uniform (Linear) sampling is not NUS. - let mut lin = Params::empty(); - lin.strings.insert("sampling".into(), "Linear".into()); - assert!(detect_nus(&[], &lin, 256).is_none()); - - // No sampling parameter at all is not NUS. - assert!(detect_nus(&[], &Params::empty(), 128).is_none()); -} - -fn serialized_nuslist(name: &[u8], values: &[u32]) -> Vec { - let mut bytes = Vec::new(); - bytes.extend_from_slice(&0x271du32.to_be_bytes()); - bytes.extend_from_slice(&(name.len() as u32).to_be_bytes()); - bytes.extend_from_slice(name); - bytes.extend_from_slice(&[0, 0]); - bytes.extend_from_slice(&0x2b2au32.to_be_bytes()); - bytes.extend_from_slice(&(values.len() as u32).to_be_bytes()); - for value in values { - bytes.extend_from_slice(&0x271au32.to_be_bytes()); - bytes.extend_from_slice(&1u32.to_be_bytes()); - bytes.extend_from_slice(&value.to_be_bytes()); - } - bytes -} - -#[test] -fn extracts_serialized_big_endian_nuslist() { - let bytes = serialized_nuslist(b"Y_NUSLIST", &[1, 2, 5, 9, 16]); - assert_eq!( - extract_nuslist(&bytes, b"Y_NUSLIST"), - Some(vec![1, 2, 5, 9, 16]) - ); -} - -#[test] -fn detect_nus_uses_valid_file_schedule_and_original_grid() { - let bytes = serialized_nuslist(b"Y_NUSLIST", &[1, 2, 5, 9]); - let mut p = Params::empty(); - p.strings.insert("sampling".into(), "NUS (Auto)".into()); - p.f64s.insert("sampling_rate".into(), 50.0); - p.f64s.insert("Y_ORIG_POINTS".into(), 16.0); - p.f64s.insert("nuslist_idx_base".into(), 1.0); - - let nus = detect_nus(&bytes, &p, 4).expect("nus detected"); - assert_eq!(nus.grid, 16); - assert_eq!(nus.schedule, Some(vec![0, 1, 4, 8])); -} - -#[test] -fn detect_nus_rejects_invalid_file_schedule() { - let mut p = Params::empty(); - p.strings.insert("sampling".into(), "NUS (Auto)".into()); - p.f64s.insert("sampling_rate".into(), 25.0); - - let wrong_count = serialized_nuslist(b"Y_NUSLIST", &[1, 2, 3]); - assert!( - detect_nus(&wrong_count, &p, 4) - .expect("nus detected") - .schedule - .is_none() - ); - - let duplicate = serialized_nuslist(b"Y_NUSLIST", &[1, 2, 2, 4]); - assert!( - detect_nus(&duplicate, &p, 4) - .expect("nus detected") - .schedule - .is_none() - ); -} diff --git a/crates/io/src/lib.rs b/crates/io/src/lib.rs index a49191dc..0662c956 100644 --- a/crates/io/src/lib.rs +++ b/crates/io/src/lib.rs @@ -1,19 +1,20 @@ -//! Data I/O: spectral format parsers producing the neutral [`NmrData`] container. +//! Data I/O. NMR imports retain the checked, evidence-bearing nmr dataset. pub mod abf2; pub mod archive; -pub mod bruker; pub mod delimited; mod format; -pub mod jcamp_dx; -pub mod jeol; mod mass_spec; pub mod mzml; pub mod nanoscope; -mod nmr_origin; +pub mod nmr_bridge; +mod nmr_input; +pub mod nmr_sampling; +pub mod nmr_series; +mod nmr_series_input; +pub mod nmr_view; pub mod origin; pub mod sciex_wiff; -pub mod varian; pub mod waters; pub mod xlsx; pub mod xps; @@ -40,7 +41,6 @@ pub struct Provenance { #[derive(Debug, Clone, Copy, PartialEq, Eq)] pub enum LoadWarningCode { ArchiveEntryFailed, - OptionalImaginaryMissing, MissingStimulus, InvalidMetadata, MissingCalibration, @@ -66,7 +66,6 @@ pub struct LoadResult { pub acquisition_identity: AcquisitionIdentity, pub format: DataFormat, pub provenance: Provenance, - nmr_origin: Option, pub warnings: Vec, } @@ -83,20 +82,10 @@ impl LoadResult { acquisition_identity, format, provenance, - nmr_origin: None, warnings, } } - pub fn with_nmr_origin(mut self, origin: NmrOrigin) -> Self { - self.nmr_origin = Some(origin); - self - } - - pub fn take_nmr_origin(&mut self) -> Option { - self.nmr_origin.take() - } - pub fn into_parts( self, ) -> ( @@ -104,7 +93,6 @@ impl LoadResult { AcquisitionIdentity, DataFormat, Provenance, - Option, Vec, ) { ( @@ -112,16 +100,11 @@ impl LoadResult { self.acquisition_identity, self.format, self.provenance, - self.nmr_origin, self.warnings, ) } } -pub use nmr_origin::{ - NmrInstrumentOrigin, NmrOrigin, NmrPortableMetadata, NmrSourceFormat, NmrSourceParameters, -}; - #[derive(Debug, Clone, PartialEq, Eq, serde::Serialize, serde::Deserialize)] pub struct AcquisitionIdentity { /// The specimen, recording, run, or other scientific subject. @@ -156,7 +139,8 @@ pub enum Domain { Frequency, } -/// Neutral, format-independent container for a single 1D acquisition. +/// Explicit calibrated samples for simulations and CRAFT input views. +/// File imports and project payloads retain the native NMR Dataset. #[derive(Debug, Clone)] pub struct NmrData { pub points: Vec, @@ -166,8 +150,8 @@ pub struct NmrData { pub carrier_ppm: f64, pub nucleus: String, pub source: String, - /// Digital-filter group delay in points, removed by the FFT stage as a - /// first-order phase ramp. Nonzero for Bruker; 0.0 when absent. + /// Explicit digital-filter delay in points; zero declares a known zero + /// delay. Inputs with unknown delay must use the native Dataset API. pub group_delay: f64, } @@ -246,6 +230,8 @@ pub enum PseudoKind { /// reconstructed or hand-entered rulers. #[derive(Debug, Clone, Copy, PartialEq, Eq)] pub enum AxisSource { + /// Exact portable axis evidence retained by the NMR library. + LibraryEvidence, /// Explicit `{v1, v2, …}` list embedded in the experiment text (exact). EmbeddedList, /// `start..stop : step` ramp descriptor embedded in the experiment text. @@ -319,43 +305,18 @@ pub fn gyromagnetic_ratio(nucleus: &str) -> Option { Some(g) } -/// Gradient-shape δ-coefficient for the effective diffusion time, matching the -/// JEOL `bpp_ste_diffusion` definitions. Defaults to the SQUARE value. -pub fn gradient_shape_factor(shape: &str) -> f64 { - match shape.trim().to_ascii_uppercase().as_str() { - "SINE" => 0.3125, - "SQUARE_SINE" => 0.30167, - "TRAPEZOID" => 0.32545, - "S_RECTANGLE" => 0.32526, - _ => 1.0 / 3.0, - } -} - -/// Non-uniform sampling (NUS) metadata for the indirect axis. Present when the -/// acquisition sampled only a subset of the nominal F1 grid; the missing -/// increments must be reconstructed before the F1 FFT. Readers recover the -/// sampling schedule when the source format stores it; otherwise `schedule` -/// stays `None` until the user supplies the list. +/// Explicit sampling declaration for programmatic two-dimensional inputs. +/// Imported schedules belong to the native Dataset, including order and duplicates. #[derive(Debug, Clone)] pub struct NusMeta { - /// Nominal full grid size N (complex increments) the schedule indexes into. pub grid: usize, - /// Acquired complex increment count M (the stored, sampled rows). pub acquired: usize, - /// Index base of a sampling list (JEOL `nuslist_idx_base`, normally 1). - pub idx_base: usize, - /// Scheduling mode label (`poisson gap`, …), surfaced for the user. - pub mode: String, - /// True for echo/anti-echo (P/N) coherence selection (`pn_type = "y"`): the - /// two stored F1 channels are P and N and need a `pn_to_shr` conversion - /// before the States-style hypercomplex assembly. - pub echo_antiecho: bool, - /// Sampling schedule from the source file or user: one nominal-grid index - /// per acquired increment, stored 0-based (`idx_base` already subtracted). + /// Zero-based logical indices, one per acquired observation. Construction + /// rejects an absent schedule; PlotX never fills missing observations. pub schedule: Option>, } -/// Neutral, format-independent container for a single 2D acquisition. `data` is +/// Explicit programmatic input, not a vendor reader or persisted payload. `data` is /// a row-major matrix of `rows` (indirect / F1) rows, each a complex FID of /// `cols` (direct / F2) points. #[derive(Debug, Clone)] @@ -400,8 +361,7 @@ impl NmrData2D { /// A loaded acquisition: 1D or 2D. Higher layers dispatch on the dimensionality. #[derive(Debug, Clone)] pub enum Acquisition { - D1(NmrData), - D2(Box), + Nmr(nmr_view::NmrSource), Electrophysiology(Box), Afm(Box), MassSpec(Box), @@ -613,15 +573,14 @@ pub struct ElectrophysiologyData { #[derive(Debug, thiserror::Error)] pub enum IoError { + #[error("NMR read failed: {0}")] + Nmr(#[source] Box), #[error("i/o error: {0}")] Io(#[from] std::io::Error), #[error("archive error: {0}")] Archive(String), - #[error("not a JEOL Delta file: bad magic (expected \"JEOL.NMR\")")] - BadMagic, - #[error("file is truncated: needed {needed} bytes at offset {offset}, have {have}")] Truncated { offset: usize, @@ -629,15 +588,12 @@ pub enum IoError { have: usize, }, - #[error("unsupported JEOL feature: {0}")] + #[error("unsupported data: {0}")] Unsupported(String), #[error("invalid ABF2 file: {0}")] InvalidAbf2(String), - #[error(transparent)] - JcampDx(#[from] jcamp_dx::JcampDxError), - #[error("invalid NanoScope file: {0}")] InvalidNanoScope(String), @@ -669,12 +625,6 @@ pub enum IoError { #[error("invalid XPS data: {0}")] InvalidXps(String), - #[error("invalid Varian/Agilent VnmrJ data: {0}")] - InvalidVarian(String), - - #[error("unsupported Varian/Agilent VnmrJ data: {0}")] - UnsupportedVarian(String), - #[error("NMR conversion failed: {0}")] NmrConversion(String), } @@ -695,15 +645,6 @@ pub fn detect_format(path: impl AsRef) -> Result { MassSpectrometryFormat::WatersMassLynxRaw, )); } - if let Some(format) = bruker::detect_processed(path) { - return Ok(format); - } - if bruker::is_bruker(path) { - return Ok(DataFormat::Nmr(NmrFormat::BrukerRaw)); - } - if varian::is_varian(path) { - return Ok(DataFormat::Nmr(NmrFormat::VarianAgilentRaw)); - } let ext = path .extension() .and_then(|e| e.to_str()) @@ -731,8 +672,6 @@ pub fn detect_format(path: impl AsRef) -> Result { "abf" if abf2::is_abf2(path) => { Ok(DataFormat::Electrophysiology(ElectrophysiologyFormat::Abf2)) } - "jdf" => Ok(DataFormat::Nmr(NmrFormat::JeolDelta)), - "dx" | "jdx" | "jcamp" => Ok(DataFormat::Nmr(NmrFormat::JcampDx1D)), "mzml" => Ok(DataFormat::MassSpectrometry(MassSpectrometryFormat::MzMl)), "wiff" => Ok(DataFormat::MassSpectrometry( MassSpectrometryFormat::SciexWiff, @@ -748,11 +687,13 @@ pub fn detect_format(path: impl AsRef) -> Result { _ if abf2::is_abf2(path) => { Ok(DataFormat::Electrophysiology(ElectrophysiologyFormat::Abf2)) } - _ if jeol::is_jdf(path) => Ok(DataFormat::Nmr(NmrFormat::JeolDelta)), - _ => Err(IoError::Unsupported(format!( - "unrecognised path {}: expected mzML, legacy SCIEX .wiff, Rigaku FI .raw/.rasx/profile .txt, a Waters .raw directory, NanoScope .spm/.pfc, ABF2 .abf, JEOL .jdf, JCAMP-DX .dx/.jdx/.jcamp, Bruker fid/ser or pdata, or a Varian/Agilent VnmrJ .fid directory", - path.display() - ))), + _ => match nmr_bridge::read_options().detect(path) { + Ok(format) => nmr_bridge::detected_format(format, path), + Err(error) if error.kind() == nmr::ReadErrorKind::Unrecognized => { + Err(IoError::Unsupported(format!("unrecognised scientific acquisition {}", path.display()))) + } + Err(error) => Err(IoError::Nmr(Box::new(error))), + }, } } @@ -760,13 +701,7 @@ pub fn load_path(path: impl AsRef) -> Result { let path = path.as_ref(); match detect_format(path)? { DataFormat::Electrophysiology(ElectrophysiologyFormat::Abf2) => abf2::load(path), - DataFormat::Nmr(NmrFormat::JeolDelta) => jeol::load_jdf_path(path), - DataFormat::Nmr(NmrFormat::BrukerRaw) => bruker::load_raw(path), - DataFormat::Nmr(NmrFormat::VarianAgilentRaw) => varian::load_raw(path), - DataFormat::Nmr(NmrFormat::BrukerProcessed1D | NmrFormat::BrukerProcessed2D) => { - bruker::load_processed(path) - } - DataFormat::Nmr(NmrFormat::JcampDx1D) => jcamp_dx::load(path), + DataFormat::Nmr(_) => nmr_bridge::load(path), DataFormat::Afm(AfmFormat::BrukerNanoScopeSpm | AfmFormat::BrukerPeakForceCapture) => { nanoscope::load(path) } diff --git a/crates/io/src/nmr_bridge.rs b/crates/io/src/nmr_bridge.rs new file mode 100644 index 00000000..301be3f7 --- /dev/null +++ b/crates/io/src/nmr_bridge.rs @@ -0,0 +1,179 @@ +//! Checked NMR import boundary. The library dataset owns all scientific facts; +//! the values below are presentation summaries, never processing inputs. + +use crate::{ + AcquisitionIdentity, DataFormat, IoError, LoadWarning, LoadWarningCode, NmrFormat, Provenance, +}; +use nmr::dataset::DescriptorRef; +use nmr::provenance::SourceKind; +use nmr::{Dataset, ExecutionContext, Format, ReadOptions, ReadPreference, ReadWarning}; +use std::{path::Path, sync::Arc}; + +#[path = "nmr_bridge_snapshot.rs"] +pub mod snapshot; + +/// PlotX's agreed import policy. Exact file selections are resolved by nmr; +/// ordinary experiment directories prefer raw and same-kind ambiguity is an error. +pub fn read_options() -> ReadOptions { + ReadOptions::new() + .preference(ReadPreference::PreferRaw) + .allow_experimental_vendor_semantics(true) +} + +pub fn read(path: &Path, context: &mut ExecutionContext<'_>) -> Result, IoError> { + read_options() + .read_with_context(path, context) + .map(Arc::new) + .map_err(|error| IoError::Nmr(Box::new(error))) +} + +pub fn load(path: &Path) -> Result { + let dataset = read(path, &mut ExecutionContext::default())?; + loaded(dataset) +} + +pub(super) fn loaded(dataset: Arc) -> Result { + Ok(crate::LoadResult { + acquisition: crate::Acquisition::Nmr(crate::nmr_view::NmrSource::new(dataset.clone())?), + acquisition_identity: identity(&dataset), + format: format(&dataset)?, + provenance: provenance(&dataset)?, + warnings: warnings(&dataset), + }) +} + +pub fn format(dataset: &Dataset) -> Result { + use nmr::{processed::Format as Processed, raw::RawFormat as Raw}; + let format = match dataset.source_format() { + Some(Format::Raw(Raw::BrukerRaw)) => NmrFormat::BrukerRaw, + Some(Format::Raw(Raw::VarianRaw)) => NmrFormat::VarianAgilentRaw, + Some(Format::Raw(Raw::JeolDelta) | Format::Processed(Processed::JeolDelta)) => { + NmrFormat::JeolDelta + } + Some(Format::Processed(Processed::JcampDx)) => NmrFormat::JcampDx1D, + Some(Format::Processed(Processed::BrukerTopSpin)) => match shape(dataset)?.len() { + 1 => NmrFormat::BrukerProcessed1D, + 2 => NmrFormat::BrukerProcessed2D, + _ => { + return Err(IoError::NmrConversion( + "unsupported Bruker spectrum rank".into(), + )); + } + }, + _ => { + return Err(IoError::NmrConversion( + "dataset has no supported import format".into(), + )); + } + }; + Ok(DataFormat::Nmr(format)) +} + +pub fn shape(dataset: &Dataset) -> Result, IoError> { + match dataset.descriptor() { + DescriptorRef::Raw(descriptor) => Ok(descriptor.logical_shape()), + DescriptorRef::Processed(descriptor) => Ok(descriptor.logical_shape()), + _ => Err(IoError::NmrConversion("unsupported NMR descriptor".into())), + } +} + +pub fn identity(dataset: &Dataset) -> AcquisitionIdentity { + let identity = dataset.identity(); + AcquisitionIdentity { + subject: identity.subject().map(str::to_owned), + acquisition: identity.acquisition().map(str::to_owned), + source_label: identity + .source_label() + .map(str::to_owned) + .unwrap_or_else(|| { + AcquisitionIdentity::from_path(dataset.selected_path().unwrap_or(Path::new(""))) + .source_label + }), + } +} + +pub fn provenance(dataset: &Dataset) -> Result { + let selected_path = dataset + .selected_path() + .ok_or_else(|| IoError::NmrConversion("dataset has no original read selection".into()))?; + let mut data = dataset + .sources() + .iter() + .filter(|source| source.kind() == SourceKind::Data); + let primary = data.next().ok_or_else(|| { + IoError::NmrConversion("imported dataset has no data source record".into()) + })?; + Ok(Provenance { + selected_path: selected_path.to_owned(), + data_path: primary.path().to_owned(), + parameter_paths: dataset + .sources() + .iter() + .filter(|source| source.kind() == SourceKind::Parameters) + .map(|source| source.path().to_owned()) + .collect(), + companion_paths: data + .map(|source| source.path().to_owned()) + .chain( + dataset + .sources() + .iter() + .filter(|source| { + !matches!(source.kind(), SourceKind::Data | SourceKind::Parameters) + }) + .map(|source| source.path().to_owned()), + ) + .collect(), + }) +} + +pub fn warnings(dataset: &Dataset) -> Vec { + dataset.warnings().iter().filter_map(|warning| { + let (code, message, path) = match warning { + // The opt-in policy is documented; retain this evidence on the NMR + // dataset without turning every successful import into an alert. + ReadWarning::ExperimentalVendorSemantics { .. } => return None, + ReadWarning::MissingOptionalSource { role, path, impact, .. } => ( + LoadWarningCode::MissingCompanion, + format!("Optional {role} is missing (affects {impact:?})."), + Some(path.clone()), + ), + ReadWarning::MissingMetadata { field, axis, impact, .. } => ( + if *impact == nmr::WarningImpact::AxisCalibration { LoadWarningCode::MissingCalibration } else { LoadWarningCode::InvalidMetadata }, + format!("NMR metadata {field:?} is missing on axis {axis:?} (affects {impact:?}); no value was inferred."), + dataset.selected_path().map(Path::to_owned), + ), + _ => (LoadWarningCode::InvalidMetadata, format!("NMR import: {warning:?}"), dataset.selected_path().map(Path::to_owned)), + }; + Some(LoadWarning { code, message, path }) + }).collect() +} + +/// Recognized NMR selections, including malformed/ambiguous acquisitions that +/// must reach the reader's diagnostic rather than be descended into as folders. +pub fn is_candidate(path: &Path) -> bool { + match read_options().detect(path) { + Ok(_) => true, + Err(error) => { + error.format().is_some() || matches!(error.kind(), nmr::ReadErrorKind::Ambiguous) + } + } +} + +pub(crate) fn detected_format(format_id: Format, path: &Path) -> Result { + use nmr::{processed::Format as Processed, raw::RawFormat as Raw}; + Ok(DataFormat::Nmr(match format_id { + Format::Raw(Raw::BrukerRaw) => NmrFormat::BrukerRaw, + Format::Raw(Raw::VarianRaw) => NmrFormat::VarianAgilentRaw, + Format::Raw(Raw::JeolDelta) | Format::Processed(Processed::JeolDelta) => { + NmrFormat::JeolDelta + } + Format::Processed(Processed::JcampDx) => NmrFormat::JcampDx1D, + // The host's format catalog distinguishes spectrum ranks. Obtain that + // fact from the library descriptor; do not inspect vendor parameters. + Format::Processed(Processed::BrukerTopSpin) => { + return self::format(read(path, &mut ExecutionContext::default())?.as_ref()); + } + _ => return Err(IoError::Unsupported(format!("NMR format {format_id:?}"))), + })) +} diff --git a/crates/io/src/nmr_bridge_snapshot.rs b/crates/io/src/nmr_bridge_snapshot.rs new file mode 100644 index 00000000..63b4c848 --- /dev/null +++ b/crates/io/src/nmr_bridge_snapshot.rs @@ -0,0 +1,36 @@ +//! A single snapshot frame for embedding in a bounded project entry. +//! The project writer owns transaction publication and compression. + +use nmr::{ + Dataset, ExecutionContext, + snapshot::{AcceptRecordedHistory, SnapshotError, SnapshotLimits}, +}; +use std::{ + io::{Read, Write}, + sync::Arc, +}; + +pub fn write( + input: &Dataset, + writer: &mut impl Write, + limits: SnapshotLimits, + context: &mut ExecutionContext<'_>, +) -> Result<(), SnapshotError> { + nmr::snapshot::write_snapshot_with_context(input, writer, limits, context) +} + +/// Accept recorded history only after integrity/model checks. Restore never +/// consults source paths. The caller must supply one bounded archive entry. +pub fn read( + reader: &mut impl Read, + limits: SnapshotLimits, + context: &mut ExecutionContext<'_>, +) -> Result, SnapshotError> { + let checked = nmr::snapshot::read_snapshot_with_context(reader, limits, context)?; + context.check_cancelled()?; + let mut trailing = [0u8; 1]; + if reader.read(&mut trailing)? != 0 { + return Err(SnapshotError::Structure); + } + Ok(Arc::new(checked.restore(AcceptRecordedHistory))) +} diff --git a/crates/io/src/nmr_input.rs b/crates/io/src/nmr_input.rs new file mode 100644 index 00000000..245e78a0 --- /dev/null +++ b/crates/io/src/nmr_input.rs @@ -0,0 +1,108 @@ +//! Explicit programmatic samples for simulations and high-level analysis. +//! File imports use `nmr_bridge::read` and never pass through this constructor. + +use crate::{Domain, IoError, NmrData, nmr_view::NmrSource}; +use nmr::acquisition::{ComponentBasis, GroupDelayState, PendingGroupDelay}; +use nmr::axis::{AxisCoordinates, AxisDomain, AxisRole, AxisUnit, FrequencyEvidence}; +use nmr::processed::{ProcessedAxis, ProcessedDataset, ProcessedOrigin, ProcessedProvenance}; +use nmr::raw::{ + ChemicalShiftReference, DirectSamples, RawAxis, RawAxisKind, RawDatasetBuilder, RawMetadata, +}; +use std::sync::Arc; + +impl TryFrom for NmrSource { + type Error = IoError; + + fn try_from(data: NmrData) -> Result { + let fail = |error: &dyn std::fmt::Display| IoError::NmrConversion(error.to_string()); + let frequency = + FrequencyEvidence::new(Some(data.observe_freq_mhz), None).map_err(|e| fail(&e))?; + let dataset = match data.domain { + Domain::Time => { + let axis = RawAxis::new( + RawAxisKind::Direct(DirectSamples::Complex), + AxisDomain::Time, + Some(AxisUnit::Second), + data.points.len(), + AxisCoordinates::Uniform { + start: 0.0, + step: 1.0 / data.spectral_width_hz, + }, + ) + .map_err(|e| fail(&e))? + .with_spectral_width_hz(Some(data.spectral_width_hz)) + .map_err(|e| fail(&e))? + .with_frequency_evidence(Some(frequency)) + .map_err(|e| fail(&e))? + .with_nucleus(Some(data.nucleus)) + .map_err(|e| fail(&e))? + .with_chemical_shift_reference(Some( + ChemicalShiftReference::user_constructed( + data.carrier_ppm, + data.observe_freq_mhz, + ) + .map_err(|e| fail(&e))?, + )) + .map_err(|e| fail(&e))? + .with_group_delay(GroupDelayState::Pending( + PendingGroupDelay::user_constructed(data.group_delay).map_err(|e| fail(&e))?, + )) + .map_err(|e| fail(&e))?; + RawDatasetBuilder::new(vec![axis], RawMetadata::default()) + .map_err(|e| fail(&e))? + .dense(data.points) + .map_err(|e| fail(&e))? + .into() + } + Domain::Frequency => { + // This input type declares a uniform ppm grid by width, carrier + // and reference frequency. Imported explicit grids bypass it. + let step = data.spectral_width_hz / data.points.len() as f64; + let axis = ProcessedAxis::new( + AxisRole::Signal, + AxisDomain::Frequency, + Some(AxisUnit::Hertz), + data.points.len(), + AxisCoordinates::Uniform { + start: -(data.points.len() as f64) / 2.0 * step, + step, + }, + ComponentBasis::Cartesian, + ) + .map_err(|e| fail(&e))? + .with_frequency_evidence(Some(frequency)) + .map_err(|e| fail(&e))? + .with_spectral_width_hz(Some(data.spectral_width_hz.abs())) + .map_err(|e| fail(&e))? + .with_nucleus(Some(data.nucleus)) + .map_err(|e| fail(&e))?; + let spectrum = ProcessedDataset::from_complex_trace( + axis, + data.points, + ProcessedProvenance::new(ProcessedOrigin::Unknown, Vec::new()) + .map_err(|e| fail(&e))?, + ) + .map_err(|e| fail(&e))?; + use nmr::processing::{ + FrequencyFrame, ProcessingOperation, ProcessingPlan, ReferenceSource, + }; + ProcessingPlan::new(vec![ProcessingOperation::ResolveFrequencyFrame { + axis: 0, + frame: FrequencyFrame::Ppm(ReferenceSource::Explicit( + ChemicalShiftReference::user_constructed( + data.carrier_ppm, + data.observe_freq_mhz, + ) + .map_err(|e| fail(&e))?, + )), + }]) + .map_err(|e| fail(&e))? + .apply(&spectrum.into()) + .map_err(|e| fail(&e))? + } + }; + let mut source = Self::new(Arc::new(dataset))?; + source.set_programmatic_label(data.source); + Ok(source) + } +} diff --git a/crates/io/src/nmr_origin.rs b/crates/io/src/nmr_origin.rs deleted file mode 100644 index f46f6053..00000000 --- a/crates/io/src/nmr_origin.rs +++ /dev/null @@ -1,66 +0,0 @@ -//! Retained provenance for imported NMR acquisitions. - -use serde::{Deserialize, Serialize}; - -#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)] -#[serde(rename_all = "snake_case")] -pub enum NmrSourceFormat { - BrukerRaw, - JeolDelta, -} - -impl NmrSourceFormat { - pub const fn label(self) -> &'static str { - match self { - Self::BrukerRaw => "Bruker", - Self::JeolDelta => "JEOL", - } - } -} - -/// Lossless acquisition parameters retained without duplicating the signal. -#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] -#[serde(tag = "vendor", rename_all = "snake_case")] -pub enum NmrSourceParameters { - Bruker { - acqus: String, - title: Option, - pulse_program: Option, - }, - Jeol { - /// Fixed header and parameter-list bytes before the signal section. - metadata_base64: String, - }, -} - -#[derive(Debug, Clone, PartialEq, Default, Serialize, Deserialize)] -pub struct NmrPortableMetadata { - pub solvent: Option, - pub temperature_k: Option, - pub transients: Option, - pub pulse_sequence: Option, -} - -#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)] -pub struct NmrInstrumentOrigin { - pub format: NmrSourceFormat, - pub source_sha256: [u8; 32], - pub portable: NmrPortableMetadata, - pub parameters: NmrSourceParameters, -} - -#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)] -#[serde(tag = "kind", rename_all = "snake_case")] -pub enum NmrOrigin { - Instrument { instrument: NmrInstrumentOrigin }, - Derived, -} - -impl NmrOrigin { - pub fn instrument(&self) -> Option<&NmrInstrumentOrigin> { - match self { - Self::Instrument { instrument } => Some(instrument), - Self::Derived => None, - } - } -} diff --git a/crates/io/src/nmr_sampling.rs b/crates/io/src/nmr_sampling.rs new file mode 100644 index 00000000..0a2d0116 --- /dev/null +++ b/crates/io/src/nmr_sampling.rs @@ -0,0 +1,82 @@ +//! Explicit import inputs; nmr validates them against the vendor acquisition. + +use crate::{IoError, LoadResult, nmr_bridge}; +use serde::{Deserialize, Serialize}; +use std::{io::Read, path::Path, sync::Arc}; + +/// Invocation data only. The checked declaration is persisted inside snapshot v1. +#[derive(Clone, Debug, Deserialize, Serialize)] +#[serde(deny_unknown_fields)] +pub struct SamplingDeclaration { + pub assertion_id: String, + pub source: String, + pub grid_shape: Vec, + pub coordinates: Vec>, + pub index_base: IndexBase, + pub component_counts: Vec, +} + +#[derive(Clone, Copy, Debug, Deserialize, Serialize)] +#[serde(rename_all = "snake_case")] +pub enum IndexBase { + Zero, + One, +} + +impl SamplingDeclaration { + pub fn into_native(self) -> Result { + let id = nmr::raw::AssertionId::try_new(self.assertion_id) + .map_err(|error| IoError::NmrConversion(error.to_string()))?; + Ok(nmr::SamplingDeclaration::new( + id, + self.source, + self.grid_shape, + self.coordinates, + match self.index_base { + IndexBase::Zero => nmr::SamplingIndexBase::Zero, + IndexBase::One => nmr::SamplingIndexBase::One, + }, + self.component_counts, + )) + } +} + +/// Bound external declaration text before decoding its nested coordinate lists. +pub fn read_declaration(path: &Path) -> Result { + const MAX_BYTES: u64 = 8 * 1024 * 1024; + let mut bytes = Vec::new(); + std::fs::File::open(path)? + .take(MAX_BYTES + 1) + .read_to_end(&mut bytes)?; + if bytes.len() as u64 > MAX_BYTES { + return Err(IoError::NmrConversion( + "sampling declaration exceeds 8 MiB".into(), + )); + } + serde_json::from_slice(&bytes) + .map_err(|error| IoError::NmrConversion(format!("invalid sampling declaration: {error}"))) +} + +pub fn read( + path: &Path, + declaration: SamplingDeclaration, + context: &mut nmr::ExecutionContext<'_>, +) -> Result, IoError> { + nmr_bridge::read_options() + .sampling_declaration(declaration.into_native()?) + .read_with_context(path, context) + .map(Arc::new) + .map_err(|error| IoError::Nmr(Box::new(error))) +} + +pub fn load(path: &Path, declaration: SamplingDeclaration) -> Result { + nmr_bridge::loaded(read( + path, + declaration, + &mut nmr::ExecutionContext::default(), + )?) +} + +pub fn load_with_declaration_file(path: &Path, declaration: &Path) -> Result { + load(path, read_declaration(declaration)?) +} diff --git a/crates/io/src/nmr_series.rs b/crates/io/src/nmr_series.rs new file mode 100644 index 00000000..d2b93681 --- /dev/null +++ b/crates/io/src/nmr_series.rs @@ -0,0 +1,170 @@ +//! Immutable application summaries of a rank-two library Dataset. + +use crate::nmr_view::{NmrAxis, NmrSource}; +use crate::{AxisSource, DiffusionMeta, Domain, IoError, PseudoAxis, PseudoKind}; +use nmr::axis::{AxisCoordinates, AxisDomain, AxisQuantity, AxisUnit}; +use std::ops::Deref; + +#[derive(Clone, Debug)] +pub struct NmrDimension { + pub nucleus: String, + pub observe_freq_mhz: Option, + pub spectral_width_hz: Option, + pub unit: Option, + pub domain: AxisDomain, +} + +impl From<&NmrAxis> for NmrDimension { + fn from(axis: &NmrAxis) -> Self { + Self { + nucleus: axis.nucleus.clone().unwrap_or_default(), + observe_freq_mhz: axis.observe_frequency_mhz(), + spectral_width_hz: axis.spectral_width_hz, + unit: axis.unit, + domain: axis.domain, + } + } +} + +#[derive(Clone, Debug)] +pub struct NusSummary { + pub grid: usize, + pub acquired: usize, + pub schedule: Vec, +} + +/// These fields are read-only through `NmrSeriesSource`. None are serialized as +/// another scientific payload or used to reconstruct library samples. +#[derive(Clone, Debug)] +pub struct SeriesSummary { + pub rows: usize, + pub cols: usize, + pub direct: NmrDimension, + pub indirect: NmrDimension, + pub source: String, + pub experiment: Option, + pub pseudo_axis: Option, + pub diffusion: Option, + pub nus: Option, +} + +#[derive(Clone, Debug)] +pub struct NmrSeriesSource { + source: NmrSource, + summary: SeriesSummary, +} + +impl Deref for NmrSeriesSource { + type Target = SeriesSummary; + fn deref(&self) -> &Self::Target { + &self.summary + } +} + +impl NmrSeriesSource { + pub fn new(source: NmrSource) -> Result { + if source.axes().len() != 2 { + return Err(IoError::NmrConversion( + "select a rank-two NMR dataset".into(), + )); + } + let axes = source.axes(); + let raw = source.dataset().as_raw(); + let quantity = raw + .map(|raw| raw.descriptor().axes()[0].quantity()) + .or_else(|| { + source + .dataset() + .as_processed() + .map(|data| data.descriptor().axes()[0].quantity()) + }) + .flatten(); + let pseudo_axis = if axes[0].domain == AxisDomain::Parameter + && !matches!(axes[0].coordinates, AxisCoordinates::Unknown) + { + Some(PseudoAxis { + name: axes[0].label.clone().unwrap_or_else(|| "Parameter".into()), + kind: match quantity { + Some(AxisQuantity::MagneticFieldGradientStrength) => PseudoKind::Gradient, + Some(AxisQuantity::TimeDelay) => PseudoKind::Delay, + _ => PseudoKind::Generic, + }, + values: axes[0].coordinate_values()?, + unit: match axes[0].unit { + Some(AxisUnit::Second) => "s", + Some(AxisUnit::TeslaPerMeter) => "mT/m", + Some(AxisUnit::Tesla) => "T", + Some(AxisUnit::Hertz) => "Hz", + Some(AxisUnit::Ppm) => "ppm", + _ => "", + } + .into(), + source: AxisSource::LibraryEvidence, + }) + } else { + None + }; + let acquisition = raw.map(|raw| raw.descriptor().acquisition()); + let diffusion = acquisition + .and_then(|metadata| metadata.diffusion()) + .and_then(|metadata| { + let shape_factor = match metadata + .gradient_shape()? + .trim() + .to_ascii_uppercase() + .as_str() + { + "SQUARE" => 1.0 / 3.0, + "SINE" => 0.3125, + "SQUARE_SINE" => 0.30167, + "TRAPEZOID" => 0.32545, + "S_RECTANGLE" => 0.32526, + _ => return None, + }; + Some(DiffusionMeta { + gamma: crate::gyromagnetic_ratio(axes[1].nucleus.as_deref()?)?, + delta: metadata.gradient_pulse_duration_seconds(), + big_delta: metadata.diffusion_time_seconds(), + tau: metadata.recovery_delay_seconds()?, + shape_factor, + }) + }); + let nus = raw + .and_then(|raw| raw.sampling_schedule()) + .map(|schedule| NusSummary { + grid: axes[0].points, + acquired: schedule.coordinates().len(), + schedule: schedule + .coordinates() + .iter() + .map(|coordinate| coordinate.as_slice()[0]) + .collect(), + }); + let summary = SeriesSummary { + rows: axes[0].points, + cols: axes[1].points, + direct: (&axes[1]).into(), + indirect: (&axes[0]).into(), + source: source.source().to_owned(), + experiment: acquisition + .and_then(|metadata| metadata.pulse_program().map(str::to_owned)), + pseudo_axis, + diffusion, + nus, + }; + Ok(Self { source, summary }) + } + + pub fn source_dataset(&self) -> &NmrSource { + &self.source + } + pub fn input_domain(&self, axis: usize) -> Result { + match self.source.axes().get(axis).map(|axis| axis.domain) { + Some(AxisDomain::Time) => Ok(Domain::Time), + Some(AxisDomain::Frequency) => Ok(Domain::Frequency), + _ => Err(IoError::NmrConversion( + "parameter axes do not accept spectral processing".into(), + )), + } + } +} diff --git a/crates/io/src/nmr_series_input.rs b/crates/io/src/nmr_series_input.rs new file mode 100644 index 00000000..68039d72 --- /dev/null +++ b/crates/io/src/nmr_series_input.rs @@ -0,0 +1,278 @@ +//! Programmatic tensor construction; vendor readers supply their own library Dataset. + +use crate::nmr_series::NmrSeriesSource; +use crate::nmr_view::NmrSource; +use crate::{Domain, IoError, NmrData2D, PseudoKind, QuadMode}; +use nmr::axis::{AxisCoordinates, AxisDomain, AxisQuantity, AxisRole, AxisUnit, FrequencyEvidence}; +use nmr::processed::{ + ComponentBasis, ProcessedAxis, ProcessedDataset, ProcessedDescriptor, ProcessedOrigin, + ProcessedProvenance, +}; +use nmr::raw::*; +use std::sync::Arc; + +fn fail(error: impl std::fmt::Display) -> IoError { + IoError::NmrConversion(error.to_string()) +} + +impl TryFrom for NmrSeriesSource { + type Error = IoError; + fn try_from(input: NmrData2D) -> Result { + if input.indirect_conjugate + || matches!(input.quad, QuadMode::StatesTppi | QuadMode::EchoAntiecho) + { + return Err(fail( + "Construct an explicit nmr component encoding for this programmatic input", + )); + } + let parameter = input.pseudo_axis.as_ref(); + let lanes = if input.domain == Domain::Time + && input.quad == QuadMode::States + && parameter.is_none() + { + 2 + } else { + 1 + }; + if !input.rows.is_multiple_of(lanes) + || input.data.len() + != input + .rows + .checked_mul(input.cols) + .ok_or_else(|| fail("tensor size overflow"))? + { + return Err(fail("programmatic tensor shape does not match samples")); + } + let rows = input + .nus + .as_ref() + .map_or(input.rows / lanes, |nus| nus.grid); + let axis_fields = |index: usize| { + let dim = if index == 0 { + &input.indirect + } else { + &input.direct + }; + let points = if index == 0 { rows } else { input.cols }; + if index == 0 + && let Some(parameter) = parameter + { + let (unit, quantity) = match parameter.kind { + PseudoKind::Gradient => ( + Some(AxisUnit::TeslaPerMeter), + Some(AxisQuantity::MagneticFieldGradientStrength), + ), + PseudoKind::Delay => (Some(AxisUnit::Second), Some(AxisQuantity::TimeDelay)), + PseudoKind::Generic => (None, None), + }; + return ( + dim, + points, + AxisDomain::Parameter, + unit, + AxisCoordinates::Explicit(parameter.values.clone()), + quantity, + ); + } + match input.domain { + Domain::Time => ( + dim, + points, + AxisDomain::Time, + Some(AxisUnit::Second), + AxisCoordinates::Uniform { + start: 0.0, + step: 1.0 / dim.spectral_width_hz, + }, + None, + ), + Domain::Frequency => { + let step = dim.spectral_width_hz / points as f64 / dim.observe_freq_mhz; + ( + dim, + points, + AxisDomain::Frequency, + Some(AxisUnit::Ppm), + AxisCoordinates::Uniform { + start: dim.carrier_ppm - points as f64 / 2.0 * step, + step, + }, + None, + ) + } + } + }; + let dataset: nmr::Dataset = if input.domain == Domain::Time { + let mut axes = Vec::new(); + for index in 0..2 { + let (dim, points, domain, unit, coordinates, quantity) = axis_fields(index); + let kind = if index == 1 { + RawAxisKind::Direct(DirectSamples::Complex) + } else if parameter.is_some() { + RawAxisKind::Parameter + } else if lanes == 2 { + RawAxisKind::Indirect(IndirectComponents::Cartesian( + ComponentEvidence::user_constructed(), + )) + } else { + RawAxisKind::Indirect(IndirectComponents::Scalar) + }; + let mut axis = + RawAxis::new(kind, domain, unit, points, coordinates).map_err(fail)?; + if domain == AxisDomain::Parameter { + axis = axis + .with_quantity(quantity) + .map_err(fail)? + .with_label(parameter.map(|parameter| parameter.name.clone())); + } else { + axis = axis + .with_nucleus((!dim.nucleus.is_empty()).then(|| dim.nucleus.clone())) + .map_err(fail)? + .with_spectral_width_hz(Some(dim.spectral_width_hz)) + .map_err(fail)? + .with_frequency_evidence(Some( + FrequencyEvidence::new(Some(dim.observe_freq_mhz), None) + .map_err(fail)?, + )) + .map_err(fail)? + .with_chemical_shift_reference(Some( + ChemicalShiftReference::user_constructed( + dim.carrier_ppm, + dim.observe_freq_mhz, + ) + .map_err(fail)?, + )) + .map_err(fail)? + .with_group_delay(if index == 1 { + GroupDelayState::Pending( + PendingGroupDelay::user_constructed(dim.group_delay) + .map_err(fail)?, + ) + } else { + GroupDelayState::NotApplicable + }) + .map_err(fail)?; + } + axes.push(axis); + } + let mut metadata = + RawMetadata::new(None, None, None, None, input.experiment.clone()).map_err(fail)?; + if let Some(meta) = input.diffusion { + let shape = if (meta.shape_factor - 1.0 / 3.0).abs() < 1e-12 { + "SQUARE" + } else { + return Err(fail( + "Declare diffusion analysis settings separately for a custom gradient shape", + )); + }; + metadata = metadata + .with_diffusion(Some( + DiffusionAcquisition::new( + 0, + "programmatic_gradient".into(), + meta.delta, + "programmatic_delta".into(), + meta.big_delta, + "programmatic_big_delta".into(), + Some((meta.tau, "programmatic_tau".into())), + Some((shape.into(), "programmatic_shape".into())), + ) + .map_err(fail)?, + )) + .map_err(fail)?; + } + let builder = RawDatasetBuilder::new(axes, metadata).map_err(fail)?; + if let Some(nus) = &input.nus { + let indices = nus + .schedule + .as_ref() + .ok_or_else(|| fail("Supply a complete NUS sampling schedule"))?; + if indices.len() != input.rows / lanes || indices.len() != nus.acquired { + return Err(fail("NUS observations do not match the schedule")); + } + let coordinates: Vec<_> = indices + .iter() + .map(|index| SamplingCoordinate::new(vec![*index])) + .collect(); + let traces = input + .data + .chunks_exact(lanes * input.cols) + .enumerate() + .map(|(index, samples)| { + SparseTrace::new( + ObservationOrdinal::new(index), + coordinates[index].clone(), + samples.to_vec(), + ) + }) + .collect(); + builder + .sparse( + traces, + SamplingSchedule::new(vec![rows], coordinates).map_err(fail)?, + ) + .map_err(fail)? + .into() + } else { + builder.dense(input.data).map_err(fail)?.into() + } + } else { + let mut axes = Vec::new(); + for index in 0..2 { + let (dim, points, domain, unit, coordinates, quantity) = axis_fields(index); + let axis = ProcessedAxis::new( + if domain == AxisDomain::Parameter { + AxisRole::ArrayParameter + } else { + AxisRole::Signal + }, + domain, + unit, + points, + coordinates, + if index == 1 { + ComponentBasis::Cartesian + } else { + ComponentBasis::Scalar + }, + ) + .map_err(fail)?; + axes.push(if domain == AxisDomain::Parameter { + axis.with_quantity(quantity).map_err(fail)? + } else { + axis.with_nucleus((!dim.nucleus.is_empty()).then(|| dim.nucleus.clone())) + .map_err(fail)? + .with_frequency_evidence(Some( + FrequencyEvidence::new(Some(dim.observe_freq_mhz), None) + .map_err(fail)?, + )) + .map_err(fail)? + .with_spectral_width_hz(Some(dim.spectral_width_hz.abs())) + .map_err(fail)? + }); + } + let samples = input + .data + .iter() + .flat_map(|value| [value.re, value.im]) + .collect(); + ProcessedDataset::from_dense_samples( + ProcessedDescriptor::new(axes).map_err(fail)?, + samples, + ProcessedProvenance::new(ProcessedOrigin::Unknown, vec![]).map_err(fail)?, + ) + .map_err(fail)? + .into() + }; + let mut source = NmrSource::new(Arc::new(dataset))?; + source.set_programmatic_label(input.source); + Self::new(source) + } +} + +impl TryFrom for NmrSeriesSource { + type Error = IoError; + fn try_from(source: NmrSource) -> Result { + Self::new(source) + } +} diff --git a/crates/io/src/nmr_view.rs b/crates/io/src/nmr_view.rs new file mode 100644 index 00000000..35f5d6a2 --- /dev/null +++ b/crates/io/src/nmr_view.rs @@ -0,0 +1,294 @@ +//! Immutable descriptor views. The library Dataset remains the scientific input. + +use crate::{IoError, NmrData}; +use nmr::acquisition::{ComponentBasis, GroupDelayState}; +use nmr::axis::{AxisCoordinates, AxisDomain, AxisRole, AxisUnit, FrequencyEvidence}; +use nmr::dataset::DescriptorRef; +use nmr::{Complex64, Dataset}; +use std::sync::Arc; + +#[derive(Clone, Debug)] +pub struct NmrAxis { + pub role: AxisRole, + pub domain: AxisDomain, + pub unit: Option, + pub points: usize, + pub coordinates: AxisCoordinates, + pub nucleus: Option, + pub label: Option, + pub frequency: Option, + pub spectral_width_hz: Option, +} + +impl NmrAxis { + pub fn observe_frequency_mhz(&self) -> Option { + self.frequency + .and_then(|value| value.observe_frequency_mhz()) + } + + pub fn coordinate_values(&self) -> Result, IoError> { + match &self.coordinates { + AxisCoordinates::Explicit(values) => Ok(values.clone()), + AxisCoordinates::Uniform { start, step } => Ok((0..self.points) + .map(|index| step.mul_add(index as f64, *start)) + .collect()), + _ => Err(IoError::NmrConversion( + "axis coordinates are unknown".into(), + )), + } + } +} + +/// The descriptor summaries cannot be mutated independently of the owned input. +#[derive(Clone, Debug)] +pub struct NmrSource { + dataset: Arc, + axes: Vec, + source_label: String, +} + +impl NmrSource { + pub fn new(dataset: Arc) -> Result { + let axes = match dataset.descriptor() { + DescriptorRef::Raw(descriptor) => descriptor + .axes() + .iter() + .map(|axis| NmrAxis { + role: axis.role(), + domain: axis.domain(), + unit: axis.unit(), + points: axis.points(), + coordinates: axis.coordinates().clone(), + nucleus: axis.nucleus().map(str::to_owned), + label: axis.label().map(str::to_owned), + frequency: axis.frequency_evidence(), + spectral_width_hz: axis.spectral_width_hz(), + }) + .collect(), + DescriptorRef::Processed(descriptor) => descriptor + .axes() + .iter() + .map(|axis| NmrAxis { + role: axis.role(), + domain: axis.domain(), + unit: axis.unit(), + points: axis.points(), + coordinates: axis.coordinates().clone(), + nucleus: axis.nucleus().map(str::to_owned), + label: axis.label().map(str::to_owned), + frequency: axis.frequency_evidence(), + spectral_width_hz: axis.spectral_width_hz(), + }) + .collect(), + _ => return Err(IoError::NmrConversion("unsupported NMR descriptor".into())), + }; + let source_label = crate::nmr_bridge::identity(&dataset).source_label; + Ok(Self { + dataset, + axes, + source_label, + }) + } + + pub fn dataset(&self) -> &Arc { + &self.dataset + } + + pub fn axes(&self) -> &[NmrAxis] { + &self.axes + } + + pub fn source(&self) -> &str { + &self.source_label + } + + pub fn identity(&self) -> crate::AcquisitionIdentity { + let mut identity = crate::nmr_bridge::identity(&self.dataset); + identity.source_label = self.source_label.clone(); + identity + } + + /// A host display label does not alter acquisition facts or canonical digests. + pub fn with_display_label(mut self, label: String) -> Self { + self.source_label = label; + self + } + + /// MHz used for chemical-shift differences, distinct from observe frequency. + /// Axes without reference evidence return `None`. + pub fn reference_frequency_mhz(&self, axis: usize) -> Option { + if let Some(raw) = self.dataset.as_raw() { + return raw + .descriptor() + .axes() + .get(axis)? + .chemical_shift_reference() + .map(|reference| reference.reference_frequency_mhz()); + } + self.dataset + .as_processed()? + .axis_evidence(axis)? + .reference_frequency_mhz() + } + + pub fn len(&self) -> usize { + self.axes.first().map_or(0, |axis| axis.points) + } + + pub fn is_empty(&self) -> bool { + self.len() == 0 + } + + pub fn nucleus(&self) -> &str { + self.axes + .first() + .and_then(|axis| axis.nucleus.as_deref()) + .unwrap_or("") + } + + pub(crate) fn set_programmatic_label(&mut self, label: String) { + self.source_label = label; + } + + pub fn direct_axis(&self) -> Result<&NmrAxis, IoError> { + self.axes + .last() + .ok_or_else(|| IoError::NmrConversion("NMR input has no axis".into())) + } + + pub fn domain(&self) -> Result { + match self.direct_axis()?.domain { + AxisDomain::Time => Ok(crate::Domain::Time), + AxisDomain::Frequency => Ok(crate::Domain::Frequency), + _ => Err(IoError::NmrConversion( + "NMR signal axis has no time or frequency domain".into(), + )), + } + } + + pub fn has_imaginary(&self, axis: usize) -> bool { + if let Some(raw) = self.dataset.as_raw() { + return raw.descriptor().axes().get(axis).is_some_and(|axis| { + matches!( + axis.kind(), + nmr::raw::RawAxisKind::Direct(nmr::raw::DirectSamples::Complex) + ) || matches!( + axis.kind(), + nmr::raw::RawAxisKind::Indirect( + nmr::raw::IndirectComponents::Cartesian(_) + | nmr::raw::IndirectComponents::SharedComplex { .. } + ) + ) + }); + } + self.dataset.as_processed().is_some_and(|processed| { + processed.descriptor().axes().get(axis).is_some_and(|axis| { + matches!( + axis.component_basis(), + ComponentBasis::Cartesian | ComponentBasis::SharedComplex { .. } + ) + }) + }) + } + + /// Full complex samples of a rank-one input. A scalar display has zero + /// imaginary values, while `has_imaginary` continues to report its true basis. + pub fn trace(&self) -> Result, IoError> { + if self.axes.len() != 1 { + return Err(IoError::NmrConversion( + "select a one-dimensional NMR trace".into(), + )); + } + if let Some(raw) = self.dataset.as_raw() { + return raw + .read_trace(&[]) + .map(|trace| trace.samples().to_vec()) + .map_err(|error| IoError::Nmr(Box::new(error))); + } + let processed = self + .dataset + .as_processed() + .ok_or_else(|| IoError::NmrConversion("unsupported NMR trace representation".into()))?; + let axis = &processed.descriptor().axes()[0]; + if !matches!( + axis.component_basis(), + ComponentBasis::Scalar | ComponentBasis::Cartesian + ) { + return Err(IoError::NmrConversion( + "decode NMR components before displaying a spectrum".into(), + )); + } + (0..axis.points()) + .map(|point| { + let sample = |component| { + processed + .data() + .get(&[point], &[component]) + .map_err(|error| IoError::NmrConversion(error.to_string())) + }; + Ok(Complex64::new( + sample(0)?, + if axis.component_count() == 2 { + sample(1)? + } else { + 0.0 + }, + )) + }) + .collect() + } + + /// CRAFT requires a calibrated complex FID. Missing facts prevent analysis; + /// they do not prevent importing, saving or displaying the library Dataset. + pub fn craft_fid(&self) -> Result { + let missing = |message: &str| IoError::NmrConversion(message.to_owned()); + let raw = self + .dataset + .as_raw() + .ok_or_else(|| missing("CRAFT requires a raw FID"))?; + if self.axes.len() != 1 || !self.has_imaginary(0) { + return Err(missing( + "CRAFT requires one complex direct acquisition axis", + )); + } + let axis = &raw.descriptor().axes()[0]; + let reference = axis + .chemical_shift_reference() + .ok_or_else(|| missing("CRAFT requires chemical-shift reference evidence"))?; + let spectral_width_hz = axis + .spectral_width_hz() + .ok_or_else(|| missing("CRAFT requires spectral width"))?; + let observe_freq_mhz = axis + .frequency_evidence() + .and_then(|value| value.observe_frequency_mhz()) + .ok_or_else(|| missing("CRAFT requires observe frequency"))?; + let group_delay = match axis.group_delay() { + GroupDelayState::Pending(delay) => delay.delay_points(), + GroupDelayState::NotApplicable => 0.0, + _ => { + return Err(missing( + "CRAFT requires known digital-filter delay evidence", + )); + } + }; + if axis.domain() != AxisDomain::Time + || axis.unit() != Some(AxisUnit::Second) + || !matches!(axis.coordinates(), AxisCoordinates::Uniform { start, step } + if *start == 0.0 && (*step * spectral_width_hz - 1.0).abs() < 1e-10) + { + return Err(missing( + "CRAFT requires a uniform FID starting at the acquisition time origin", + )); + } + Ok(NmrData { + points: self.trace()?, + domain: crate::Domain::Time, + spectral_width_hz, + observe_freq_mhz, + carrier_ppm: reference.carrier_ppm(), + nucleus: axis.nucleus().unwrap_or("").to_owned(), + source: crate::nmr_bridge::identity(&self.dataset).source_label, + group_delay, + }) + } +} diff --git a/crates/io/src/varian.rs b/crates/io/src/varian.rs deleted file mode 100644 index 08dac627..00000000 --- a/crates/io/src/varian.rs +++ /dev/null @@ -1,302 +0,0 @@ -//! Varian and Agilent VNMR/VnmrJ raw acquisition reader. - -mod fid; -mod procpar; - -use crate::{ - Acquisition, DataFormat, Dim, Domain, IoError, LoadResult, NmrData, NmrData2D, NmrFormat, - Provenance, QuadMode, -}; -use procpar::Procpar; -use std::path::{Path, PathBuf}; - -pub fn is_varian(path: &Path) -> bool { - resolve(path).is_some_and(|(_, fid, procpar)| fid.is_file() && procpar.is_file()) -} - -fn resolve(path: &Path) -> Option<(PathBuf, PathBuf, PathBuf)> { - let dir = if path.is_dir() { - path.to_path_buf() - } else if path.file_name()?.to_str()? == "fid" { - path.parent()?.to_path_buf() - } else { - return None; - }; - Some((dir.clone(), dir.join("fid"), dir.join("procpar"))) -} - -pub fn load_raw(path: &Path) -> Result { - let (dir, data_path, procpar_path) = resolve(path) - .ok_or_else(|| IoError::InvalidVarian("select a .fid directory or its fid file".into()))?; - if !data_path.is_file() || !procpar_path.is_file() { - return Err(IoError::InvalidVarian( - "a VnmrJ dataset requires sibling fid and procpar files".into(), - )); - } - let params = Procpar::parse(&std::fs::read_to_string(&procpar_path)?)?; - let mut raw = fid::parse(&std::fs::read(&data_path)?)?; - // VNMR stores direct-dimension quadrature with the opposite sense to the - // forward-FFT convention used by PlotX. Normalize it at the importer seam - // so every downstream transform and time-domain model shares one axis - // convention. - for trace in &mut raw.traces { - for point in trace { - point.im = -point.im; - } - } - reject_unsupported(¶ms)?; - let acquisition = assemble(&dir, ¶ms, raw)?; - Ok(LoadResult::new( - acquisition, - crate::AcquisitionIdentity { - subject: sample_name(&dir, ¶ms), - acquisition: experiment_name(¶ms), - source_label: dir - .file_stem() - .and_then(|name| name.to_str()) - .unwrap_or("Untitled NMR") - .to_owned(), - }, - DataFormat::Nmr(NmrFormat::VarianAgilentRaw), - Provenance { - selected_path: path.to_path_buf(), - data_path, - parameter_paths: vec![procpar_path], - companion_paths: Vec::new(), - }, - Vec::new(), - )) -} - -fn reject_unsupported(p: &Procpar) -> Result<(), IoError> { - if p.number("ni2").unwrap_or(0.0) > 1.0 || p.number("ni3").unwrap_or(0.0) > 1.0 { - return Err(IoError::UnsupportedVarian( - "3D and 4D acquisitions are not supported".into(), - )); - } - if p.string("apptype") - .is_some_and(|s| s.to_ascii_lowercase().contains("imaging")) - { - return Err(IoError::UnsupportedVarian( - "MRI and imaging acquisitions are not supported".into(), - )); - } - if ["sampling", "nus", "nuslist"].iter().any(|name| { - p.string(name) - .is_some_and(|s| !s.is_empty() && !s.eq_ignore_ascii_case("n")) - }) { - return Err(IoError::UnsupportedVarian( - "non-uniform sampling is not supported".into(), - )); - } - Ok(()) -} - -fn assemble(dir: &Path, p: &Procpar, raw: fid::FidData) -> Result { - let procpar_np = exact_positive_usize(p.number("np")) - .ok_or_else(|| IoError::InvalidVarian("procpar is missing positive integer np".into()))?; - if procpar_np != raw.np { - return Err(IoError::InvalidVarian(format!( - "dimension mismatch: procpar np is {procpar_np}, but the fid header declares {}", - raw.np - ))); - } - let direct = direct_dim(p)?; - let total = raw.traces.len(); - let ni = exact_positive_usize(p.number("ni")); - let (phase_count, quad) = phase_layout(p)?; - let array = p.string("array").unwrap_or("").trim(); - if ni.unwrap_or(1) == 1 && total == 1 && array.is_empty() { - let source = description(dir, p, &direct, None); - return Ok(Acquisition::D1(NmrData { - points: raw.traces.into_iter().next().unwrap(), - domain: Domain::Time, - spectral_width_hz: direct.spectral_width_hz, - observe_freq_mhz: direct.observe_freq_mhz, - carrier_ppm: direct.carrier_ppm, - nucleus: direct.nucleus, - source, - group_delay: 0.0, - })); - } - let ni = ni.ok_or_else(|| { - IoError::UnsupportedVarian("multiple traces require a positive integer ni".into()) - })?; - if !array.is_empty() && array != "phase" { - return Err(IoError::UnsupportedVarian(format!( - "parameter arrays other than phase are not supported (array={array})" - ))); - } - let expected = ni - .checked_mul(phase_count) - .ok_or_else(|| IoError::InvalidVarian("2D trace count overflow".into()))?; - if total != expected { - return Err(IoError::UnsupportedVarian(format!( - "trace layout mismatch: fid contains {total} traces, but ni × phase_count is {expected}" - ))); - } - let seq = p - .string("seqfil") - .map(|s| s.to_ascii_lowercase()) - .filter(|s| !s.is_empty()); - let indirect = indirect_dim(p, seq.as_deref(), &direct)?; - let cols = raw.np / 2; - let source = description(dir, p, &direct, Some(&indirect)); - Ok(Acquisition::D2(Box::new(NmrData2D { - data: raw.traces.into_iter().flatten().collect(), - rows: total, - cols, - domain: Domain::Time, - direct, - indirect, - quad, - indirect_conjugate: false, - experiment: seq, - pseudo_axis: None, - diffusion: None, - nus: None, - source, - }))) -} - -fn phase_layout(p: &Procpar) -> Result<(usize, QuadMode), IoError> { - match p.numbers("phase").as_deref() { - None | Some([1.0]) => Ok((1, QuadMode::Complex)), - Some([1.0, 2.0]) => Ok((2, QuadMode::States)), - Some(values) => Err(IoError::UnsupportedVarian(format!( - "unsupported phase table {values:?}; only phase=1 and States phase=1,2 are supported" - ))), - } -} - -fn direct_dim(p: &Procpar) -> Result { - let mut direct = dim(p, "sw", "sfrq", "tof", "tn")?; - // VNMRJ uses rfl/rfp to recalibrate the displayed direct-axis reference. - // With a full spectral width, rfl is measured from the high-frequency - // edge and rfp is the chemical shift assigned at that location. - if let Some(rfl) = p.number("rfl").filter(|value| value.is_finite()) { - let rfp = p - .number("rfp") - .filter(|value| value.is_finite()) - .unwrap_or(0.0); - direct.carrier_ppm = rfp + (direct.spectral_width_hz * 0.5 - rfl) / direct.observe_freq_mhz; - } - Ok(direct) -} -fn indirect_dim(p: &Procpar, seq: Option<&str>, direct: &Dim) -> Result { - let homo = seq.is_some_and(|s| { - ["cosy", "tocsy", "noesy", "roesy"] - .iter() - .any(|name| s.contains(name)) - }); - let hetero = seq.is_some_and(|s| ["hsqc", "hmqc", "hmbc"].iter().any(|name| s.contains(name))); - if homo { - return dim_with_sw1(p, direct); - } - if hetero { - return dim(p, "sw1", "dfrq", "dof", "dn"); - } - let tn = normalize_nucleus(p.string("tn").unwrap_or("X")); - let dn = normalize_nucleus(p.string("dn").unwrap_or("X")); - if dn != "X" && dn != tn { - dim(p, "sw1", "dfrq", "dof", "dn") - } else if dn == "X" || dn == tn { - dim_with_sw1(p, direct) - } else { - Err(IoError::UnsupportedVarian( - "unknown sequence has ambiguous indirect channel".into(), - )) - } -} -fn dim_with_sw1(p: &Procpar, direct: &Dim) -> Result { - Ok(Dim { - spectral_width_hz: required_positive(p, "sw1")?, - observe_freq_mhz: direct.observe_freq_mhz, - carrier_ppm: direct.carrier_ppm, - nucleus: direct.nucleus.clone(), - group_delay: 0.0, - }) -} -fn dim(p: &Procpar, sw: &str, freq: &str, offset: &str, nucleus: &str) -> Result { - let spectral_width_hz = required_positive(p, sw)?; - let observe_freq_mhz = required_positive(p, freq)?; - let carrier_ppm = p - .number(offset) - .filter(|v| v.is_finite()) - .ok_or_else(|| IoError::InvalidVarian(format!("procpar is missing finite {offset}")))? - / observe_freq_mhz; - Ok(Dim { - spectral_width_hz, - observe_freq_mhz, - carrier_ppm, - nucleus: normalize_nucleus(p.string(nucleus).unwrap_or("X")), - group_delay: 0.0, - }) -} -fn required_positive(p: &Procpar, name: &str) -> Result { - p.number(name) - .filter(|v| v.is_finite() && *v > 0.0) - .ok_or_else(|| IoError::InvalidVarian(format!("procpar is missing positive finite {name}"))) -} -fn exact_positive_usize(v: Option) -> Option { - let v = v?; - if v.is_finite() && v > 0.0 && v.fract() == 0.0 && v <= usize::MAX as f64 { - Some(v as usize) - } else { - None - } -} -fn normalize_nucleus(value: &str) -> String { - let s = value.trim().trim_matches('"').replace(' ', ""); - let upper = s.to_ascii_uppercase(); - match upper.as_str() { - "H1" | "1H" | "PROTON" => "1H".into(), - "C13" | "13C" => "13C".into(), - "N15" | "15N" => "15N".into(), - "F19" | "19F" => "19F".into(), - "P31" | "31P" => "31P".into(), - "" | "OFF" | "NONE" => "X".into(), - _ => s, - } -} -fn description(dir: &Path, p: &Procpar, direct: &Dim, indirect: Option<&Dim>) -> String { - let nuclei = match indirect { - Some(indirect) => format!("{}/{}", direct.nucleus, indirect.nucleus), - None => direct.nucleus.clone(), - }; - sample_name(dir, p) - .into_iter() - .chain(std::iter::once(nuclei)) - .chain(experiment_name(p)) - .collect::>() - .join(" — ") -} - -fn sample_name(dir: &Path, p: &Procpar) -> Option { - ["samplename", "sample", "name", "filename"] - .into_iter() - .find_map(|name| p.string(name).and_then(nonempty)) - .map(str::to_owned) - .or_else(|| { - dir.file_stem() - .and_then(|name| name.to_str()) - .and_then(nonempty) - .map(str::to_owned) - }) -} - -fn experiment_name(p: &Procpar) -> Option { - ["pslabel", "seqfil"] - .into_iter() - .find_map(|name| p.string(name).and_then(nonempty)) - .map(str::to_owned) -} - -fn nonempty(value: &str) -> Option<&str> { - let value = value.trim(); - (!value.is_empty()).then_some(value) -} - -#[cfg(test)] -#[path = "varian/tests.rs"] -mod tests; diff --git a/crates/io/src/varian/fid.rs b/crates/io/src/varian/fid.rs deleted file mode 100644 index b78bbdec..00000000 --- a/crates/io/src/varian/fid.rs +++ /dev/null @@ -1,200 +0,0 @@ -use crate::IoError; -use num_complex::Complex64; - -const FILE_HEADER: usize = 32; -const BLOCK_HEADER: usize = 28; -const S_DATA: i16 = 0x1; -const S_SPEC: i16 = 0x2; -const S_32: i16 = 0x4; -const S_FLOAT: i16 = 0x8; -const S_COMPLEX: i16 = 0x10; -const S_HYPERCOMPLEX: i16 = 0x20; -const S_DDR: i16 = 0x80; -const S_SECND: i16 = 0x100; -const S_TRANSF: i16 = 0x200; -const S_3D: i16 = 0x400; -const SAMPLE_STATUS: i16 = S_32 | S_FLOAT; -const NB_HEADER_MASK: i32 = 0x0000f; -const NB_NI3: i32 = 0x10000; -const VERSION_FILE_ID_MASK: i16 = 0x07c0; -const VERSION_FID_FILE: i16 = 0x0040; - -#[derive(Debug)] -pub(super) struct FidData { - pub(super) traces: Vec>, - pub(super) np: usize, -} - -pub(super) fn parse(bytes: &[u8]) -> Result { - if bytes.len() < FILE_HEADER { - return truncated(0, FILE_HEADER, bytes.len()); - } - let nblocks = positive_i32(bytes, 0, "nblocks")?; - let ntraces = positive_i32(bytes, 4, "ntraces")?; - let np = positive_i32(bytes, 8, "np")?; - let ebytes = positive_i32(bytes, 12, "ebytes")?; - let tbytes = positive_i32(bytes, 16, "tbytes")?; - let bbytes = positive_i32(bytes, 20, "bbytes")?; - let version_id = i16::from_be_bytes(bytes[24..26].try_into().unwrap()); - let status = i16::from_be_bytes(bytes[26..28].try_into().unwrap()); - let raw_nbheaders = i32::from_be_bytes(bytes[28..32].try_into().unwrap()); - if raw_nbheaders & NB_NI3 != 0 { - return Err(unsupported( - "3D and 4D block-header layouts are not supported", - )); - } - if raw_nbheaders & !(NB_NI3 | NB_HEADER_MASK) != 0 { - return Err(invalid("nbheaders contains unknown layout flags")); - } - let nbheaders = usize::try_from(raw_nbheaders & NB_HEADER_MASK) - .ok() - .filter(|count| *count > 0) - .ok_or_else(|| invalid("nbheaders must declare at least one block header"))?; - if np % 2 != 0 { - return Err(invalid("np must be a positive even number")); - } - if status & S_DATA == 0 || !is_complex_fid(status) { - return Err(unsupported("fid is not complex time-domain data")); - } - if status & (S_SPEC | S_HYPERCOMPLEX) != 0 { - return Err(unsupported( - "processed spectra and hypercomplex payloads are not supported", - )); - } - if status & (S_SECND | S_TRANSF | S_3D) != 0 { - return Err(unsupported( - "transformed, transposed, and 3D payloads are not supported", - )); - } - let file_id = version_id & VERSION_FILE_ID_MASK; - if file_id != 0 && file_id != VERSION_FID_FILE { - return Err(unsupported( - "the software-version header identifies a processed data file", - )); - } - let sample = match (status & S_FLOAT != 0, status & S_32 != 0, ebytes) { - (false, false, 2) => Sample::I16, - (false, true, 4) => Sample::I32, - (true, _, 4) => Sample::F32, - _ => { - return Err(unsupported( - "status flags and ebytes do not describe int16, int32, or float32 samples", - )); - } - }; - let expected_tbytes = np - .checked_mul(ebytes) - .ok_or_else(|| invalid("trace size overflow"))?; - if tbytes != expected_tbytes { - return Err(invalid("tbytes does not equal np * ebytes")); - } - let headers_bytes = nbheaders - .checked_mul(BLOCK_HEADER) - .ok_or_else(|| invalid("block header size overflow"))?; - let trace_bytes = ntraces - .checked_mul(tbytes) - .ok_or_else(|| invalid("block trace size overflow"))?; - let minimum_bbytes = headers_bytes - .checked_add(trace_bytes) - .ok_or_else(|| invalid("block size overflow"))?; - if bbytes < minimum_bbytes { - return Err(invalid("bbytes is smaller than its headers and traces")); - } - let declared = nblocks - .checked_mul(bbytes) - .and_then(|n| FILE_HEADER.checked_add(n)) - .ok_or_else(|| invalid("file size overflow"))?; - if bytes.len() < declared { - return truncated(0, declared, bytes.len()); - } - if bytes.len() != declared { - return Err(invalid( - "file length does not match the declared block layout", - )); - } - - let total = nblocks - .checked_mul(ntraces) - .ok_or_else(|| invalid("trace count overflow"))?; - let mut traces = Vec::with_capacity(total); - for block in 0..nblocks { - let block_at = FILE_HEADER + block * bbytes; - let scale = i16::from_be_bytes(bytes[block_at..block_at + 2].try_into().unwrap()); - let block_status = - i16::from_be_bytes(bytes[block_at + 2..block_at + 4].try_into().unwrap()); - if block_status & S_DATA == 0 - || (block_status & S_COMPLEX == 0 && status & S_DDR == 0) - || block_status & S_SPEC != 0 - { - return Err(invalid( - "block header status is inconsistent with complex time-domain data", - )); - } - if block_status & S_HYPERCOMPLEX != 0 { - return Err(unsupported("hypercomplex block payloads are not supported")); - } - if block_status & SAMPLE_STATUS != status & SAMPLE_STATUS { - return Err(invalid( - "block sample type flags disagree with the file header", - )); - } - let factor = 2.0_f64.powi(i32::from(scale)); - if !factor.is_finite() { - return Err(invalid("block scale is out of range")); - } - for trace in 0..ntraces { - let at = block_at + headers_bytes + trace * tbytes; - let mut points = Vec::with_capacity(np / 2); - for pair in 0..np / 2 { - let real_at = at + pair * 2 * ebytes; - points.push(Complex64::new( - sample.read(bytes, real_at) * factor, - sample.read(bytes, real_at + ebytes) * factor, - )); - } - traces.push(points); - } - } - Ok(FidData { traces, np }) -} - -fn is_complex_fid(status: i16) -> bool { - status & (S_COMPLEX | S_DDR) != 0 -} - -#[derive(Clone, Copy)] -enum Sample { - I16, - I32, - F32, -} -impl Sample { - fn read(self, b: &[u8], at: usize) -> f64 { - match self { - Self::I16 => i16::from_be_bytes(b[at..at + 2].try_into().unwrap()) as f64, - Self::I32 => i32::from_be_bytes(b[at..at + 4].try_into().unwrap()) as f64, - Self::F32 => f32::from_be_bytes(b[at..at + 4].try_into().unwrap()) as f64, - } - } -} - -fn positive_i32(bytes: &[u8], at: usize, name: &str) -> Result { - let value = i32::from_be_bytes(bytes[at..at + 4].try_into().unwrap()); - usize::try_from(value) - .ok() - .filter(|v| *v > 0) - .ok_or_else(|| invalid(format!("{name} must be positive"))) -} -fn invalid(message: impl Into) -> IoError { - IoError::InvalidVarian(message.into()) -} -fn unsupported(message: impl Into) -> IoError { - IoError::UnsupportedVarian(message.into()) -} -fn truncated(offset: usize, needed: usize, have: usize) -> Result { - Err(IoError::Truncated { - offset, - needed, - have, - }) -} diff --git a/crates/io/src/varian/procpar.rs b/crates/io/src/varian/procpar.rs deleted file mode 100644 index 4cb47eb2..00000000 --- a/crates/io/src/varian/procpar.rs +++ /dev/null @@ -1,187 +0,0 @@ -use crate::IoError; -use std::collections::HashMap; - -#[derive(Debug, Clone)] -pub(super) enum Value { - Number(f64), - Text(String), -} - -#[derive(Debug, Default)] -pub(super) struct Procpar { - values: HashMap>, -} - -impl Procpar { - pub(super) fn parse(text: &str) -> Result { - let mut lines = text.lines().enumerate().peekable(); - let mut values = HashMap::new(); - while let Some((line_no, header)) = lines.next() { - if header.trim().is_empty() { - continue; - } - let fields = tokens(header).map_err(|e| invalid(line_no, e))?; - if fields.len() < 3 { - return Err(invalid( - line_no, - "parameter header has fewer than three fields", - )); - } - let name = fields[0].clone(); - let basic_type: i32 = fields[2] - .parse() - .map_err(|_| invalid(line_no, "invalid basic type"))?; - if basic_type != 1 && basic_type != 2 { - return Err(invalid(line_no, "unsupported basic type")); - } - let (value_line_no, first) = lines - .next() - .ok_or_else(|| invalid(line_no, "missing value record"))?; - let mut value_tokens = tokens(first).map_err(|e| invalid(value_line_no, e))?; - let count = parse_count(&mut value_tokens, value_line_no, "value")?; - while value_tokens.len() < count { - let (continuation_no, continuation) = lines - .next() - .ok_or_else(|| invalid(value_line_no, "truncated value record"))?; - value_tokens.extend(tokens(continuation).map_err(|e| invalid(continuation_no, e))?); - } - if value_tokens.len() != count { - return Err(invalid(value_line_no, "value count does not match record")); - } - let parsed = value_tokens - .into_iter() - .map(|token| { - if basic_type == 1 { - token.parse::().map(Value::Number).map_err(|_| { - invalid( - value_line_no, - "numeric parameter contains non-numeric value", - ) - }) - } else { - Ok(Value::Text(token)) - } - }) - .collect::, _>>()?; - - let (enum_line_no, enum_first) = lines - .next() - .ok_or_else(|| invalid(value_line_no, "missing enumeration record"))?; - let mut enum_tokens = tokens(enum_first).map_err(|e| invalid(enum_line_no, e))?; - let enum_count = parse_count(&mut enum_tokens, enum_line_no, "enumeration")?; - while enum_tokens.len() < enum_count { - let (continuation_no, continuation) = lines - .next() - .ok_or_else(|| invalid(enum_line_no, "truncated enumeration record"))?; - enum_tokens.extend(tokens(continuation).map_err(|e| invalid(continuation_no, e))?); - } - if enum_tokens.len() != enum_count { - return Err(invalid( - enum_line_no, - "enumeration count does not match record", - )); - } - values.insert(name, parsed); - } - Ok(Self { values }) - } - - pub(super) fn numbers(&self, name: &str) -> Option> { - self.values - .get(name)? - .iter() - .map(|v| match v { - Value::Number(n) => Some(*n), - Value::Text(_) => None, - }) - .collect() - } - - pub(super) fn number(&self, name: &str) -> Option { - self.numbers(name)?.first().copied() - } - - pub(super) fn strings(&self, name: &str) -> Option> { - self.values - .get(name)? - .iter() - .map(|v| match v { - Value::Text(s) => Some(s.as_str()), - Value::Number(_) => None, - }) - .collect() - } - - pub(super) fn string(&self, name: &str) -> Option<&str> { - self.strings(name)?.first().copied() - } -} - -fn parse_count(tokens: &mut Vec, line: usize, kind: &str) -> Result { - if tokens.is_empty() { - return Err(invalid(line, format!("missing {kind} count"))); - } - let count = tokens - .remove(0) - .parse::() - .map_err(|_| invalid(line, format!("invalid {kind} count")))?; - Ok(count) -} - -fn invalid(line: usize, message: impl Into) -> IoError { - IoError::InvalidVarian(format!("procpar line {}: {}", line + 1, message.into())) -} - -fn tokens(line: &str) -> Result, &'static str> { - let mut out = Vec::new(); - let mut chars = line.chars().peekable(); - while let Some(c) = chars.next() { - if c.is_whitespace() { - continue; - } - if c == '"' { - let mut value = String::new(); - let mut closed = false; - while let Some(c) = chars.next() { - if c == '"' { - closed = true; - break; - } - if c == '\\' { - value.push(chars.next().ok_or("unterminated escape in quoted string")?); - } else { - value.push(c); - } - } - if !closed { - return Err("unterminated quoted string"); - } - out.push(value); - } else { - let mut value = String::from(c); - while chars.peek().is_some_and(|c| !c.is_whitespace()) { - value.push(chars.next().unwrap()); - } - out.push(value); - } - } - Ok(out) -} - -#[cfg(test)] -mod tests { - use super::*; - - #[test] - fn parses_numbers_strings_arrays_and_enums() { - let p = Procpar::parse("sw 1 1 0 0 0 0 0 0 1 0\n2 1000 2000\n1 5000\ncomment 1 2 0 0 0 0 0 0 1 0\n1 \"a value with spaces\"\n2 \"yes\" \"no\"\n").unwrap(); - assert_eq!(p.numbers("sw"), Some(vec![1000.0, 2000.0])); - assert_eq!(p.string("comment"), Some("a value with spaces")); - } - - #[test] - fn rejects_bad_counts_and_quotes() { - assert!(Procpar::parse("x 1 1\n2 1\n0\n").is_err()); - assert!(Procpar::parse("x 1 2\n1 \"oops\n0\n").is_err()); - } -} diff --git a/crates/io/src/varian/tests.rs b/crates/io/src/varian/tests.rs deleted file mode 100644 index dc70db3f..00000000 --- a/crates/io/src/varian/tests.rs +++ /dev/null @@ -1,296 +0,0 @@ -use super::*; -use num_complex::Complex64; -use std::sync::atomic::{AtomicU64, Ordering}; - -fn record(name: &str, basic: i32, values: &str) -> String { - format!("{name} 1 {basic}\n{values}\n0\n") -} - -fn base_procpar() -> String { - [ - record("np", 1, "1 4"), - record("sw", 1, "1 4000"), - record("sfrq", 1, "1 500"), - record("tof", 1, "1 2500"), - record("tn", 2, "1 \"H1\""), - record("array", 2, "1 \"\""), - ] - .concat() -} - -#[derive(Clone, Copy)] -enum Encoding { - I16, - I32, - F32, -} - -fn fid_bytes(blocks: &[Vec>], encoding: Encoding, scales: &[i16]) -> Vec { - let nblocks = blocks.len(); - let ntraces = blocks[0].len(); - let np = blocks[0][0].len(); - let ebytes = match encoding { - Encoding::I16 => 2, - Encoding::I32 | Encoding::F32 => 4, - }; - let tbytes = np * ebytes; - let bbytes = 28 + ntraces * tbytes; - let status = 0x11 - | match encoding { - Encoding::I16 => 0, - Encoding::I32 => 0x4, - Encoding::F32 => 0xc, - }; - let mut out = Vec::new(); - for value in [nblocks, ntraces, np, ebytes, tbytes, bbytes] { - out.extend_from_slice(&(value as i32).to_be_bytes()); - } - out.extend_from_slice(&0_i16.to_be_bytes()); - out.extend_from_slice(&(status as i16).to_be_bytes()); - out.extend_from_slice(&1_i32.to_be_bytes()); - for (block, &scale) in blocks.iter().zip(scales) { - out.extend_from_slice(&scale.to_be_bytes()); - out.extend_from_slice(&(status as i16).to_be_bytes()); - out.extend_from_slice(&[0; 24]); - for trace in block { - for value in trace { - match encoding { - Encoding::I16 => out.extend_from_slice(&(*value as i16).to_be_bytes()), - Encoding::I32 => out.extend_from_slice(&(*value as i32).to_be_bytes()), - Encoding::F32 => out.extend_from_slice(&(*value as f32).to_be_bytes()), - } - } - } - } - out -} - -fn dataset(procpar: &str, fid: &[u8]) -> std::path::PathBuf { - static NEXT: AtomicU64 = AtomicU64::new(0); - let dir = std::env::temp_dir().join(format!( - "plotx_varian_{}_{}.fid", - std::process::id(), - NEXT.fetch_add(1, Ordering::Relaxed) - )); - let _ = std::fs::remove_dir_all(&dir); - std::fs::create_dir_all(&dir).unwrap(); - std::fs::write(dir.join("procpar"), procpar).unwrap(); - std::fs::write(dir.join("fid"), fid).unwrap(); - dir -} - -#[test] -fn loads_directory_and_fid_with_provenance_and_metadata() { - let mut procpar = base_procpar(); - procpar.push_str(&record("samplename", 2, "1 \"Test sample\"")); - procpar.push_str(&record("pslabel", 2, "1 \"PROTON\"")); - let dir = dataset( - &procpar, - &fid_bytes(&[vec![vec![1., 2., 3., 4.]]], Encoding::I16, &[1]), - ); - assert_eq!( - crate::detect_format(&dir).unwrap(), - DataFormat::Nmr(crate::NmrFormat::VarianAgilentRaw) - ); - for selected in [&dir, &dir.join("fid")] { - let loaded = load_raw(selected).unwrap(); - assert_eq!(loaded.provenance.selected_path, *selected); - assert_eq!(loaded.provenance.data_path, dir.join("fid")); - assert_eq!(loaded.provenance.parameter_paths, vec![dir.join("procpar")]); - let Acquisition::D1(data) = loaded.acquisition else { - panic!("expected 1D") - }; - assert_eq!( - data.points, - vec![Complex64::new(2., -4.), Complex64::new(6., -8.)] - ); - assert_eq!( - ( - data.spectral_width_hz, - data.observe_freq_mhz, - data.carrier_ppm - ), - (4000., 500., 5.) - ); - assert_eq!(data.nucleus, "1H"); - assert_eq!(data.source, "Test sample — 1H — PROTON"); - } - std::fs::remove_dir_all(dir).unwrap(); -} - -#[test] -fn direct_axis_uses_vnmrj_rfl_rfp_reference_when_present() { - let mut procpar = base_procpar(); - procpar.push_str(&record("rfl", 1, "1 1500")); - procpar.push_str(&record("rfp", 1, "1 1.25")); - let dir = dataset( - &procpar, - &fid_bytes(&[vec![vec![1., 2., 3., 4.]]], Encoding::I16, &[0]), - ); - let loaded = load_raw(&dir).unwrap(); - let Acquisition::D1(data) = loaded.acquisition else { - panic!("expected 1D") - }; - assert!((data.carrier_ppm - 2.25).abs() < 1e-12); - std::fs::remove_dir_all(dir).unwrap(); -} - -#[test] -fn reads_all_sample_widths_and_block_major_trace_minor_order() { - for encoding in [Encoding::I16, Encoding::I32, Encoding::F32] { - let bytes = fid_bytes( - &[ - vec![vec![1., 2.], vec![3., 4.]], - vec![vec![5., 6.], vec![7., 8.]], - ], - encoding, - &[0, 1], - ); - let raw = fid::parse(&bytes).unwrap(); - assert_eq!( - raw.traces.iter().flatten().copied().collect::>(), - vec![ - Complex64::new(1., 2.), - Complex64::new(3., 4.), - Complex64::new(10., 12.), - Complex64::new(14., 16.) - ] - ); - } -} - -#[test] -fn loads_homonuclear_and_heteronuclear_states_2d() { - let raw = fid_bytes( - &[ - vec![vec![1., 2., 3., 4.], vec![5., 6., 7., 8.]], - vec![vec![9., 10., 11., 12.], vec![13., 14., 15., 16.]], - ], - Encoding::I32, - &[0, 0], - ); - for (seq, indirect) in [("gcosy", (500., 5., "1H")), ("ghsqc", (125., 80., "13C"))] { - let mut p = base_procpar(); - p.push_str(&record("ni", 1, "1 2")); - p.push_str(&record("phase", 1, "2 1 2")); - p.push_str(&record("array", 2, "1 \"phase\"")); - p.push_str(&record("sw1", 1, "1 20000")); - p.push_str(&record("seqfil", 2, &format!("1 \"{seq}\""))); - if seq == "ghsqc" { - p.push_str(&record("dfrq", 1, "1 125")); - p.push_str(&record("dof", 1, "1 10000")); - p.push_str(&record("dn", 2, "1 \"C13\"")); - } - let dir = dataset(&p, &raw); - let Acquisition::D2(data) = load_raw(&dir).unwrap().acquisition else { - panic!("expected 2D") - }; - assert_eq!((data.rows, data.cols, data.quad), (4, 2, QuadMode::States)); - assert_eq!( - ( - data.indirect.observe_freq_mhz, - data.indirect.carrier_ppm, - data.indirect.nucleus.as_str() - ), - indirect - ); - assert_eq!(data.experiment.as_deref(), Some(seq)); - assert!(!data.indirect_conjugate); - std::fs::remove_dir_all(dir).unwrap(); - } -} - -#[test] -fn rejects_unsupported_two_entry_phase_table() { - let raw = fid_bytes( - &[ - vec![vec![1., 2., 3., 4.], vec![5., 6., 7., 8.]], - vec![vec![9., 10., 11., 12.], vec![13., 14., 15., 16.]], - ], - Encoding::I32, - &[0, 0], - ); - let mut p = base_procpar(); - p.push_str(&record("ni", 1, "1 2")); - p.push_str(&record("phase", 1, "2 1 3")); - p.push_str(&record("array", 2, "1 \"phase\"")); - let dir = dataset(&p, &raw); - - assert!(matches!(load_raw(&dir), Err(IoError::UnsupportedVarian(_)))); - std::fs::remove_dir_all(dir).unwrap(); -} - -#[test] -fn rejects_procpar_np_disagreement() { - let mut p = base_procpar(); - p.push_str(&record("np", 1, "1 6")); - let dir = dataset( - &p, - &fid_bytes(&[vec![vec![1., 2., 3., 4.]]], Encoding::I16, &[0]), - ); - - assert!(matches!(load_raw(&dir), Err(IoError::InvalidVarian(_)))); - std::fs::remove_dir_all(dir).unwrap(); -} - -#[test] -fn rejects_corrupt_and_unsupported_layouts() { - let mut odd = fid_bytes(&[vec![vec![1., 2., 3., 4.]]], Encoding::I16, &[0]); - odd[8..12].copy_from_slice(&3_i32.to_be_bytes()); - assert!(matches!(fid::parse(&odd), Err(IoError::InvalidVarian(_)))); - let mut truncated = fid_bytes(&[vec![vec![1., 2.]]], Encoding::I16, &[0]); - truncated.pop(); - assert!(matches!( - fid::parse(&truncated), - Err(IoError::Truncated { .. }) - )); - let mut spectrum = fid_bytes(&[vec![vec![1., 2.]]], Encoding::I16, &[0]); - spectrum[26..28].copy_from_slice(&0x13_i16.to_be_bytes()); - assert!(matches!( - fid::parse(&spectrum), - Err(IoError::UnsupportedVarian(_)) - )); -} - -#[test] -fn accepts_ddr_fid_without_legacy_complex_bit() { - let mut ddr = fid_bytes(&[vec![vec![1., 2., 3., 4.]]], Encoding::F32, &[0]); - let status = 0x00c9_i16; - ddr[26..28].copy_from_slice(&status.to_be_bytes()); - ddr[34..36].copy_from_slice(&status.to_be_bytes()); - - let raw = fid::parse(&ddr).unwrap(); - assert_eq!( - raw.traces[0], - vec![Complex64::new(1., 2.), Complex64::new(3., 4.)] - ); -} - -#[test] -fn rejects_processed_and_higher_dimensional_header_flags() { - let base = fid_bytes(&[vec![vec![1., 2., 3., 4.]]], Encoding::F32, &[0]); - for status_bit in [0x2_i16, 0x100, 0x200, 0x400] { - let mut bytes = base.clone(); - let status = i16::from_be_bytes(bytes[26..28].try_into().unwrap()) | status_bit; - bytes[26..28].copy_from_slice(&status.to_be_bytes()); - assert!(matches!( - fid::parse(&bytes), - Err(IoError::UnsupportedVarian(_)) - )); - } - - let mut ni3 = base.clone(); - ni3[28..32].copy_from_slice(&0x10001_i32.to_be_bytes()); - assert!(matches!( - fid::parse(&ni3), - Err(IoError::UnsupportedVarian(_)) - )); -} - -#[test] -fn rejects_block_sample_type_disagreement() { - let mut bytes = fid_bytes(&[vec![vec![1., 2., 3., 4.]]], Encoding::F32, &[0]); - let integer = fid_bytes(&[vec![vec![1., 2., 3., 4.]]], Encoding::I32, &[0]); - bytes[34..36].copy_from_slice(&integer[34..36]); - assert!(matches!(fid::parse(&bytes), Err(IoError::InvalidVarian(_)))); -} diff --git a/crates/io/tests/bruker_processed.rs b/crates/io/tests/bruker_processed.rs deleted file mode 100644 index ad40e06a..00000000 --- a/crates/io/tests/bruker_processed.rs +++ /dev/null @@ -1,131 +0,0 @@ -use plotx_io::{Acquisition, DataFormat, Domain, LoadWarningCode}; - -fn fixture(name: &str) -> std::path::PathBuf { - std::env::temp_dir().join(format!( - "plotx_{name}_{}_{}", - std::process::id(), - std::thread::current().name().unwrap_or("test") - )) -} - -#[test] -fn loads_big_endian_scaled_1r_from_experiment_directory() { - let root = fixture("bruker_processed_1d"); - let experiment = root.join("sample").join("3"); - let proc_dir = experiment.join("pdata").join("1"); - std::fs::create_dir_all(&proc_dir).unwrap(); - std::fs::write( - experiment.join("acqus"), - "##$TD= 8\n##$EXP= \n##$PULPROG= \n", - ) - .unwrap(); - std::fs::write( - proc_dir.join("procs"), - "##$SI= 4\n##$DTYPP= 0\n##$BYTORDP= 1\n##$NC_proc= 1\n\ - ##$SW_p= 4000\n##$SF= 400\n##$OFFSET= 10\n##$AXNUC= <1H>\n", - ) - .unwrap(); - let bytes: Vec = [1i32, 2, 3, 4] - .into_iter() - .flat_map(i32::to_be_bytes) - .collect(); - std::fs::write(proc_dir.join("1r"), bytes).unwrap(); - - assert_eq!( - plotx_io::detect_format(&experiment).unwrap(), - DataFormat::Nmr(plotx_io::NmrFormat::BrukerProcessed1D) - ); - let loaded = plotx_io::load_path(&experiment).unwrap(); - assert_eq!( - loaded.format, - DataFormat::Nmr(plotx_io::NmrFormat::BrukerProcessed1D) - ); - assert_eq!( - loaded.acquisition_identity.subject.as_deref(), - Some("sample") - ); - assert_eq!( - loaded.acquisition_identity.acquisition.as_deref(), - Some("PROTON") - ); - assert!( - loaded - .provenance - .parameter_paths - .contains(&experiment.join("acqus")) - ); - assert!( - loaded - .warnings - .iter() - .any(|warning| { warning.code == LoadWarningCode::OptionalImaginaryMissing }) - ); - let data = match loaded.acquisition { - Acquisition::D1(data) => data, - Acquisition::D2(_) => panic!("expected 1D"), - Acquisition::Electrophysiology(_) => panic!("expected NMR"), - Acquisition::Afm(_) => panic!("expected NMR"), - Acquisition::MassSpec(_) | Acquisition::Xrd(_) => panic!("expected NMR"), - Acquisition::Xps(_) => panic!("expected NMR"), - }; - assert_eq!(data.domain, Domain::Frequency); - assert_eq!( - data.points.iter().map(|value| value.re).collect::>(), - vec![8.0, 6.0, 4.0, 2.0] - ); - assert_eq!(data.carrier_ppm, 5.0); - assert_eq!(data.nucleus, "1H"); - - std::fs::remove_dir_all(root).unwrap(); -} - -#[test] -fn loads_2rr_and_reverses_both_frequency_axes() { - let root = fixture("bruker_processed_2d"); - let proc_dir = root.join("sample").join("7").join("pdata").join("2"); - std::fs::create_dir_all(&proc_dir).unwrap(); - std::fs::write( - proc_dir.join("procs"), - "##$SI= 3\n##$DTYPP= 0\n##$BYTORDP= 0\n##$NC_proc= 0\n\ - ##$SW_p= 3000\n##$SF= 600\n##$OFFSET= 9\n##$AXNUC= <1H>\n", - ) - .unwrap(); - std::fs::write( - proc_dir.join("proc2s"), - "##$SI= 2\n##$SW_p= 2000\n##$SF= 100\n##$OFFSET= 120\n##$AXNUC= <13C>\n", - ) - .unwrap(); - let bytes: Vec = [1i32, 2, 3, 4, 5, 6] - .into_iter() - .flat_map(i32::to_le_bytes) - .collect(); - std::fs::write(proc_dir.join("2rr"), bytes).unwrap(); - - let loaded = plotx_io::load_path(&proc_dir).unwrap(); - assert_eq!( - loaded.format, - DataFormat::Nmr(plotx_io::NmrFormat::BrukerProcessed2D) - ); - assert_eq!( - loaded.acquisition_identity.subject.as_deref(), - Some("sample") - ); - let data = match loaded.acquisition { - Acquisition::D2(data) => *data, - Acquisition::D1(_) => panic!("expected 2D"), - Acquisition::Electrophysiology(_) => panic!("expected NMR"), - Acquisition::Afm(_) => panic!("expected NMR"), - Acquisition::MassSpec(_) | Acquisition::Xrd(_) => panic!("expected NMR"), - Acquisition::Xps(_) => panic!("expected NMR"), - }; - assert_eq!(data.domain, Domain::Frequency); - assert_eq!((data.rows, data.cols), (2, 3)); - assert_eq!( - data.data.iter().map(|value| value.re).collect::>(), - vec![6.0, 5.0, 4.0, 3.0, 2.0, 1.0] - ); - assert_eq!(data.direct.carrier_ppm, 6.5); - assert_eq!(data.indirect.carrier_ppm, 110.0); - - std::fs::remove_dir_all(root).unwrap(); -} diff --git a/crates/io/tests/fixtures/nmr/README.md b/crates/io/tests/fixtures/nmr/README.md new file mode 100644 index 00000000..44737127 --- /dev/null +++ b/crates/io/tests/fixtures/nmr/README.md @@ -0,0 +1,30 @@ +# PlotX synthetic NMR integration inputs + +These small, original byte fixtures contain no experimental or personal data. +They are distributed under the repository license. Binary payloads are source +test inputs, not build output. None establishes independent vendor correctness. + +| Input | Construction and expected scientific meaning | +| --- | --- | +| `jcamp-hz.dx` | AFFN, descending coordinates 4,3,2,1 Hz; YFACTOR=2; scalar values 2,4,6,8. | +| `jcamp-ppm.dx` | Identical input with XUNITS=PPM; coordinates remain 4,3,2,1 ppm. The missing-observe-frequency test removes both OBSERVE records from a temporary copy. | +| `bruker-1d` | Little-endian int32 fid `[1,2,3,4]`; two complex samples `(1,2),(3,4)`; explicit GRPDLY=0. | +| `bruker-1d/pdata/1` | int32 1r `[1,2,3,4]`, NC_proc=1; scalar 2,4,6,8 at 10,7.5,5,2.5 ppm. | +| `bruker-states-tppi` | Continuous ser of int32 1..16; four physical rows, two logical increments and two lanes; FnMODE=5. | +| `bruker-states` | Same bytes and dimensions with FnMODE=4; States has no alternate-increment sign modulation. | +| `bruker-nus` | Same payload, FnTYPE=2, FnMODE=6, NusTD=8; full grid 4, two observations in order `[3,1]`; missing coordinates remain absent. | +| `jeol-complex.jdf` | JDF v1, direct complex axis, float64 LE sections R=1,2,3,4 and I=0,0,0,0; 1 ms dwell, no established delay or nucleus. The descriptor remains complex despite all-zero I. | +| `varian.fid` | File version 1, status 0x11, one 28-byte block header, int16 BE samples 1,2,3,4 and block scale 1; public values `(2,-4),(6,-8)`. Eleven-field procpar headers. | +| `varian-short-header.fid` | Reproduces the old PlotX test's three-field procpar records. Invalid input, not a request to relax the library's format validation. | +| `varian-v0-status.fid` | Full procpar with file version 0 and status 0x11. nmr uses the version-0 complex bit 0x40, so this is scalar. The old PlotX reader treated 0x10 as complex for every version; review that separate issue against real version-0 files. | + +The integration tests verify import policy, coordinate and component preservation, +sampling declarations, and offline snapshot v1 round trips. + +## Sampling-declaration fixture + +`jeol-nus-missing.jdf` is generated by `generate_jeol_nus.py`, using the synthetic +layout exercised in nmr's `tests/unified.rs` and `tests/unified/jeol.rs`. Four +Cartesian planes contain `plane*100 + row*10 + column`, with an indirect grid of +8, four observations, three valid direct points, and no embedded sampling list. +It contains no real vendor data and cannot establish vendor interpretation. diff --git a/crates/io/tests/fixtures/nmr/bruker-1d/acqus b/crates/io/tests/fixtures/nmr/bruker-1d/acqus new file mode 100644 index 00000000..6c5a3fd2 --- /dev/null +++ b/crates/io/tests/fixtures/nmr/bruker-1d/acqus @@ -0,0 +1,13 @@ +##TITLE=PlotX synthetic NMR integration fixture +##$TD= 4 +##$PARMODE= 0 +##$AQ_mod= 3 +##$BYTORDA= 0 +##$DTYPA= 0 +##$SW_h= 4000 +##$SFO1= 400 +##$BF1= 400 +##$O1= 0 +##$NUC1= <1H> +##$GRPDLY= 0 +##END= diff --git a/crates/io/tests/fixtures/nmr/bruker-1d/fid b/crates/io/tests/fixtures/nmr/bruker-1d/fid new file mode 100644 index 0000000000000000000000000000000000000000..7adcc1a0c387bebd3852519f61ed85db3608bbff GIT binary patch literal 16 ScmZQ%U|?VZVrC#_0b&3EU;qmM literal 0 HcmV?d00001 diff --git a/crates/io/tests/fixtures/nmr/bruker-1d/pdata/1/1r b/crates/io/tests/fixtures/nmr/bruker-1d/pdata/1/1r new file mode 100644 index 0000000000000000000000000000000000000000..7adcc1a0c387bebd3852519f61ed85db3608bbff GIT binary patch literal 16 ScmZQ%U|?VZVrC#_0b&3EU;qmM literal 0 HcmV?d00001 diff --git a/crates/io/tests/fixtures/nmr/bruker-1d/pdata/1/procs b/crates/io/tests/fixtures/nmr/bruker-1d/pdata/1/procs new file mode 100644 index 00000000..376f33a4 --- /dev/null +++ b/crates/io/tests/fixtures/nmr/bruker-1d/pdata/1/procs @@ -0,0 +1,10 @@ +##TITLE=PlotX synthetic processed fixture +##$SI= 4 +##$DTYPP= 0 +##$BYTORDP= 0 +##$NC_proc= 1 +##$SW_p= 4000 +##$SF= 400 +##$OFFSET= 10 +##$AXNUC= <1H> +##END= diff --git a/crates/io/tests/fixtures/nmr/bruker-nus/acqu2s b/crates/io/tests/fixtures/nmr/bruker-nus/acqu2s new file mode 100644 index 00000000..5073e16b --- /dev/null +++ b/crates/io/tests/fixtures/nmr/bruker-nus/acqu2s @@ -0,0 +1,10 @@ +##TITLE=PlotX synthetic indirect axis +##$TD= 4 +##$FnMODE= 6 +##$NusTD= 8 +##$SW_h= 1000 +##$SFO1= 100 +##$BF1= 100 +##$O1= 0 +##$NUC1= <13C> +##END= diff --git a/crates/io/tests/fixtures/nmr/bruker-nus/acqus b/crates/io/tests/fixtures/nmr/bruker-nus/acqus new file mode 100644 index 00000000..bb4dd799 --- /dev/null +++ b/crates/io/tests/fixtures/nmr/bruker-nus/acqus @@ -0,0 +1,16 @@ +##TITLE=PlotX synthetic NMR integration fixture +##$TD= 4 +##$PARMODE= 1 +##$AQSEQ= 0 +##$FnTYPE= 2 +##$GO_block_size= +##$AQ_mod= 3 +##$BYTORDA= 0 +##$DTYPA= 0 +##$SW_h= 4000 +##$SFO1= 400 +##$BF1= 400 +##$O1= 0 +##$NUC1= <1H> +##$GRPDLY= 0 +##END= diff --git a/crates/io/tests/fixtures/nmr/bruker-nus/nuslist b/crates/io/tests/fixtures/nmr/bruker-nus/nuslist new file mode 100644 index 00000000..f00580c4 --- /dev/null +++ b/crates/io/tests/fixtures/nmr/bruker-nus/nuslist @@ -0,0 +1,2 @@ +3 +1 diff --git a/crates/io/tests/fixtures/nmr/bruker-nus/ser b/crates/io/tests/fixtures/nmr/bruker-nus/ser new file mode 100644 index 0000000000000000000000000000000000000000..576ebfc54812b51a82b17f60738a1d9ab9351ccd GIT binary patch literal 64 ocmWN=i4A}t5CE_f +##END= diff --git a/crates/io/tests/fixtures/nmr/bruker-states-tppi/acqus b/crates/io/tests/fixtures/nmr/bruker-states-tppi/acqus new file mode 100644 index 00000000..a3defc6b --- /dev/null +++ b/crates/io/tests/fixtures/nmr/bruker-states-tppi/acqus @@ -0,0 +1,16 @@ +##TITLE=PlotX synthetic NMR integration fixture +##$TD= 4 +##$PARMODE= 1 +##$AQSEQ= 0 +##$FnTYPE= 0 +##$GO_block_size= +##$AQ_mod= 3 +##$BYTORDA= 0 +##$DTYPA= 0 +##$SW_h= 4000 +##$SFO1= 400 +##$BF1= 400 +##$O1= 0 +##$NUC1= <1H> +##$GRPDLY= 0 +##END= diff --git a/crates/io/tests/fixtures/nmr/bruker-states-tppi/ser b/crates/io/tests/fixtures/nmr/bruker-states-tppi/ser new file mode 100644 index 0000000000000000000000000000000000000000..576ebfc54812b51a82b17f60738a1d9ab9351ccd GIT binary patch literal 64 ocmWN=i4A}t5CE_f +##END= diff --git a/crates/io/tests/fixtures/nmr/bruker-states/acqus b/crates/io/tests/fixtures/nmr/bruker-states/acqus new file mode 100644 index 00000000..a3defc6b --- /dev/null +++ b/crates/io/tests/fixtures/nmr/bruker-states/acqus @@ -0,0 +1,16 @@ +##TITLE=PlotX synthetic NMR integration fixture +##$TD= 4 +##$PARMODE= 1 +##$AQSEQ= 0 +##$FnTYPE= 0 +##$GO_block_size= +##$AQ_mod= 3 +##$BYTORDA= 0 +##$DTYPA= 0 +##$SW_h= 4000 +##$SFO1= 400 +##$BF1= 400 +##$O1= 0 +##$NUC1= <1H> +##$GRPDLY= 0 +##END= diff --git a/crates/io/tests/fixtures/nmr/bruker-states/ser b/crates/io/tests/fixtures/nmr/bruker-states/ser new file mode 100644 index 0000000000000000000000000000000000000000..576ebfc54812b51a82b17f60738a1d9ab9351ccd GIT binary patch literal 64 ocmWN=i4A}t5CE_fI", blob, offset, value) +struct.pack_into("<4I", blob, 1360, 64, 0, 2, 144) +for index, (name, kind, unit, value) in enumerate([ + ("y_orig_points", 1, 0, 8), ("y_sweep", 2, 13, 1000.0) +]): + offset = 1376 + index * 64 + blob[offset + 6:offset + 8] = bytes([1, unit]) + struct.pack_into(" +##XYDATA=(X++(Y..Y)) +4 1 2 3 4 +##END= diff --git a/crates/io/tests/fixtures/nmr/jcamp-ppm.dx b/crates/io/tests/fixtures/nmr/jcamp-ppm.dx new file mode 100644 index 00000000..0dd1dd08 --- /dev/null +++ b/crates/io/tests/fixtures/nmr/jcamp-ppm.dx @@ -0,0 +1,15 @@ +##TITLE=PlotX synthetic PPM fixture +##JCAMP-DX=5.00 +##DATA TYPE=NMR SPECTRUM +##XUNITS=PPM +##YUNITS=ARBITRARY UNITS +##XFACTOR=1 +##YFACTOR=2 +##FIRSTX=4 +##LASTX=1 +##NPOINTS=4 +##.OBSERVE FREQUENCY=400 +##.OBSERVE NUCLEUS=<1H> +##XYDATA=(X++(Y..Y)) +4 1 2 3 4 +##END= diff --git a/crates/io/tests/fixtures/nmr/jeol-complex.jdf b/crates/io/tests/fixtures/nmr/jeol-complex.jdf new file mode 100644 index 0000000000000000000000000000000000000000..843a8ed8198f5f69c51391b144df8b8d2aa29601 GIT binary patch literal 1440 zcmeZr_4m>9^9^ETWME%L^ArN*GA+yDbEunOC19Ai(fg{Wj zas(eCN5IzmPY@IoNdbi|S-*Mn&E_e5dUAQTdvSgp1QLePs`w|0d@I=St$*T5eU9c` z^a<;U^RHWUk#=5t1MLm`^$m2c>Jv^(ngbvs!qo${piT?y1qD^ZkkD30f>sQq6m4F!A*x^U-4I>nF`ibXfWNS@R0zh5gWt dw4P#xGfZ;LGqgBCDKuZl9`0d@ALwvem`}X6OWyzh literal 0 HcmV?d00001 diff --git a/crates/io/tests/fixtures/nmr/varian-short-header.fid/fid b/crates/io/tests/fixtures/nmr/varian-short-header.fid/fid new file mode 100644 index 0000000000000000000000000000000000000000..6ee1c9d161e8bde06a9ac508cb202cac641f9f01 GIT binary patch literal 68 mcmZQzU|?i`U=|?F1jHObtOCS>K+Fh)AR0Sh1gd9dU;zLRJOE<= literal 0 HcmV?d00001 diff --git a/crates/io/tests/fixtures/nmr/varian-short-header.fid/procpar b/crates/io/tests/fixtures/nmr/varian-short-header.fid/procpar new file mode 100644 index 00000000..5bfa1564 --- /dev/null +++ b/crates/io/tests/fixtures/nmr/varian-short-header.fid/procpar @@ -0,0 +1,18 @@ +np 1 1 +1 4 +0 +sw 1 1 +1 4000 +0 +sfrq 1 1 +1 500 +0 +tof 1 1 +1 2500 +0 +tn 1 2 +1 "H1" +0 +array 1 2 +1 "" +0 diff --git a/crates/io/tests/fixtures/nmr/varian-v0-status.fid/fid b/crates/io/tests/fixtures/nmr/varian-v0-status.fid/fid new file mode 100644 index 0000000000000000000000000000000000000000..6ee1c9d161e8bde06a9ac508cb202cac641f9f01 GIT binary patch literal 68 mcmZQzU|?i`U=|?F1jHObtOCS>K+Fh)AR0Sh1gd9dU;zLRJOE<= literal 0 HcmV?d00001 diff --git a/crates/io/tests/fixtures/nmr/varian-v0-status.fid/procpar b/crates/io/tests/fixtures/nmr/varian-v0-status.fid/procpar new file mode 100644 index 00000000..a1333bf0 --- /dev/null +++ b/crates/io/tests/fixtures/nmr/varian-v0-status.fid/procpar @@ -0,0 +1,18 @@ +np 1 1 32768 0 0 2 1 0 1 64 +1 4 +0 +sw 1 1 32768 0 0 2 1 0 1 64 +1 4000 +0 +sfrq 1 1 32768 0 0 2 1 0 1 64 +1 500 +0 +tof 1 1 32768 0 0 2 1 0 1 64 +1 2500 +0 +tn 1 2 32768 0 0 2 1 0 1 64 +1 "H1" +0 +array 1 2 32768 0 0 2 1 0 1 64 +1 "" +0 diff --git a/crates/io/tests/fixtures/nmr/varian.fid/fid b/crates/io/tests/fixtures/nmr/varian.fid/fid new file mode 100644 index 0000000000000000000000000000000000000000..ee46105e32e307ef9ea19188a92cc21ff0e832eb GIT binary patch literal 68 mcmZQzU|?i`U=|?F1jHObtO67j1dBsxAQKB<1gd9dU;zLRX8>dX literal 0 HcmV?d00001 diff --git a/crates/io/tests/fixtures/nmr/varian.fid/procpar b/crates/io/tests/fixtures/nmr/varian.fid/procpar new file mode 100644 index 00000000..a1333bf0 --- /dev/null +++ b/crates/io/tests/fixtures/nmr/varian.fid/procpar @@ -0,0 +1,18 @@ +np 1 1 32768 0 0 2 1 0 1 64 +1 4 +0 +sw 1 1 32768 0 0 2 1 0 1 64 +1 4000 +0 +sfrq 1 1 32768 0 0 2 1 0 1 64 +1 500 +0 +tof 1 1 32768 0 0 2 1 0 1 64 +1 2500 +0 +tn 1 2 32768 0 0 2 1 0 1 64 +1 "H1" +0 +array 1 2 32768 0 0 2 1 0 1 64 +1 "" +0 diff --git a/crates/io/tests/nmr_bridge.rs b/crates/io/tests/nmr_bridge.rs new file mode 100644 index 00000000..2edb24d4 --- /dev/null +++ b/crates/io/tests/nmr_bridge.rs @@ -0,0 +1,397 @@ +use nmr::{ + DatasetKind, ExecutionContext, + axis::{AxisDomain, AxisUnit}, + raw::GroupDelayState, +}; +use plotx_io::nmr_bridge; +use std::{path::PathBuf, sync::Arc}; + +fn fixture(name: &str) -> PathBuf { + PathBuf::from(env!("CARGO_MANIFEST_DIR")) + .join("tests/fixtures/nmr") + .join(name) +} + +fn read(name: &str) -> Arc { + nmr_bridge::read(&fixture(name), &mut ExecutionContext::default()).unwrap() +} + +#[test] +fn directory_prefers_raw_but_processed_file_selection_is_respected() { + let raw = read("bruker-1d"); + assert_eq!(raw.kind(), DatasetKind::Raw); + let direct = &raw.as_raw().unwrap().descriptor().axes()[0]; + assert!( + matches!(direct.group_delay(), GroupDelayState::Pending(delay) if delay.delay_points() == 0.0) + ); + assert_eq!( + raw.as_raw().unwrap().read_trace(&[]).unwrap().samples(), + &[nmr::Complex64::new(1.0, 2.0), nmr::Complex64::new(3.0, 4.0)] + ); + let processed = read("bruker-1d/pdata/1/1r"); + assert_eq!(processed.kind(), DatasetKind::Processed); + let processed = processed.as_processed().unwrap(); + assert_eq!(processed.descriptor().component_counts(), [1]); + assert_eq!(processed.data().samples(), [2.0, 4.0, 6.0, 8.0]); + assert_eq!( + processed.descriptor().axes()[0] + .coordinate_iter() + .unwrap() + .collect::>(), + [10.0, 7.5, 5.0, 2.5] + ); +} + +#[test] +fn jcamp_keeps_explicit_coordinates_scale_and_scalar_descriptor() { + let input = read("jcamp-hz.dx"); + let processed = input.as_processed().unwrap(); + let axis = &processed.descriptor().axes()[0]; + assert_eq!(axis.domain(), AxisDomain::Frequency); + assert_eq!(axis.unit(), Some(AxisUnit::Hertz)); + assert_eq!(axis.component_count(), 1); + assert_eq!( + axis.coordinate_iter().unwrap().collect::>(), + [4.0, 3.0, 2.0, 1.0] + ); + assert_eq!(processed.data().samples(), [2.0, 4.0, 6.0, 8.0]); + assert_eq!( + nmr_bridge::provenance(&input).unwrap().selected_path, + fixture("jcamp-hz.dx") + ); + assert_eq!( + nmr_bridge::identity(&input).subject.as_deref(), + input.identity().subject() + ); + assert_eq!(nmr_bridge::identity(&input).source_label, "jcamp-hz"); +} + +#[test] +fn ppm_jcamp_keeps_coordinates_without_inventing_observe_frequency() { + let temp = tempfile::tempdir().unwrap(); + let path = temp.path().join("no-observe.dx"); + let text = std::fs::read_to_string(fixture("jcamp-ppm.dx")).unwrap(); + let text = text + .lines() + .filter(|line| !line.starts_with("##.OBSERVE")) + .collect::>() + .join("\n"); + std::fs::write(&path, text).unwrap(); + let input = nmr_bridge::read(&path, &mut ExecutionContext::default()).unwrap(); + let processed = input.as_processed().unwrap(); + let axis = &processed.descriptor().axes()[0]; + assert_eq!(axis.unit(), Some(AxisUnit::Ppm)); + assert_eq!( + axis.frequency_evidence() + .and_then(|e| e.observe_frequency_mhz()), + None + ); + assert_eq!(axis.nucleus(), None); + assert_eq!( + axis.coordinate_iter().unwrap().collect::>(), + [4.0, 3.0, 2.0, 1.0] + ); + assert_eq!(processed.descriptor().component_counts(), [1]); + assert_eq!(processed.data().samples(), [2.0, 4.0, 6.0, 8.0]); +} + +#[test] +fn states_tppi_retains_component_lanes() { + let input = read("bruker-states-tppi"); + let raw = input.as_raw().unwrap(); + assert_eq!(raw.descriptor().logical_shape(), [2, 2]); + assert_eq!(raw.descriptor().component_lanes(), [2, 1]); + let trace = raw.read_trace(&[1]).unwrap(); + assert_eq!( + trace.samples(), + &[ + nmr::Complex64::new(9., 10.), + nmr::Complex64::new(11., 12.), + nmr::Complex64::new(13., 14.), + nmr::Complex64::new(15., 16.) + ] + ); +} + +#[test] +fn states_decoding_keeps_both_lanes_without_tppi_modulation() { + use nmr::processing::{ProcessingOperation, ProcessingPlan}; + let input = read("bruker-states"); + let output = ProcessingPlan::new(vec![ProcessingOperation::ComponentTransform { axis: 0 }]) + .unwrap() + .apply(&input) + .unwrap(); + let data = output.as_dense_processed().unwrap(); + assert_eq!(data.component_counts(), [2, 2]); + for row in 0..2 { + for lane in 0..2 { + for col in 0..2 { + for channel in 0..2 { + let expected = (1 + row * 8 + lane * 4 + col * 2 + channel) as f64; + assert_eq!(data.get(&[row, col], &[lane, channel]).unwrap(), expected); + } + } + } + } +} + +#[test] +fn jeol_policy_does_not_alert_and_zero_imaginary_values_remain_complex() { + let input = read("jeol-complex.jdf"); + assert!(input.warnings().iter().any(|warning| matches!( + warning, + nmr::ReadWarning::ExperimentalVendorSemantics { .. } + ))); + assert!( + !nmr_bridge::warnings(&input) + .iter() + .any(|warning| warning.message.contains("ExperimentalVendorSemantics")) + ); + let raw = input.as_raw().unwrap(); + assert!(matches!( + raw.descriptor().axes()[0].kind(), + nmr::raw::RawAxisKind::Direct(nmr::raw::DirectSamples::Complex) + )); + assert!( + raw.read_trace(&[]) + .unwrap() + .samples() + .iter() + .all(|sample| sample.im == 0.0) + ); +} + +#[test] +fn snapshot_restores_after_source_removal_with_exact_context_and_corruption_checks() { + let temp = tempfile::tempdir().unwrap(); + let path = temp.path().join("source.dx"); + std::fs::copy(fixture("jcamp-hz.dx"), &path).unwrap(); + let input = nmr_bridge::read(&path, &mut ExecutionContext::default()).unwrap(); + let limits = nmr::snapshot::SnapshotLimits::default(); + let mut bytes = Vec::new(); + nmr_bridge::snapshot::write(&input, &mut bytes, limits, &mut ExecutionContext::default()) + .unwrap(); + std::fs::remove_file(path).unwrap(); + let restored = nmr_bridge::snapshot::read( + &mut bytes.as_slice(), + limits, + &mut ExecutionContext::default(), + ) + .unwrap(); + assert_eq!(restored.canonical_digests(), input.canonical_digests()); + assert_eq!(restored.warnings(), input.warnings()); + assert_eq!(restored.sources(), input.sources()); + assert_eq!(restored.selected_path(), input.selected_path()); + assert!(restored.metadata().accepted_archive()); + let mut trailing = bytes.clone(); + trailing.push(0); + assert!( + nmr_bridge::snapshot::read( + &mut trailing.as_slice(), + limits, + &mut ExecutionContext::default() + ) + .is_err() + ); + bytes[30] ^= 1; + assert!( + nmr_bridge::snapshot::read( + &mut bytes.as_slice(), + limits, + &mut ExecutionContext::default() + ) + .is_err() + ); +} + +#[test] +fn cancelled_read_and_snapshot_budget_are_reported() { + let token = nmr::CancellationToken::new(); + token.cancel(); + let error = nmr_bridge::read( + &fixture("bruker-1d"), + &mut ExecutionContext::default().with_cancellation(token), + ) + .unwrap_err(); + assert!( + matches!(error, plotx_io::IoError::Nmr(error) if error.kind() == nmr::ReadErrorKind::Cancelled) + ); + let limits = nmr::snapshot::SnapshotLimits { + max_bytes: 32, + ..Default::default() + }; + assert!( + nmr_bridge::snapshot::write( + &read("bruker-1d"), + &mut Vec::new(), + limits, + &mut ExecutionContext::default() + ) + .is_err() + ); +} + +#[test] +fn varian_keeps_scaling_sign_and_rejects_the_old_simplified_test_header() { + let input = read("varian.fid"); + assert_eq!( + input.as_raw().unwrap().read_trace(&[]).unwrap().samples(), + &[nmr::Complex64::new(2., -4.), nmr::Complex64::new(6., -8.)] + ); + assert!( + matches!(nmr_bridge::read(&fixture("varian-short-header.fid"), &mut ExecutionContext::default()), + Err(plotx_io::IoError::Nmr(error)) if error.kind() == nmr::ReadErrorKind::InvalidMetadata) + ); +} + +#[test] +fn sparse_snapshot_keeps_observation_order_and_missing_points() { + let input = read("bruker-nus"); + assert!( + !nmr_bridge::warnings(&input) + .iter() + .any(|warning| warning.message.contains("ExperimentalVendorSemantics")) + ); + let limits = nmr::snapshot::SnapshotLimits::default(); + let mut bytes = Vec::new(); + nmr_bridge::snapshot::write(&input, &mut bytes, limits, &mut ExecutionContext::default()) + .unwrap(); + let restored = nmr_bridge::snapshot::read( + &mut bytes.as_slice(), + limits, + &mut ExecutionContext::default(), + ) + .unwrap(); + assert_eq!(restored.canonical_digests(), input.canonical_digests()); + let raw = restored.as_raw().unwrap(); + assert!(raw.data().is_sparse()); + assert_eq!( + raw.sampling_schedule() + .unwrap() + .coordinates() + .iter() + .map(|coordinate| coordinate.as_slice()[0]) + .collect::>(), + [3, 1] + ); + assert_eq!( + raw.read_trace(&[0]).unwrap_err().kind(), + nmr::ReadErrorKind::UnsampledCoordinate + ); + assert_eq!( + raw.read_trace(&[3]).unwrap().samples()[0], + nmr::Complex64::new(1., 2.) + ); +} + +#[test] +fn ambiguous_processed_directories_and_truncated_payloads_are_errors() { + let temp = tempfile::tempdir().unwrap(); + for number in [1, 2] { + let path = temp.path().join("pdata").join(number.to_string()); + std::fs::create_dir_all(&path).unwrap(); + for name in ["procs", "1r"] { + std::fs::copy( + fixture(&format!("bruker-1d/pdata/1/{name}")), + path.join(name), + ) + .unwrap(); + } + } + assert!( + matches!(nmr_bridge::read(temp.path(), &mut ExecutionContext::default()), Err(plotx_io::IoError::Nmr(error)) + if error.kind() == nmr::ReadErrorKind::Ambiguous) + ); + let path = temp.path().join("pdata/1/1r"); + std::fs::write(&path, [0u8; 3]).unwrap(); + assert!(nmr_bridge::read(&path, &mut ExecutionContext::default()).is_err()); +} + +#[test] +fn raw_snapshot_reprocesses_without_vendor_files_and_preserves_processed_history() { + let temp = tempfile::tempdir().unwrap(); + for name in ["acqus", "fid"] { + std::fs::copy( + fixture(&format!("bruker-1d/{name}")), + temp.path().join(name), + ) + .unwrap(); + } + let input = nmr_bridge::read(temp.path(), &mut ExecutionContext::default()).unwrap(); + let plan = nmr::processing::ProcessingPlan::new(vec![ + nmr::processing::ProcessingOperation::FourierTransform { + axis: 0, + transform: nmr::processing::FourierTransform::default(), + }, + ]) + .unwrap(); + let expected = plan.apply(&input).unwrap(); + let limits = nmr::snapshot::SnapshotLimits::default(); + let mut bytes = Vec::new(); + nmr_bridge::snapshot::write(&input, &mut bytes, limits, &mut ExecutionContext::default()) + .unwrap(); + temp.close().unwrap(); + let restored = nmr_bridge::snapshot::read( + &mut bytes.as_slice(), + limits, + &mut ExecutionContext::default(), + ) + .unwrap(); + let processed = plan.apply(&restored).unwrap(); + assert_eq!(processed.canonical_digests(), expected.canonical_digests()); + assert_eq!( + processed.as_dense_processed(), + expected.as_dense_processed() + ); + bytes.clear(); + nmr_bridge::snapshot::write( + &processed, + &mut bytes, + limits, + &mut ExecutionContext::default(), + ) + .unwrap(); + let archived = nmr_bridge::snapshot::read( + &mut bytes.as_slice(), + limits, + &mut ExecutionContext::default(), + ) + .unwrap(); + let history = archived + .as_processed() + .unwrap() + .provenance() + .history() + .unwrap(); + let replay = history + .replay_raw( + restored.as_raw().unwrap(), + nmr::processing::ProcessingOptions::new(), + ) + .unwrap(); + assert_eq!(replay.data(), processed.as_dense_processed().unwrap()); +} + +#[test] +fn supported_jcamp_versions_preserve_ppm_coordinates_and_scaling() { + let dir = tempfile::tempdir().unwrap(); + let text = std::fs::read_to_string(fixture("jcamp-ppm.dx")).unwrap(); + for version in ["5.00", "5.01"] { + let path = dir.path().join(format!("v{version}.dx")); + std::fs::write( + &path, + text.replace("##JCAMP-DX=5.00", &format!("##JCAMP-DX={version}")), + ) + .unwrap(); + let source = nmr_bridge::read(&path, &mut ExecutionContext::default()).unwrap(); + let processed = source.as_processed().unwrap(); + assert_eq!( + processed.descriptor().axes()[0] + .coordinate_iter() + .unwrap() + .collect::>(), + [4.0, 3.0, 2.0, 1.0] + ); + assert_eq!(processed.data().samples(), [2.0, 4.0, 6.0, 8.0]); + } +} diff --git a/crates/io/tests/nmr_group_delay.rs b/crates/io/tests/nmr_group_delay.rs new file mode 100644 index 00000000..67db1b04 --- /dev/null +++ b/crates/io/tests/nmr_group_delay.rs @@ -0,0 +1,105 @@ +//! Original valid vendor-delay test inputs through the unified public reader. +use nmr::raw::GroupDelayState; +use plotx_io::nmr_bridge; +use std::{path::PathBuf, sync::Arc}; + +fn fixture(name: &str) -> PathBuf { + PathBuf::from(env!("CARGO_MANIFEST_DIR")) + .join("tests/fixtures/nmr") + .join(name) +} + +fn delay(input: &nmr::Dataset) -> f64 { + match input.as_raw().unwrap().descriptor().axes()[0].group_delay() { + GroupDelayState::Pending(value) => value.delay_points(), + other => panic!("expected known delay, got {other:?}"), + } +} + +#[test] +fn group_delay_prefers_the_original_explicit_grpdly() { + let dir = tempfile::tempdir().unwrap(); + std::fs::copy(fixture("bruker-1d/fid"), dir.path().join("fid")).unwrap(); + let text = std::fs::read_to_string(fixture("bruker-1d/acqus")) + .unwrap() + .replace( + "##$GRPDLY= 0", + "##$GRPDLY= 67.98\n##$DSPFVS= 21\n##$DECIM= 2080", + ); + std::fs::write(dir.path().join("acqus"), text).unwrap(); + let input = nmr_bridge::read(dir.path(), &mut nmr::ExecutionContext::default()).unwrap(); + assert!((delay(&input) - 67.98).abs() < 1e-9); +} + +#[test] +fn group_delay_falls_back_to_table() { + let dir = tempfile::tempdir().unwrap(); + std::fs::copy(fixture("bruker-1d/fid"), dir.path().join("fid")).unwrap(); + let text = std::fs::read_to_string(fixture("bruker-1d/acqus")) + .unwrap() + .replace("##$GRPDLY= 0", "##$GRPDLY= -1\n##$DSPFVS= 12\n##$DECIM= 16"); + std::fs::write(dir.path().join("acqus"), text).unwrap(); + let input = nmr_bridge::read(dir.path(), &mut nmr::ExecutionContext::default()).unwrap(); + // Retain the original Bruker parser test's input and numerical contract. + // The reader owns hardware lookup; PlotX must not implement a second table. + let actual = input.as_raw().unwrap().descriptor().axes()[0].group_delay(); + assert!( + matches!(actual, GroupDelayState::Pending(value) if (value.delay_points() - 71.625).abs() < 1e-9), + "GRPDLY=-1 / DSPFVS=12 / DECIM=16 must resolve to 71.625 points; got {actual:?}" + ); +} + +fn jeol_filter(parameters: &[(&str, &str)]) -> Arc { + let original = std::fs::read(fixture("jeol-complex.jdf")).unwrap(); + let old_start = u32::from_be_bytes(original[1284..1288].try_into().unwrap()) as usize; + let parameter_bytes = 16 + 64 * parameters.len(); + let data_start = 1360 + parameter_bytes; + let mut bytes = vec![0; data_start]; + bytes[..1360].copy_from_slice(&original[..1360]); + bytes[1212..1216].copy_from_slice(&1360u32.to_be_bytes()); + bytes[1216..1220].copy_from_slice(&(parameter_bytes as u32).to_be_bytes()); + bytes[1284..1288].copy_from_slice(&(data_start as u32).to_be_bytes()); + for (offset, value) in [ + (1360, 64), + (1364, 0), + (1368, parameters.len()), + (1372, parameter_bytes), + ] { + bytes[offset..offset + 4].copy_from_slice(&(value as u32).to_le_bytes()); + } + for (index, (name, value)) in parameters.iter().enumerate() { + let start = 1376 + 64 * index; + bytes[start + 16..start + 16 + value.len()].copy_from_slice(value.as_bytes()); + bytes[start + 36..start + 36 + name.len()].copy_from_slice(name.as_bytes()); + } + bytes.extend_from_slice(&original[old_start..]); + let dir = tempfile::tempdir().unwrap(); + let path = dir.path().join("filter.jdf"); + std::fs::write(&path, bytes).unwrap(); + nmr_bridge::read(&path, &mut nmr::ExecutionContext::default()).unwrap() +} + +#[test] +fn group_delay_from_the_original_jeol_fir_cascades() { + for (orders, factors, expected) in [ + ("2 41 74", "6 2", 239.0 / 12.0), + ("2 15 73", "2 2", 19.75), + ] { + let input = jeol_filter(&[ + ("digital_filter", "TRUE"), + ("orders", orders), + ("factors", factors), + ]); + let g = delay(&input); + assert!((g - expected).abs() < 1e-9, "got {g}"); + } + let disabled = jeol_filter(&[ + ("digital_filter", "FALSE"), + ("orders", "2 41 74"), + ("factors", "6 2"), + ]); + assert_eq!( + disabled.as_raw().unwrap().descriptor().axes()[0].group_delay(), + &GroupDelayState::NotApplicable + ); +} diff --git a/crates/io/tests/nmr_sampling.rs b/crates/io/tests/nmr_sampling.rs new file mode 100644 index 00000000..e11769e4 --- /dev/null +++ b/crates/io/tests/nmr_sampling.rs @@ -0,0 +1,149 @@ +use nmr::ExecutionContext; +use plotx_io::{ + nmr_bridge, + nmr_sampling::{self, IndexBase, SamplingDeclaration}, +}; +use std::path::{Path, PathBuf}; + +fn fixture(name: &str) -> PathBuf { + PathBuf::from(env!("CARGO_MANIFEST_DIR")) + .join("tests/fixtures/nmr") + .join(name) +} + +fn declaration(grid: usize, rows: &[usize]) -> SamplingDeclaration { + SamplingDeclaration { + assertion_id: "plotx-test-schedule".into(), + source: "synthetic user table".into(), + grid_shape: vec![grid], + coordinates: rows.iter().map(|&row| vec![row]).collect(), + index_base: IndexBase::One, + component_counts: vec![2], + } +} + +fn read(path: &Path, declaration: SamplingDeclaration) -> std::sync::Arc { + nmr_sampling::read(path, declaration, &mut ExecutionContext::default()).unwrap() +} + +#[test] +fn bruker_declaration_checks_vendor_evidence_and_preserves_duplicate_observations_offline() { + let dir = tempfile::tempdir().unwrap(); + for name in ["ser", "acqus", "acqu2s"] { + std::fs::copy( + fixture(&format!("bruker-nus/{name}")), + dir.path().join(name), + ) + .unwrap(); + } + assert!(nmr_bridge::load(dir.path()).is_err()); + let declared = declaration(4, &[4, 2]); + let input = read(dir.path(), declared.clone()); + let raw = input.as_raw().unwrap(); + let schedule = raw.sampling_schedule().unwrap(); + assert_eq!( + schedule.declaration(), + Some(&declared.clone().into_native().unwrap()) + ); + assert_eq!( + schedule + .coordinates() + .iter() + .map(|c| c.as_slice()) + .collect::>(), + [&[3], &[1]] + ); + assert!(!dir.path().join("nuslist").exists()); + for name in ["ser", "acqus", "acqu2s"] { + assert_eq!( + std::fs::read(dir.path().join(name)).unwrap(), + std::fs::read(fixture(&format!("bruker-nus/{name}"))).unwrap() + ); + } + let mut bad_lanes = declared.clone(); + bad_lanes.component_counts = vec![1]; + for invalid in [ + declaration(5, &[4, 2]), + declaration(4, &[5, 2]), + declaration(4, &[0, 2]), + declaration(4, &[4]), + bad_lanes, + ] { + assert!(nmr_sampling::load(dir.path(), invalid).is_err()); + } + let repeated = read(dir.path(), declaration(4, &[2, 2])); + let traces = repeated.as_raw().unwrap().data().sparse_traces().unwrap(); + assert_eq!(traces[0].coordinate(), traces[1].coordinate()); + assert_ne!(traces[0].samples(), traces[1].samples()); + let mut bytes = Vec::new(); + nmr_bridge::snapshot::write( + &repeated, + &mut bytes, + Default::default(), + &mut ExecutionContext::default(), + ) + .unwrap(); + dir.close().unwrap(); + let restored = nmr_bridge::snapshot::read( + &mut bytes.as_slice(), + Default::default(), + &mut ExecutionContext::default(), + ) + .unwrap(); + assert_eq!(restored.canonical_digests(), repeated.canonical_digests()); + assert_eq!( + restored.as_raw().unwrap().sampling_schedule(), + repeated.as_raw().unwrap().sampling_schedule() + ); + assert!(nmr_sampling::load(&fixture("bruker-nus"), declared).is_ok()); + assert!(nmr_sampling::load(&fixture("bruker-nus"), declaration(4, &[2, 4])).is_err()); + assert!(nmr_sampling::load(&fixture("bruker-1d/pdata/1/1r"), declaration(4, &[4, 2])).is_err()); +} + +#[test] +fn jeol_declaration_reaches_checked_reader_with_four_components_and_no_source_edits() { + let path = fixture("jeol-nus-missing.jdf"); + let before = std::fs::read(&path).unwrap(); + assert!(nmr_bridge::load(&path).is_err()); + let input = read(&path, declaration(8, &[1, 2, 2, 8])); + assert!( + !nmr_bridge::warnings(&input) + .iter() + .any(|warning| warning.message.contains("ExperimentalVendorSemantics")) + ); + let raw = input.as_raw().unwrap(); + assert_eq!(raw.descriptor().logical_shape(), [8, 3]); + let traces = raw.data().sparse_traces().unwrap(); + assert_eq!(traces[1].coordinate(), traces[2].coordinate()); + assert_ne!(traces[1].samples(), traces[2].samples()); + // The F1 imaginary lane uses the opposite quadrature orientation. + assert_eq!( + traces[0].samples(), + &[ + nmr::Complex64::new(0.0, -100.0), + nmr::Complex64::new(1.0, -101.0), + nmr::Complex64::new(2.0, -102.0), + nmr::Complex64::new(-200.0, 300.0), + nmr::Complex64::new(-201.0, 301.0), + nmr::Complex64::new(-202.0, 302.0), + ] + ); + assert_eq!(std::fs::read(path).unwrap(), before); +} + +#[test] +fn declaration_json_rejects_unknown_fields_and_cancelled_read() { + let dir = tempfile::tempdir().unwrap(); + let path = dir.path().join("schedule.json"); + let declared = declaration(4, &[4, 2]); + std::fs::write(&path, serde_json::to_vec(&declared).unwrap()).unwrap(); + assert!(nmr_sampling::load_with_declaration_file(&fixture("bruker-nus"), &path).is_ok()); + let mut json = serde_json::to_value(&declared).unwrap(); + json["index_bsae"] = "one".into(); + std::fs::write(&path, serde_json::to_vec(&json).unwrap()).unwrap(); + assert!(nmr_sampling::read_declaration(&path).is_err()); + let token = nmr::CancellationToken::new(); + token.cancel(); + let mut context = ExecutionContext::default().with_cancellation(token); + assert!(nmr_sampling::read(&fixture("bruker-nus"), declared, &mut context).is_err()); +} diff --git a/crates/io/tests/nmr_view.rs b/crates/io/tests/nmr_view.rs new file mode 100644 index 00000000..c52f4828 --- /dev/null +++ b/crates/io/tests/nmr_view.rs @@ -0,0 +1,193 @@ +use nmr::{ExecutionContext, axis::AxisUnit}; +use plotx_io::{nmr_bridge, nmr_view::NmrSource}; +use std::path::PathBuf; + +fn fixture(name: &str) -> PathBuf { + PathBuf::from(env!("CARGO_MANIFEST_DIR")) + .join("tests/fixtures/nmr") + .join(name) +} + +#[test] +fn ppm_view_preserves_missing_metadata_and_scalar_components() { + let dir = tempfile::tempdir().unwrap(); + let path = dir.path().join("spectrum.dx"); + let text = std::fs::read_to_string(fixture("jcamp-ppm.dx")).unwrap(); + let text = text + .lines() + .filter(|line| !line.starts_with("##.OBSERVE")) + .collect::>() + .join("\n"); + std::fs::write(&path, text).unwrap(); + let source = + NmrSource::new(nmr_bridge::read(&path, &mut ExecutionContext::default()).unwrap()).unwrap(); + assert_eq!(source.axes()[0].observe_frequency_mhz(), None); + assert_eq!(source.axes()[0].unit, Some(AxisUnit::Ppm)); + assert_eq!( + source.axes()[0].coordinate_values().unwrap(), + [4., 3., 2., 1.] + ); + assert!(!source.has_imaginary(0)); + assert_eq!( + source + .trace() + .unwrap() + .iter() + .map(|v| v.re) + .collect::>(), + [2., 4., 6., 8.] + ); + assert!(source.craft_fid().is_err()); +} + +#[test] +fn craft_view_requires_delay_evidence_and_keeps_explicit_zero() { + let dir = tempfile::tempdir().unwrap(); + let text = std::fs::read_to_string(fixture("bruker-1d/acqus")).unwrap(); + std::fs::copy(fixture("bruker-1d/fid"), dir.path().join("fid")).unwrap(); + std::fs::write(dir.path().join("acqus"), &text).unwrap(); + let source = + NmrSource::new(nmr_bridge::read(dir.path(), &mut ExecutionContext::default()).unwrap()) + .unwrap(); + let fid = source.craft_fid().unwrap(); + assert_eq!(fid.group_delay, 0.0); + assert_eq!(fid.observe_freq_mhz, 400.0); + assert_eq!(fid.points, source.trace().unwrap()); + let text = text + .lines() + .filter(|line| !line.starts_with("##$GRPDLY")) + .collect::>() + .join("\n"); + std::fs::write(dir.path().join("acqus"), text).unwrap(); + let source = + NmrSource::new(nmr_bridge::read(dir.path(), &mut ExecutionContext::default()).unwrap()) + .unwrap(); + assert!( + source + .craft_fid() + .unwrap_err() + .to_string() + .contains("delay evidence") + ); +} + +#[test] +fn all_zero_imaginary_channel_is_still_present() { + let source = NmrSource::new( + nmr_bridge::read( + &fixture("jeol-complex.jdf"), + &mut ExecutionContext::default(), + ) + .unwrap(), + ) + .unwrap(); + assert!(source.has_imaginary(0)); + assert!(source.trace().unwrap().iter().all(|value| value.im == 0.0)); +} + +#[test] +fn reference_frequency_survives_binning_without_becoming_observe_frequency() { + use nmr::axis::{AxisCoordinates, AxisDomain, AxisRole, FrequencyEvidence}; + use nmr::processed::{ + ComponentBasis, ProcessedAxis, ProcessedDataset, ProcessedOrigin, ProcessedProvenance, + }; + use nmr::processing::{ + BinAggregation, FrequencyFrame, ProcessingOperation as Op, ProcessingPlan, ReferenceSource, + SpectrumOperation, + }; + let axis = ProcessedAxis::new( + AxisRole::Signal, + AxisDomain::Frequency, + Some(AxisUnit::Hertz), + 8, + AxisCoordinates::Uniform { + start: -4.0, + step: 1.0, + }, + ComponentBasis::Cartesian, + ) + .unwrap() + .with_frequency_evidence(Some(FrequencyEvidence::new(Some(500.005), None).unwrap())) + .unwrap(); + let input = ProcessedDataset::from_complex_trace( + axis, + vec![nmr::Complex64::new(1.0, 2.0); 8], + ProcessedProvenance::new(ProcessedOrigin::Unknown, vec![]).unwrap(), + ) + .unwrap(); + let output = ProcessingPlan::new(vec![ + Op::ResolveFrequencyFrame { + axis: 0, + frame: FrequencyFrame::Ppm(ReferenceSource::Explicit( + nmr::raw::ChemicalShiftReference::user_constructed(10.0, 500.0).unwrap(), + )), + }, + Op::Spectrum { + axis: 0, + operation: SpectrumOperation::Bin { + width: 0.004, + aggregation: BinAggregation::Mean, + }, + }, + ]) + .unwrap() + .apply(&input.into()) + .unwrap(); + let source = NmrSource::new(std::sync::Arc::new(output)).unwrap(); + assert_eq!(source.axes()[0].observe_frequency_mhz(), Some(500.005)); + assert_eq!(source.reference_frequency_mhz(0), Some(500.0)); + assert_eq!(source.len(), 4); + let coordinates = source.axes()[0].coordinate_values().unwrap(); + assert!((coordinates[1] - coordinates[0] - 0.004).abs() < 1e-12); + let mut bytes = Vec::new(); + nmr_bridge::snapshot::write( + source.dataset(), + &mut bytes, + Default::default(), + &mut ExecutionContext::default(), + ) + .unwrap(); + let restored = NmrSource::new( + nmr_bridge::snapshot::read( + &mut bytes.as_slice(), + Default::default(), + &mut ExecutionContext::default(), + ) + .unwrap(), + ) + .unwrap(); + assert_eq!(restored.reference_frequency_mhz(0), Some(500.0)); + assert_eq!(restored.axes()[0].observe_frequency_mhz(), Some(500.005)); + assert_eq!(restored.axes()[0].coordinate_values().unwrap(), coordinates); +} + +#[test] +fn processed_sf_is_reference_evidence_without_observe_carrier_or_filter_claims() { + let source = NmrSource::new( + nmr_bridge::read( + &fixture("bruker-1d/pdata/1/1r"), + &mut ExecutionContext::default(), + ) + .unwrap(), + ) + .unwrap(); + assert_eq!(source.reference_frequency_mhz(0), Some(400.0)); + assert_eq!(source.axes()[0].observe_frequency_mhz(), None); + assert_eq!(source.reference_frequency_mhz(1), None); + let evidence = source + .dataset() + .as_processed() + .unwrap() + .axis_evidence(0) + .unwrap(); + assert!(evidence.reference_evidence().is_some()); + assert!(evidence.chemical_shift_reference().is_none()); + assert_eq!( + evidence.group_delay(), + nmr::processed::ProcessedGroupDelay::Unknown + ); + assert_eq!( + source.axes()[0].coordinate_values().unwrap(), + [10.0, 7.5, 5.0, 2.5] + ); +} diff --git a/crates/processing/Cargo.toml b/crates/processing/Cargo.toml index fb9f0fef..f6f1b9b0 100644 --- a/crates/processing/Cargo.toml +++ b/crates/processing/Cargo.toml @@ -11,9 +11,9 @@ name = "plotx_processing" path = "src/lib.rs" [dependencies] +nmr.workspace = true plotx-io.workspace = true plotx-analysis.workspace = true num-complex.workspace = true -rustfft.workspace = true thiserror.workspace = true serde.workspace = true diff --git a/crates/processing/src/arithmetic.rs b/crates/processing/src/arithmetic.rs index 830cc815..b7f424b7 100644 --- a/crates/processing/src/arithmetic.rs +++ b/crates/processing/src/arithmetic.rs @@ -1,9 +1,10 @@ -//! Spectrum arithmetic: dataset ± dataset (with a scale on the second operand) -//! and constant scale/offset, producing a new standalone frequency-domain trace. - +//! Spectrum arithmetic executes in nmr and retains both parents for replay. use crate::Spectrum; -use num_complex::Complex64; -use std::fmt; +use nmr::processing::{ + LinearCombination, ProcessingOperation, ProcessingOptions, ProcessingPlan, SpectrumOperation, +}; +use plotx_io::nmr_view::NmrSource; +use std::sync::Arc; #[derive(Debug, Clone, Copy, PartialEq, Eq)] pub enum SpectrumBinaryOp { @@ -20,124 +21,115 @@ impl SpectrumBinaryOp { } } -#[derive(Debug, Clone, PartialEq, Eq)] +#[derive(Debug, thiserror::Error)] pub enum ArithmeticError { - NucleusMismatch { a: String, b: String }, - EmptyOperand, -} - -impl fmt::Display for ArithmeticError { - fn fmt(&self, formatter: &mut fmt::Formatter<'_>) -> fmt::Result { - match self { - Self::NucleusMismatch { a, b } => { - write!( - formatter, - "Nuclei differ ({a} vs {b}); pick two spectra of the same nucleus." - ) - } - Self::EmptyOperand => formatter.write_str("Both spectra need at least one point."), - } - } + #[error(transparent)] + Library(#[from] nmr::processing::ProcessingError), + #[error(transparent)] + View(#[from] plotx_io::IoError), } -impl std::error::Error for ArithmeticError {} - pub fn same_grid(a: &Spectrum, b: &Spectrum) -> bool { - a.ppm.len() == b.ppm.len() + a.unit == b.unit + && a.ppm.len() == b.ppm.len() && a.ppm .iter() .zip(&b.ppm) .all(|(x, y)| (x - y).abs() <= 1e-9 * x.abs().max(y.abs()).max(1.0)) } -/// `a op k·b` on `a`'s axis. `b` is linearly interpolated onto `a`'s grid; -/// points of `a` outside `b`'s range use `b = 0`. +pub fn validate_combination(a: &NmrSource, b: &NmrSource) -> Result<(), ArithmeticError> { + LinearCombination::new(1.0)?.prepare(a.dataset(), b.dataset(), ProcessingOptions::default())?; + Ok(()) +} + pub fn combine_spectra( - a: &Spectrum, - b: &Spectrum, + a: &NmrSource, + b: &NmrSource, op: SpectrumBinaryOp, k: f64, -) -> Result { - if a.is_empty() || b.is_empty() { - return Err(ArithmeticError::EmptyOperand); - } - if a.nucleus.trim() != b.nucleus.trim() { - return Err(ArithmeticError::NucleusMismatch { - a: a.nucleus.clone(), - b: b.nucleus.clone(), - }); - } +) -> Result { let scale = match op { SpectrumBinaryOp::Add => k, SpectrumBinaryOp::Subtract => -k, }; - let b_on_a = if same_grid(a, b) { - b.values.clone() - } else { - resample_linear(&b.ppm, &b.values, &a.ppm) - }; - let mut out = a.clone(); - for (v, w) in out.values.iter_mut().zip(&b_on_a) { - *v += scale * w; - } - Ok(out) -} - -/// `scale·a + offset` (the offset raises the real channel only). -pub fn scale_offset_spectrum(a: &Spectrum, scale: f64, offset: f64) -> Spectrum { - let mut out = a.clone(); - for v in &mut out.values { - *v = scale * *v + Complex64::new(offset, 0.0); - } - out + let output = LinearCombination::new(scale)? + .prepare(a.dataset(), b.dataset(), ProcessingOptions::default())? + .execute_with_context(&mut nmr::ExecutionContext::default())?; + Ok(NmrSource::new(Arc::new(output))?) } -fn resample_linear(src_ppm: &[f64], src: &[Complex64], dst_ppm: &[f64]) -> Vec { - let ascending = src_ppm.first() <= src_ppm.last(); - let (axis, values): (Vec, Vec) = if ascending { - (src_ppm.to_vec(), src.to_vec()) - } else { - ( - src_ppm.iter().rev().copied().collect(), - src.iter().rev().copied().collect(), - ) - }; - dst_ppm - .iter() - .map(|&x| { - let (lo, hi) = (axis[0], axis[axis.len() - 1]); - if x < lo || x > hi { - return Complex64::new(0.0, 0.0); - } - let j = axis.partition_point(|&p| p < x); - if j == 0 { - return values[0]; - } - if j >= axis.len() { - return values[values.len() - 1]; - } - let (x0, x1) = (axis[j - 1], axis[j]); - let span = x1 - x0; - if span <= 0.0 { - return values[j]; - } - let t = (x - x0) / span; - values[j - 1] * (1.0 - t) + values[j] * t - }) - .collect() +pub fn scale_offset_spectrum( + a: &NmrSource, + scale: f64, + offset: f64, +) -> Result { + let output = ProcessingPlan::new(vec![ProcessingOperation::Spectrum { + axis: 0, + operation: SpectrumOperation::Affine { + scale, + real_offset: offset, + }, + }])? + .apply(a.dataset())?; + Ok(NmrSource::new(Arc::new(output))?) } #[cfg(test)] mod tests { use super::*; + use num_complex::Complex64; + + fn input(spec: &Spectrum) -> Result { + use nmr::{axis::*, processed::*}; + let fail = + |error: &dyn std::fmt::Display| plotx_io::IoError::NmrConversion(error.to_string()); + let axis = ProcessedAxis::new( + AxisRole::Signal, + AxisDomain::Frequency, + Some(spec.unit), + spec.len(), + AxisCoordinates::Explicit(spec.ppm.clone()), + ComponentBasis::Cartesian, + ) + .map_err(|e| fail(&e))? + .with_nucleus(Some(spec.nucleus.clone())) + .map_err(|e| fail(&e))?; + let data = ProcessedDataset::from_complex_trace( + axis, + spec.values.clone(), + ProcessedProvenance::new(ProcessedOrigin::Unknown, vec![]).map_err(|e| fail(&e))?, + ) + .map_err(|e| fail(&e))?; + Ok(NmrSource::new(Arc::new(data.into()))?) + } + fn view(source: NmrSource) -> Spectrum { + crate::nmr_execution::view_1d(&source) + .unwrap() + .as_frequency() + .unwrap() + .clone() + } + fn combine_spectra( + a: &Spectrum, + b: &Spectrum, + op: SpectrumBinaryOp, + k: f64, + ) -> Result { + super::combine_spectra(&input(a)?, &input(b)?, op, k).map(view) + } + fn scale_offset_spectrum(a: &Spectrum, scale: f64, offset: f64) -> Spectrum { + view(super::scale_offset_spectrum(&input(a).unwrap(), scale, offset).unwrap()) + } fn spec(ppm: Vec, re: Vec, nucleus: &str) -> Spectrum { let values = re.into_iter().map(|r| Complex64::new(r, 0.0)).collect(); Spectrum { ppm, values, - hz_per_point: 1.0, - observe_freq_mhz: 400.0, + unit: nmr::axis::AxisUnit::Ppm, + hz_per_point: Some(1.0), + observe_freq_mhz: Some(400.0), nucleus: nucleus.into(), } } @@ -194,17 +186,14 @@ mod tests { let a = spec(vec![0.0, 1.0], vec![1.0, 1.0], "1H"); let b = spec(vec![0.0, 1.0], vec![1.0, 1.0], "13C"); let err = combine_spectra(&a, &b, SpectrumBinaryOp::Add, 1.0).unwrap_err(); - assert!(matches!(err, ArithmeticError::NucleusMismatch { .. })); + assert!(err.to_string().contains("incompatible")); } #[test] fn empty_operand_is_rejected() { let a = spec(vec![], vec![], "1H"); let b = spec(vec![0.0], vec![1.0], "1H"); - assert!(matches!( - combine_spectra(&a, &b, SpectrumBinaryOp::Add, 1.0), - Err(ArithmeticError::EmptyOperand) - )); + assert!(combine_spectra(&a, &b, SpectrumBinaryOp::Add, 1.0).is_err()); } #[test] diff --git a/crates/processing/src/autophase.rs b/crates/processing/src/autophase.rs deleted file mode 100644 index 9a285c69..00000000 --- a/crates/processing/src/autophase.rs +++ /dev/null @@ -1,478 +0,0 @@ -//! Automatic phase determination beyond the single tallest-peak rule. - -use crate::AutoPhaseMethod; -use crate::phase; -use num_complex::Complex64; -use plotx_analysis::peaks::{DetectParams, detect_peaks, estimate_noise}; -use std::f64::consts::{PI, TAU}; - -/// Cap on the point count fed to the iterative optimizers. -const SEARCH_POINTS: usize = 1024; - -/// Weight on the negative-intensity penalty in the ACME entropy cost, applied to -/// a spectrum normalized to unit peak magnitude. Large enough to break the 180° -/// sign ambiguity the derivative entropy cannot see, following Chen et al. -const ACME_PENALTY: f64 = 1000.0; - -pub fn compute(values: &[Complex64], method: AutoPhaseMethod) -> (f64, f64, f64) { - match method { - AutoPhaseMethod::RobustConsensus => robust_consensus(values), - AutoPhaseMethod::AbsorptivePeak => absorptive_peak(values), - AutoPhaseMethod::Entropy => optimized(values, acme_cost), - AutoPhaseMethod::NegativeMinimization => optimized(values, negative_cost), - AutoPhaseMethod::PeakRegression => peak_regression(values), - } -} - -/// Build candidates with deliberately different failure modes and select among -/// them using a scale-independent objective. The winning candidate is refined -/// once more against that common objective. Testing each candidate's pi-shifted -/// counterpart makes the sign decision explicit. -fn robust_consensus(values: &[Complex64]) -> (f64, f64, f64) { - let (mut dec, frac) = decimate(values, SEARCH_POINTS); - if dec.len() < 4 { - return absorptive_peak(values); - } - let scale = dec.iter().map(|c| c.norm()).fold(0.0_f64, f64::max); - if !scale.is_finite() || scale <= f64::MIN_POSITIVE { - return (0.0, 0.0, phase::peak_pivot_frac(values)); - } - for value in &mut dec { - *value /= scale; - } - let strategies = [ - absorptive_peak(values), - optimized(values, acme_cost), - optimized(values, negative_cost), - peak_regression(values), - ]; - // Every candidate is judged with its zero-order phase pinned so the tallest - // peak is absorptive, leaving `p1` (the ramp) as the only free variable. This - // resolves the isolated-peak case — where the dominant bin stays positive at - // any orientation, so entropy and negative power alone are degenerate — while - // still letting negative power rank the ramp on overlapping spectra. - let objective = |p1: f64| consensus_cost(&dec, &frac, snap_zero_order_to_peak(values, p1), p1); - let mut best_p1 = 0.0; - let mut best_cost = f64::INFINITY; - for (_, phase1, _) in strategies { - let cost = objective(phase1); - if cost < best_cost { - best_p1 = phase1; - best_cost = cost; - } - } - let p1 = pattern_search_1d(&objective, best_p1); - to_pivoted(values, snap_zero_order_to_peak(values, p1), p1) -} - -/// Zero-order phase that lands the tallest bin on the positive real axis given -/// the ramp `p1`. Referencing the consensus ramp to the peak's own argument keeps -/// a clean isolated peak exactly absorptive, which the global objective — flat to -/// a degree or so near its optimum — cannot pin on its own. -fn snap_zero_order_to_peak(values: &[Complex64], p1: f64) -> f64 { - let Some((index, peak)) = values - .iter() - .enumerate() - .max_by(|a, b| a.1.norm().total_cmp(&b.1.norm())) - else { - return 0.0; - }; - if peak.norm() <= f64::MIN_POSITIVE { - return 0.0; - } - let peak_frac = index as f64 / (values.len() - 1).max(1) as f64; - peak.arg() - p1 * peak_frac -} - -/// Normalized derivative entropy rewards sharp absorptive lines while negative -/// power resolves the remaining sign and ramp ambiguity. With the caller pinning -/// the tallest peak absorptive, negative power is what separates a correct ramp -/// (every peak upright) from a wrong one. Both terms are dimensionless, so ranking -/// is invariant under spectrum intensity scaling. -fn consensus_cost(values: &[Complex64], frac: &[f64], p0: f64, p1: f64) -> f64 { - let real = phased_real(values, frac, p0, p1); - let derivatives: Vec = real.windows(2).map(|w| (w[1] - w[0]).abs()).collect(); - let derivative_sum: f64 = derivatives.iter().sum(); - if derivative_sum <= f64::MIN_POSITIVE { - return f64::INFINITY; - } - let entropy = derivatives.iter().fold(0.0, |acc, derivative| { - let probability = derivative / derivative_sum; - if probability > 0.0 { - acc - probability * probability.ln() - } else { - acc - } - }); - let max_entropy = (derivatives.len().max(2) as f64).ln(); - entropy / max_entropy + 4.0 * negative_cost(values, frac, p0, p1) -} - -/// Zeroth-order only: rotate the tallest peak onto the positive real axis. -fn absorptive_peak(values: &[Complex64]) -> (f64, f64, f64) { - let pivot = phase::peak_pivot_frac(values); - let peak = values - .iter() - .max_by(|a, b| a.norm().total_cmp(&b.norm())) - .copied() - .unwrap_or(Complex64::new(0.0, 0.0)); - (peak.arg(), 0.0, pivot) -} - -/// Grid-seeded pattern search over `(phase0, phase1)` on a decimated, unit-peak -/// spectrum, minimizing `cost`. Falls back to the zero-order rule when there are -/// too few points to fit a ramp. -fn optimized( - values: &[Complex64], - cost: fn(&[Complex64], &[f64], f64, f64) -> f64, -) -> (f64, f64, f64) { - let (mut dec, frac) = decimate(values, SEARCH_POINTS); - if dec.len() < 4 { - return absorptive_peak(values); - } - let m = dec.iter().map(|c| c.norm()).fold(0.0_f64, f64::max); - if m <= 0.0 { - return (0.0, 0.0, phase::peak_pivot_frac(values)); - } - for c in &mut dec { - *c /= m; - } - let obj = |p0: f64, p1: f64| cost(&dec, &frac, p0, p1); - let (p0, p1) = coarse_grid(&obj); - let (p0, p1) = pattern_search(&obj, p0, p1); - to_pivoted(values, p0, p1) -} - -/// Detect peaks on the magnitude spectrum, read each one's dispersive angle, and -/// least-squares fit a phase ramp `arg = phase0 + phase1·frac` through them -/// (weighted by peak height). The classic multi-peak linear phasing; needs at -/// least two resolved peaks, else it defers to the zero-order rule. -fn peak_regression(values: &[Complex64]) -> (f64, f64, f64) { - let n = values.len(); - if n < 3 { - return absorptive_peak(values); - } - let mag: Vec = values.iter().map(|c| c.norm()).collect(); - let xs: Vec = (0..n).map(|i| i as f64).collect(); - let sigma = estimate_noise(&mag); - let params = DetectParams { - min_height: Some(6.0 * sigma), - min_prominence: 5.0 * sigma, - min_spacing: None, - max_count: Some(32), - }; - let mut peaks = detect_peaks(&xs, &mag, ¶ms); - if peaks.len() < 2 { - return absorptive_peak(values); - } - peaks.sort_by_key(|a| a.index); - - let denom = (n - 1).max(1) as f64; - // Unwrap successive peak angles so a ramp within ±π per gap fits cleanly. - let mut angles = Vec::with_capacity(peaks.len()); - let mut prev = 0.0; - for (k, p) in peaks.iter().enumerate() { - let raw = values[p.index].arg(); - let a = if k == 0 { - raw - } else { - prev + wrap_to_pi(raw - prev) - }; - angles.push(a); - prev = a; - } - - let (mut sw, mut swx, mut swy, mut swxx, mut swxy) = (0.0, 0.0, 0.0, 0.0, 0.0); - for (p, &y) in peaks.iter().zip(&angles) { - let w = p.y; - let x = p.index as f64 / denom; - sw += w; - swx += w * x; - swy += w * y; - swxx += w * x * x; - swxy += w * x * y; - } - let det = sw * swxx - swx * swx; - if det.abs() <= f64::MIN_POSITIVE { - return absorptive_peak(values); - } - let p0 = (swxx * swy - swx * swxy) / det; - let p1 = (sw * swxy - swx * swy) / det; - to_pivoted(values, p0, p1) -} - -/// ACME (Chen et al. 2002): Shannon entropy of the normalized absolute first -/// derivative of the real spectrum, plus a penalty for negative intensity. -fn acme_cost(values: &[Complex64], frac: &[f64], p0: f64, p1: f64) -> f64 { - let re: Vec = phased_real(values, frac, p0, p1); - let mut deriv: Vec = re.windows(2).map(|w| (w[1] - w[0]).abs()).collect(); - let sum: f64 = deriv.iter().sum(); - if sum <= 0.0 { - return f64::INFINITY; - } - let mut entropy = 0.0; - for d in &mut deriv { - let p = *d / sum; - if p > 0.0 { - entropy -= p * p.ln(); - } - } - let penalty: f64 = re.iter().filter(|&&y| y < 0.0).map(|y| y * y).sum(); - entropy + ACME_PENALTY * penalty -} - -/// Fraction of the real spectrum's power carried by its negative parts; zero when -/// every point is non-negative (a purely absorptive, upright spectrum). -fn negative_cost(values: &[Complex64], frac: &[f64], p0: f64, p1: f64) -> f64 { - let mut neg = 0.0; - let mut total = 0.0; - for (c, &fr) in values.iter().zip(frac) { - let (s, co) = (p0 + p1 * fr).sin_cos(); - let r = c.re * co + c.im * s; - total += r * r; - if r < 0.0 { - neg += r * r; - } - } - if total <= 0.0 { - f64::INFINITY - } else { - neg / total - } -} - -fn phased_real(values: &[Complex64], frac: &[f64], p0: f64, p1: f64) -> Vec { - values - .iter() - .zip(frac) - .map(|(c, &fr)| { - let (s, co) = (p0 + p1 * fr).sin_cos(); - c.re * co + c.im * s - }) - .collect() -} - -/// Coarse scan over `phase0 ∈ [-π, π)` and `phase1 ∈ [-2π, 2π]` for a robust -/// starting point that dodges the local minima the refinement would fall into. -fn coarse_grid(obj: &impl Fn(f64, f64) -> f64) -> (f64, f64) { - const N0: usize = 48; - const N1: usize = 25; - let mut best = (0.0, 0.0); - let mut best_cost = f64::INFINITY; - for i in 0..N0 { - let p0 = -PI + TAU * i as f64 / N0 as f64; - for j in 0..N1 { - let p1 = -2.0 * PI + 4.0 * PI * j as f64 / (N1 - 1) as f64; - let c = obj(p0, p1); - if c < best_cost { - best_cost = c; - best = (p0, p1); - } - } - } - best -} - -/// Hooke–Jeeves pattern search: probe ±step on each axis, step toward any -/// improvement, halve the step when stuck. Refines the grid seed to < 0.01°. -fn pattern_search(obj: &impl Fn(f64, f64) -> f64, mut p0: f64, mut p1: f64) -> (f64, f64) { - let mut step = PI / 18.0; - let mut best = obj(p0, p1); - for _ in 0..80 { - let mut improved = false; - for &(d0, d1) in &[(step, 0.0), (-step, 0.0), (0.0, step), (0.0, -step)] { - let c = obj(p0 + d0, p1 + d1); - if c < best { - best = c; - p0 += d0; - p1 += d1; - improved = true; - } - } - if !improved { - step *= 0.5; - if step < 1e-4 { - break; - } - } - } - (p0, p1) -} - -/// One-dimensional Hooke–Jeeves search over the ramp `p1` alone, used when the -/// zero-order phase is a pinned function of `p1`. Same probe-and-halve schedule. -fn pattern_search_1d(obj: &impl Fn(f64) -> f64, mut p1: f64) -> f64 { - let mut step = PI / 18.0; - let mut best = obj(p1); - for _ in 0..80 { - let mut improved = false; - for &delta in &[step, -step] { - let cost = obj(p1 + delta); - if cost < best { - best = cost; - p1 += delta; - improved = true; - } - } - if !improved { - step *= 0.5; - if step < 1e-4 { - break; - } - } - } - p1 -} - -/// Re-express a pivot-at-origin phase `φ(frac) = p0 + p1·frac` about the tallest -/// peak, so the returned pivot matches the on-plot handle of the other methods. -fn to_pivoted(values: &[Complex64], p0: f64, p1: f64) -> (f64, f64, f64) { - let pivot = phase::peak_pivot_frac(values); - (p0 + p1 * pivot, p1, pivot) -} - -/// Downsample to at most `max` points by max-magnitude pooling: each stride-wide -/// block contributes its tallest point. Plain stride sampling would step over the -/// narrow peaks of a large spectrum (a 160k-point spectrum decimates with stride -/// 160) and feed the optimizer mostly noise, so phasing would minimize the entropy -/// of noise. Keeping each block's peak preserves the lineshape the cost functions -/// need while holding the working length bounded. -fn decimate(values: &[Complex64], max: usize) -> (Vec, Vec) { - let n = values.len(); - if n == 0 { - return (Vec::new(), Vec::new()); - } - let denom = (n - 1).max(1) as f64; - let stride = n.div_ceil(max).max(1); - let mut vals = Vec::new(); - let mut fracs = Vec::new(); - let mut i = 0; - while i < n { - let end = (i + stride).min(n); - let j = (i..end) - .max_by(|&a, &b| values[a].norm().total_cmp(&values[b].norm())) - .unwrap_or(i); - vals.push(values[j]); - fracs.push(j as f64 / denom); - i = end; - } - (vals, fracs) -} - -fn wrap_to_pi(mut a: f64) -> f64 { - while a > PI { - a -= TAU; - } - while a < -PI { - a += TAU; - } - a -} - -#[cfg(test)] -mod tests { - use super::*; - - fn lorentzian(n: usize, center: usize, width: f64) -> Vec { - (0..n) - .map(|i| { - let d = (i as f64 - center as f64) / width; - // Absorption + i·dispersion of a Lorentzian. - Complex64::new(1.0 / (1.0 + d * d), -d / (1.0 + d * d)) - }) - .collect() - } - - fn scramble(values: &[Complex64], p0: f64, p1: f64) -> Vec { - let denom = (values.len() - 1).max(1) as f64; - values - .iter() - .enumerate() - .map(|(i, c)| { - let phi = p0 + p1 * (i as f64 / denom); - c * Complex64::from_polar(1.0, phi) - }) - .collect() - } - - fn apply(values: &[Complex64], p: (f64, f64, f64)) -> Vec { - let denom = (values.len() - 1).max(1) as f64; - values - .iter() - .enumerate() - .map(|(i, c)| { - let phi = p.0 + p.1 * (i as f64 / denom - p.2); - (c * Complex64::from_polar(1.0, -phi)).re - }) - .collect() - } - - fn upright(re: &[f64]) -> bool { - let (imin, &min) = re - .iter() - .enumerate() - .min_by(|a, b| a.1.total_cmp(b.1)) - .unwrap(); - let max = re.iter().cloned().fold(f64::MIN, f64::max); - // Absorptive peak dominates; no deep negative lobe. - max > 0.5 && min > -0.15 * max && imin != re.len() / 2 - } - - #[test] - fn entropy_recovers_scrambled_phase() { - let clean = lorentzian(512, 200, 4.0); - let bad = scramble(&clean, 1.1, 0.7); - let p = compute(&bad, AutoPhaseMethod::Entropy); - assert!(upright(&apply(&bad, p))); - } - - #[test] - fn negative_minimization_recovers_scrambled_phase() { - let clean = lorentzian(512, 300, 5.0); - let bad = scramble(&clean, -0.9, 0.5); - let p = compute(&bad, AutoPhaseMethod::NegativeMinimization); - assert!(upright(&apply(&bad, p))); - } - - #[test] - fn peak_regression_fits_two_peaks() { - let mut clean = lorentzian(1024, 250, 4.0); - for (i, c) in lorentzian(1024, 750, 4.0).into_iter().enumerate() { - clean[i] += c; - } - let bad = scramble(&clean, 0.4, 1.2); - let p = compute(&bad, AutoPhaseMethod::PeakRegression); - assert!(upright(&apply(&bad, p))); - } - - #[test] - fn methods_are_stable_on_degenerate_input() { - for m in [ - AutoPhaseMethod::RobustConsensus, - AutoPhaseMethod::Entropy, - AutoPhaseMethod::NegativeMinimization, - AutoPhaseMethod::PeakRegression, - ] { - let (p0, p1, piv) = compute(&[], m); - assert!(p0.is_finite() && p1.is_finite() && piv.is_finite()); - } - } - - #[test] - fn robust_consensus_handles_scaled_overlapping_peaks() { - let mut clean = lorentzian(768, 330, 6.0); - for (i, value) in lorentzian(768, 342, 8.0).into_iter().enumerate() { - clean[i] += value * 0.65; - } - let bad: Vec<_> = scramble(&clean, -1.2, 1.4) - .into_iter() - .map(|value| value * 2.5e5) - .collect(); - let p = compute(&bad, AutoPhaseMethod::RobustConsensus); - let corrected = apply(&bad, p); - let max = corrected.iter().copied().fold(f64::NEG_INFINITY, f64::max); - let min = corrected.iter().copied().fold(f64::INFINITY, f64::min); - assert!(max > 1.0e5); - assert!(min > -0.2 * max, "negative residual {min} vs peak {max}"); - } -} diff --git a/crates/processing/src/baseline.rs b/crates/processing/src/baseline.rs deleted file mode 100644 index 2082f243..00000000 --- a/crates/processing/src/baseline.rs +++ /dev/null @@ -1,355 +0,0 @@ -use crate::{BaselineMethod, Spectrum}; - -/// Subtract a baseline from the real channel in place, per `method`. -pub fn apply(spec: &mut Spectrum, method: BaselineMethod) { - match method { - BaselineMethod::Offset => correct_offset(spec), - BaselineMethod::Polynomial { order } => subtract_polynomial(spec, order as usize), - BaselineMethod::AsymmetricLeastSquares { - smoothness, - asymmetry, - iterations, - } => subtract_asymmetric_least_squares(spec, smoothness, asymmetry, iterations as usize), - } -} - -/// Estimate a smooth baseline with Eilers' asymmetric least-squares method and -/// subtract it from the real channel. Peaks receive the small asymmetric weight, -/// while points at or below the estimate anchor the baseline. The linear system -/// is symmetric positive definite and pentadiagonal, so each iteration is O(n). -fn subtract_asymmetric_least_squares( - spec: &mut Spectrum, - smoothness: f64, - asymmetry: f64, - iterations: usize, -) { - let n = spec.values.len(); - if n < 3 { - correct_offset(spec); - return; - } - let y = spec.real(); - let lambda = smoothness.clamp(1.0, 1.0e12); - let p = asymmetry.clamp(1.0e-6, 0.5); - let mut weights = vec![1.0; n]; - let mut baseline = vec![0.0; n]; - for _ in 0..iterations.clamp(1, 100) { - let main_penalty = |i: usize| match i { - 0 if n == 3 => 1.0, - 1 if n == 3 => 4.0, - 2 if n == 3 => 1.0, - 0 => 1.0, - 1 => 5.0, - i if i + 2 == n => 5.0, - i if i + 1 == n => 1.0, - _ => 6.0, - }; - let main: Vec = weights - .iter() - .enumerate() - .map(|(i, weight)| weight + lambda * main_penalty(i)) - .collect(); - let first: Vec = (0..n - 1) - .map(|i| { - if i == 0 || i + 2 == n { - -2.0 * lambda - } else { - -4.0 * lambda - } - }) - .collect(); - let second = vec![lambda; n - 2]; - let rhs: Vec = weights.iter().zip(&y).map(|(w, value)| w * value).collect(); - let Some(solution) = solve_symmetric_pentadiagonal(&main, &first, &second, &rhs) else { - correct_offset(spec); - return; - }; - baseline = solution; - for i in 0..n { - weights[i] = if y[i] > baseline[i] { p } else { 1.0 - p }; - } - } - for (value, base) in spec.values.iter_mut().zip(baseline) { - value.re -= base; - } -} - -/// Banded Cholesky solve for a symmetric positive-definite matrix with two -/// populated off-diagonals. `first[i]` is A[i,i+1], `second[i]` is A[i,i+2]. -fn solve_symmetric_pentadiagonal( - main: &[f64], - first: &[f64], - second: &[f64], - rhs: &[f64], -) -> Option> { - let n = main.len(); - if rhs.len() != n || first.len() + 1 != n || second.len() + 2 != n { - return None; - } - let mut diagonal = vec![0.0; n]; - let mut lower1 = vec![0.0; n]; - let mut lower2 = vec![0.0; n]; - for i in 0..n { - if i >= 2 { - lower2[i] = second[i - 2] / diagonal[i - 2]; - } - if i >= 1 { - let cross = if i >= 2 { - lower2[i] * lower1[i - 1] - } else { - 0.0 - }; - lower1[i] = (first[i - 1] - cross) / diagonal[i - 1]; - } - let remainder = main[i] - lower1[i] * lower1[i] - lower2[i] * lower2[i]; - if !remainder.is_finite() || remainder <= f64::MIN_POSITIVE { - return None; - } - diagonal[i] = remainder.sqrt(); - } - let mut solution = vec![0.0; n]; - for i in 0..n { - let mut value = rhs[i]; - if i >= 1 { - value -= lower1[i] * solution[i - 1]; - } - if i >= 2 { - value -= lower2[i] * solution[i - 2]; - } - solution[i] = value / diagonal[i]; - } - for i in (0..n).rev() { - let mut value = solution[i]; - if i + 1 < n { - value -= lower1[i + 1] * solution[i + 1]; - } - if i + 2 < n { - value -= lower2[i + 2] * solution[i + 2]; - } - solution[i] = value / diagonal[i]; - } - Some(solution) -} - -/// Subtract a constant offset, estimated from the quietest region of the -/// spectrum, from the real channel in place. -pub fn correct_offset(spec: &mut Spectrum) { - if spec.values.is_empty() { - return; - } - let offset = estimate_offset(&spec.real()); - for c in &mut spec.values { - c.re -= offset; - } -} - -// Fit a polynomial to the low-lying points of the real channel and subtract it, -// so a rolling or sloped baseline is flattened without peaks (which ride above -// the baseline) dragging the fit up. The fit index is mapped to `[-1, 1]` to -// keep the normal equations well-conditioned for higher orders. -fn subtract_polynomial(spec: &mut Spectrum, order: usize) { - let n = spec.values.len(); - let m = order + 1; - if n <= m { - correct_offset(spec); - return; - } - let real = spec.real(); - let mut sorted = real.clone(); - sorted.sort_by(|a, b| a.partial_cmp(b).unwrap_or(std::cmp::Ordering::Equal)); - let threshold = sorted[n / 2]; - - let t_of = |i: usize| 2.0 * i as f64 / (n - 1) as f64 - 1.0; - let mut ata = vec![vec![0.0; m]; m]; - let mut atb = vec![0.0; m]; - let mut anchors = 0usize; - let mut powers = vec![0.0; m]; - for (i, &value) in real.iter().enumerate() { - if value > threshold { - continue; - } - let t = t_of(i); - powers[0] = 1.0; - for k in 1..m { - powers[k] = powers[k - 1] * t; - } - for a in 0..m { - atb[a] += powers[a] * value; - for b in 0..m { - ata[a][b] += powers[a] * powers[b]; - } - } - anchors += 1; - } - if anchors < m { - correct_offset(spec); - return; - } - let coeffs = match plotx_analysis::fit::solve_linear(&ata, &atb) { - Some(c) => c, - None => { - correct_offset(spec); - return; - } - }; - for i in 0..n { - let t = t_of(i); - let mut tp = 1.0; - let mut base = 0.0; - for &c in &coeffs { - base += c * tp; - tp *= t; - } - spec.values[i].re -= base; - } -} - -fn estimate_offset(real: &[f64]) -> f64 { - if real.is_empty() { - return 0.0; - } - let mut sorted: Vec = real.to_vec(); - sorted.sort_by(|a, b| a.partial_cmp(b).unwrap_or(std::cmp::Ordering::Equal)); - // Robust centre + scale of the whole channel: with sparse (positive) peaks the - // median sits on the baseline noise, and MAD/0.6745 estimates its σ. - let median = median_sorted(&sorted); - let mut dev: Vec = sorted.iter().map(|&v| (v - median).abs()).collect(); - dev.sort_by(|a, b| a.partial_cmp(b).unwrap_or(std::cmp::Ordering::Equal)); - let sigma = median_sorted(&dev) / 0.674_489_75; - if sigma <= f64::MIN_POSITIVE { - return median; - } - // Average the points inside a ±3σ band about the median: peaks are excluded and - // the retained noise is symmetric, so the mean is an unbiased estimate of the - // baseline centre — unlike the lowest decile, whose median sat ~1.6σ too low. - let (lo, hi) = (median - 3.0 * sigma, median + 3.0 * sigma); - let (mut sum, mut count) = (0.0, 0usize); - for &v in &sorted { - if v >= lo && v <= hi { - sum += v; - count += 1; - } - } - if count == 0 { - median - } else { - sum / count as f64 - } -} - -fn median_sorted(sorted: &[f64]) -> f64 { - let n = sorted.len(); - if n == 0 { - 0.0 - } else if n % 2 == 1 { - sorted[n / 2] - } else { - 0.5 * (sorted[n / 2 - 1] + sorted[n / 2]) - } -} - -#[cfg(test)] -mod tests { - use super::*; - use num_complex::Complex64; - - #[test] - fn removes_constant_offset() { - let mut values: Vec = (0..100).map(|_| Complex64::new(5.0, 0.0)).collect(); - values[50] = Complex64::new(105.0, 0.0); - let mut s = Spectrum { - ppm: (0..100).map(|i| i as f64).collect(), - values, - hz_per_point: 1.0, - observe_freq_mhz: 400.0, - nucleus: "1H".into(), - }; - correct_offset(&mut s); - assert!(s.values[0].re.abs() < 1e-9); - assert!((s.values[50].re - 100.0).abs() < 1e-9); - } - - #[test] - fn offset_estimate_is_unbiased_for_noisy_baseline() { - let offset = 10.0; - let real: Vec = (0..400) - .map(|i| { - let noise = (i as f64 * 0.7).sin() * 2.0; - let peak = if i % 137 == 0 { 100.0 } else { 0.0 }; - offset + noise + peak - }) - .collect(); - let est = estimate_offset(&real); - assert!( - (est - offset).abs() < 0.3, - "offset estimate {est} vs {offset}" - ); - } - - #[test] - fn polynomial_flattens_a_sloped_baseline() { - let n = 200; - let values: Vec = (0..n) - .map(|i| { - let ramp = 3.0 + 0.05 * i as f64; - let peak = if i == 150 { 500.0 } else { 0.0 }; - Complex64::new(ramp + peak, 0.0) - }) - .collect(); - let mut s = Spectrum { - ppm: (0..n).map(|i| i as f64).collect(), - values, - hz_per_point: 1.0, - observe_freq_mhz: 400.0, - nucleus: "1H".into(), - }; - apply(&mut s, BaselineMethod::Polynomial { order: 1 }); - for i in 0..n { - if i == 150 { - continue; - } - assert!( - s.values[i].re.abs() < 1e-6, - "baseline at {i} = {}", - s.values[i].re - ); - } - assert!((s.values[150].re - 500.0).abs() < 1e-6); - } - - #[test] - fn asymmetric_least_squares_removes_curved_baseline_without_erasing_peaks() { - let n = 600; - let values: Vec = (0..n) - .map(|i| { - let x = 2.0 * i as f64 / (n - 1) as f64 - 1.0; - let baseline = 8.0 + 4.0 * x + 7.0 * x * x; - let peak = 120.0 * (-((i as f64 - 190.0) / 8.0).powi(2)).exp() - + 75.0 * (-((i as f64 - 430.0) / 13.0).powi(2)).exp(); - Complex64::new(baseline + peak, 0.0) - }) - .collect(); - let mut spectrum = Spectrum { - ppm: (0..n).map(|i| i as f64).collect(), - values, - hz_per_point: 1.0, - observe_freq_mhz: 400.0, - nucleus: "1H".into(), - }; - // This baseline is deliberately steep, so it exercises the solver with a - // smoothness looser than the `AUTO` preset — which is tuned for the gentle - // baselines and broad peaks covered by the crate's quality-contract tests. - apply( - &mut spectrum, - BaselineMethod::AsymmetricLeastSquares { - smoothness: 1.0e4, - asymmetry: 0.001, - iterations: 20, - }, - ); - assert!(spectrum.values[20].re.abs() < 1.0); - assert!(spectrum.values[570].re.abs() < 1.0); - assert!(spectrum.values[190].re > 100.0); - assert!(spectrum.values[430].re > 60.0); - } -} diff --git a/crates/processing/src/cleanup.rs b/crates/processing/src/cleanup.rs deleted file mode 100644 index 22b97163..00000000 --- a/crates/processing/src/cleanup.rs +++ /dev/null @@ -1,360 +0,0 @@ -//! Spectrum cleanup steps: smoothing, normalization, binning, reverse, invert. - -use crate::{BinMethod, BinParams, NormalizeMethod, SmoothMethod, Spectrum}; -use num_complex::Complex64; - -pub fn smooth(spec: &mut Spectrum, method: SmoothMethod) { - match method { - SmoothMethod::MovingAverage { window } => moving_average(&mut spec.values, window as usize), - SmoothMethod::SavitzkyGolay { window, poly_order } => { - savitzky_golay(&mut spec.values, window as usize, poly_order as usize) - } - } -} - -/// Gaussian smoothing for real-valued detection helpers that need a stable, -/// symmetric kernel but are not persisted processing steps. -pub fn gaussian_smooth_real(values: &[f64], sigma: f64) -> Option> { - if values.is_empty() - || !sigma.is_finite() - || sigma <= 0.0 - || values.iter().any(|value| !value.is_finite()) - { - return None; - } - let radius = (3.0 * sigma).ceil() as isize; - Some( - (0..values.len()) - .map(|index| { - let mut weighted = 0.0; - let mut total = 0.0; - for offset in -radius..=radius { - let source = - (index as isize + offset).clamp(0, values.len() as isize - 1) as usize; - let weight = (-0.5 * (offset as f64 / sigma).powi(2)).exp(); - weighted += values[source] * weight; - total += weight; - } - weighted / total - }) - .collect(), - ) -} - -fn moving_average(values: &mut Vec, window: usize) { - let n = values.len(); - let w = (window.max(3) | 1).min(if n % 2 == 1 { n } else { n.saturating_sub(1) }); - if n < 3 || w < 3 { - return; - } - let h = w / 2; - let mut out = Vec::with_capacity(n); - for i in 0..n { - let lo = i.saturating_sub(h); - let hi = (i + h + 1).min(n); - let sum: Complex64 = values[lo..hi].iter().sum(); - out.push(sum / (hi - lo) as f64); - } - *values = out; -} - -/// Least-squares polynomial smoothing: each point is replaced by the value of a -/// degree-`order` polynomial fitted over an odd `window` around it. Edge points -/// reuse the boundary window, evaluated off-center, so a polynomial signal of -/// degree ≤ `order` is reproduced exactly everywhere. -fn savitzky_golay(values: &mut Vec, window: usize, order: usize) { - let n = values.len(); - let w = (window.max(3) | 1).min(if n % 2 == 1 { n } else { n.saturating_sub(1) }); - if n < 3 || w < 3 { - return; - } - let m = order.clamp(1, w - 1) + 1; - let h = w / 2; - let x = |i: usize| i as f64 - h as f64; - - let mut gram = vec![vec![0.0; m]; m]; - for i in 0..w { - let mut powers = vec![1.0; m]; - for k in 1..m { - powers[k] = powers[k - 1] * x(i); - } - for r in 0..m { - for c in 0..m { - gram[r][c] += powers[r] * powers[c]; - } - } - } - let mut gram_inv = vec![vec![0.0; m]; m]; - for k in 0..m { - let mut e = vec![0.0; m]; - e[k] = 1.0; - let Some(col) = plotx_analysis::fit::solve_linear(&gram, &e) else { - return; - }; - for r in 0..m { - gram_inv[r][k] = col[r]; - } - } - let sample_powers: Vec> = (0..w) - .map(|i| { - let mut powers = vec![1.0; m]; - for k in 1..m { - powers[k] = powers[k - 1] * x(i); - } - powers - }) - .collect(); - // projection[k][i]: coefficient k of the fitted polynomial from sample i. - let projection: Vec> = (0..m) - .map(|r| { - sample_powers - .iter() - .map(|powers| (0..m).map(|k| gram_inv[r][k] * powers[k]).sum()) - .collect() - }) - .collect(); - // weights[p][i]: smoothing weights when evaluating at offset p in the window. - let weights: Vec> = (0..w) - .map(|p| { - let mut powers = vec![1.0; m]; - for k in 1..m { - powers[k] = powers[k - 1] * x(p); - } - (0..w) - .map(|i| (0..m).map(|k| powers[k] * projection[k][i]).sum()) - .collect() - }) - .collect(); - - let mut out = Vec::with_capacity(n); - for i in 0..n { - let (start, p) = if i < h { - (0, i) - } else if i + h >= n { - (n - w, i - (n - w)) - } else { - (i - h, h) - }; - let mut acc = Complex64::new(0.0, 0.0); - for (j, &weight) in weights[p].iter().enumerate() { - acc += values[start + j] * weight; - } - out.push(acc); - } - *values = out; -} - -pub fn normalize(spec: &mut Spectrum, method: NormalizeMethod) { - let scale = match method { - NormalizeMethod::MaxPeak => spec.values.iter().map(|c| c.norm()).fold(0.0, f64::max), - NormalizeMethod::TotalArea => { - spec.values.iter().map(|c| c.re.abs()).sum::() * axis_step(&spec.ppm) - } - NormalizeMethod::Constant { divisor } => divisor, - }; - if scale.is_finite() && scale.abs() > f64::MIN_POSITIVE { - for c in &mut spec.values { - *c /= scale; - } - } -} - -/// Aggregate runs of points into bins of `width` axis units. The axis becomes -/// the per-bin mean position and `hz_per_point` grows by the same factor, so -/// axis metadata stays consistent with the reduced point count. -pub fn bin(spec: &mut Spectrum, params: BinParams) { - let n = spec.values.len(); - let step = axis_step(&spec.ppm); - if n == 0 || !params.width.is_finite() || params.width <= 0.0 { - return; - } - let per = ((params.width / step).round() as usize).max(1); - if per <= 1 { - return; - } - let bins = n.div_ceil(per); - let mut values = Vec::with_capacity(bins); - let mut ppm = Vec::with_capacity(bins); - for start in (0..n).step_by(per) { - let end = (start + per).min(n); - let count = (end - start) as f64; - let sum: Complex64 = spec.values[start..end].iter().sum(); - values.push(match params.method { - BinMethod::Sum => sum, - BinMethod::Mean => sum / count, - }); - ppm.push(spec.ppm[start..end].iter().sum::() / count); - } - spec.values = values; - spec.ppm = ppm; - spec.hz_per_point *= per as f64; -} - -/// Mirror the intensities along the axis; the axis itself keeps its ordering. -pub fn reverse(spec: &mut Spectrum) { - spec.values.reverse(); -} - -pub fn invert(spec: &mut Spectrum) { - for c in &mut spec.values { - *c = -*c; - } -} - -/// Effective spacing of the axis consumed by binning. -pub fn axis_step(ppm: &[f64]) -> f64 { - if ppm.len() < 2 { - return 1.0; - } - let span = (ppm[ppm.len() - 1] - ppm[0]).abs(); - if span > 0.0 { - span / (ppm.len() - 1) as f64 - } else { - 1.0 - } -} - -#[cfg(test)] -mod tests { - use super::*; - - fn spec_from(real: Vec) -> Spectrum { - let n = real.len(); - Spectrum { - ppm: (0..n).map(|i| i as f64 * 0.01).collect(), - values: real.into_iter().map(|v| Complex64::new(v, 0.0)).collect(), - hz_per_point: 1.0, - observe_freq_mhz: 400.0, - nucleus: "1H".into(), - } - } - - #[test] - fn moving_average_preserves_a_constant_and_averages_neighbors() { - let mut s = spec_from(vec![4.0; 50]); - smooth(&mut s, SmoothMethod::MovingAverage { window: 5 }); - for c in &s.values { - assert!((c.re - 4.0).abs() < 1e-12); - } - let mut s = spec_from(vec![0.0, 0.0, 3.0, 0.0, 0.0]); - smooth(&mut s, SmoothMethod::MovingAverage { window: 3 }); - assert!((s.values[1].re - 1.0).abs() < 1e-12); - assert!((s.values[2].re - 1.0).abs() < 1e-12); - assert!((s.values[3].re - 1.0).abs() < 1e-12); - } - - #[test] - fn savitzky_golay_reproduces_a_cubic_exactly_including_edges() { - let cubic: Vec = (0..80) - .map(|i| { - let t = i as f64 * 0.1; - 2.0 + 3.0 * t - 1.5 * t * t + 0.25 * t * t * t - }) - .collect(); - let mut s = spec_from(cubic.clone()); - smooth( - &mut s, - SmoothMethod::SavitzkyGolay { - window: 9, - poly_order: 3, - }, - ); - for (c, expected) in s.values.iter().zip(&cubic) { - assert!((c.re - expected).abs() < 1e-9, "{} vs {expected}", c.re); - } - } - - #[test] - fn savitzky_golay_attenuates_noise() { - let noisy: Vec = (0..200) - .map(|i| if i % 2 == 0 { 1.0 } else { -1.0 }) - .collect(); - let mut s = spec_from(noisy); - smooth( - &mut s, - SmoothMethod::SavitzkyGolay { - window: 11, - poly_order: 2, - }, - ); - let rms: f64 = - (s.values.iter().map(|c| c.re * c.re).sum::() / s.values.len() as f64).sqrt(); - assert!(rms < 0.5, "rms {rms}"); - } - - #[test] - fn max_peak_normalization_scales_the_tallest_peak_to_one() { - let mut s = spec_from(vec![1.0, -2.0, 8.0, 0.5]); - normalize(&mut s, NormalizeMethod::MaxPeak); - let max = s.values.iter().map(|c| c.norm()).fold(0.0, f64::max); - assert!((max - 1.0).abs() < 1e-12); - assert!((s.values[1].re + 0.25).abs() < 1e-12); - } - - #[test] - fn total_area_normalization_makes_the_integral_one() { - let mut s = spec_from((0..100).map(|i| if i == 50 { 20.0 } else { 2.0 }).collect()); - normalize(&mut s, NormalizeMethod::TotalArea); - let dx = 0.01; - let area: f64 = s.values.iter().map(|c| c.re.abs()).sum::() * dx; - assert!((area - 1.0).abs() < 1e-12, "area {area}"); - } - - #[test] - fn constant_normalization_divides_and_ignores_zero() { - let mut s = spec_from(vec![4.0, 6.0]); - normalize(&mut s, NormalizeMethod::Constant { divisor: 2.0 }); - assert!((s.values[0].re - 2.0).abs() < 1e-12); - normalize(&mut s, NormalizeMethod::Constant { divisor: 0.0 }); - assert!((s.values[0].re - 2.0).abs() < 1e-12); - } - - #[test] - fn binning_reduces_points_and_keeps_axis_and_metadata_consistent() { - let mut s = spec_from((0..100).map(|i| i as f64).collect()); - bin( - &mut s, - BinParams { - width: 0.05, - method: BinMethod::Mean, - }, - ); - assert_eq!(s.values.len(), 20); - assert_eq!(s.ppm.len(), 20); - assert!((s.values[0].re - 2.0).abs() < 1e-12); - assert!((s.values[1].re - 7.0).abs() < 1e-12); - assert!((s.ppm[0] - 0.02).abs() < 1e-12); - assert!((s.hz_per_point - 5.0).abs() < 1e-12); - - let mut s = spec_from(vec![1.0; 10]); - bin( - &mut s, - BinParams { - width: 0.04, - method: BinMethod::Sum, - }, - ); - assert_eq!(s.values.len(), 3); - assert!((s.values[0].re - 4.0).abs() < 1e-12); - assert!((s.values[2].re - 2.0).abs() < 1e-12); - } - - #[test] - fn reverse_mirrors_intensities_and_keeps_the_axis() { - let mut s = spec_from(vec![1.0, 2.0, 3.0]); - let ppm = s.ppm.clone(); - reverse(&mut s); - assert_eq!(s.ppm, ppm); - assert!((s.values[0].re - 3.0).abs() < 1e-12); - reverse(&mut s); - assert!((s.values[0].re - 1.0).abs() < 1e-12); - } - - #[test] - fn invert_negates_intensities() { - let mut s = spec_from(vec![1.0, -2.0]); - invert(&mut s); - assert!((s.values[0].re + 1.0).abs() < 1e-12); - assert!((s.values[1].re - 2.0).abs() < 1e-12); - } -} diff --git a/crates/processing/src/craft.rs b/crates/processing/src/craft.rs index ff488661..edbbc362 100644 --- a/crates/processing/src/craft.rs +++ b/crates/processing/src/craft.rs @@ -7,6 +7,8 @@ use serde::{Deserialize, Serialize}; mod diagnostics; mod fitting; +mod nmr_preview; +pub use nmr_preview::preview_spectrum; mod preflight; mod reconstruction; mod regions; @@ -81,19 +83,26 @@ pub struct CraftRegionId(pub u64); #[derive(Clone, Copy, Debug, PartialEq, Serialize, Deserialize)] pub struct CraftReference { pub acquisition_carrier_ppm: f64, + /// Frequency defining one ppm, independently of the observed transmitter frequency. + pub reference_frequency_mhz: f64, pub offset_ppm: f64, } impl CraftReference { - pub const fn new(acquisition_carrier_ppm: f64, offset_ppm: f64) -> Self { + pub const fn new( + acquisition_carrier_ppm: f64, + reference_frequency_mhz: f64, + offset_ppm: f64, + ) -> Self { Self { acquisition_carrier_ppm, + reference_frequency_mhz, offset_ppm, } } pub fn acquisition(data: &NmrData) -> Self { - Self::new(data.carrier_ppm, 0.0) + Self::new(data.carrier_ppm, data.observe_freq_mhz, 0.0) } pub fn effective_carrier_ppm(self) -> f64 { @@ -102,6 +111,8 @@ impl CraftReference { pub fn validate(self, data: &NmrData) -> Result<(), CraftError> { if self.acquisition_carrier_ppm.is_finite() + && self.reference_frequency_mhz.is_finite() + && self.reference_frequency_mhz > 0.0 && self.offset_ppm.is_finite() && self.effective_carrier_ppm().is_finite() && self.acquisition_carrier_ppm == data.carrier_ppm @@ -349,8 +360,10 @@ pub fn process_craft_cancellable( .map(|region| { let region = region.normalized(); ( - (region.start_ppm - reference.effective_carrier_ppm()) * data.observe_freq_mhz, - (region.end_ppm - reference.effective_carrier_ppm()) * data.observe_freq_mhz, + (region.start_ppm - reference.effective_carrier_ppm()) + * reference.reference_frequency_mhz, + (region.end_ppm - reference.effective_carrier_ppm()) + * reference.reference_frequency_mhz, ) }) .collect::>(); @@ -414,7 +427,7 @@ pub fn process_craft_cancellable( region: CraftRegionId(0), frequency_hz, chemical_shift_ppm: reference.effective_carrier_ppm() - + frequency_hz / data.observe_freq_mhz, + + frequency_hz / reference.reference_frequency_mhz, amplitude_t0: component.amplitude, phase_rad: component.phase_rad, decay_rate_s_inv: component.decay_rate_s_inv, @@ -430,7 +443,7 @@ pub fn process_craft_cancellable( }) .collect(); let selections = if params.regions.is_empty() { - let half_width_ppm = sw / (2.0 * data.observe_freq_mhz); + let half_width_ppm = sw / (2.0 * reference.reference_frequency_mhz); vec![CraftRegion::new( CraftRegionId(0), reference.effective_carrier_ppm() - half_width_ppm, diff --git a/crates/processing/src/craft/nmr_preview.rs b/crates/processing/src/craft/nmr_preview.rs new file mode 100644 index 00000000..000321fa --- /dev/null +++ b/crates/processing/src/craft/nmr_preview.rs @@ -0,0 +1,111 @@ +//! Diagnostic transforms over the CRAFT input view use the NMR library kernels. + +use super::{CraftError, CraftReference}; +use crate::Spectrum; +use nmr::axis::{AxisCoordinates, AxisDomain, AxisUnit, FrequencyEvidence}; +use nmr::processing::{ + FourierTransform, FrequencyFrame, ProcessingOperation as Op, ProcessingPlan, ReferenceSource, + SpectrumOperation, Window, ZeroFill, +}; +use nmr::raw::{ + ChemicalShiftReference, DirectSamples, RawAxis, RawAxisKind, RawDatasetBuilder, RawMetadata, +}; +use plotx_io::NmrData; + +/// Transform the selected modeling interval with a matched exponential window. +/// The diagnostic has no digital-filter correction: only magnitudes are used. +pub fn preview_spectrum( + data: &NmrData, + reference: CraftReference, + skip: usize, +) -> Result { + let fail = |error: &dyn std::fmt::Display| CraftError::Preflight(error.to_string()); + if data.domain != plotx_io::Domain::Time { + return Err(CraftError::InvalidInput); + } + reference.validate(data)?; + let retained = data + .points + .len() + .checked_sub(skip) + .filter(|count| *count >= 3) + .ok_or(CraftError::InvalidInput)?; + let target = retained + .checked_next_power_of_two() + .ok_or(CraftError::InvalidInput)?; + let axis = RawAxis::new( + RawAxisKind::Direct(DirectSamples::Complex), + AxisDomain::Time, + Some(AxisUnit::Second), + data.points.len(), + AxisCoordinates::Uniform { + start: 0.0, + step: 1.0 / data.spectral_width_hz, + }, + ) + .map_err(|error| fail(&error))? + .with_spectral_width_hz(Some(data.spectral_width_hz)) + .map_err(|error| fail(&error))? + .with_frequency_evidence(Some( + FrequencyEvidence::new(Some(data.observe_freq_mhz), None).map_err(|error| fail(&error))?, + )) + .map_err(|error| fail(&error))? + .with_chemical_shift_reference(Some( + ChemicalShiftReference::user_constructed( + reference.effective_carrier_ppm(), + reference.reference_frequency_mhz, + ) + .map_err(|error| fail(&error))?, + )) + .map_err(|error| fail(&error))?; + // This is a disposable analysis view, not a substitute for the acquisition + // Dataset and its provenance in application storage. + let input = RawDatasetBuilder::new(vec![axis], RawMetadata::default()) + .map_err(|error| fail(&error))? + .dense(data.points.clone()) + .map_err(|error| fail(&error))?; + let plan = ProcessingPlan::new(vec![ + Op::Spectrum { + axis: 0, + operation: SpectrumOperation::RetainRange { + start: skip, + end: data.points.len(), + }, + }, + Op::Window { + axis: 0, + window: Window::exponential(data.spectral_width_hz / retained as f64) + .map_err(|error| fail(&error))?, + }, + Op::ZeroFill { + axis: 0, + zero_fill: ZeroFill::new(target).map_err(|error| fail(&error))?, + }, + Op::FourierTransform { + axis: 0, + transform: FourierTransform::default(), + }, + Op::ResolveFrequencyFrame { + axis: 0, + frame: FrequencyFrame::Ppm(ReferenceSource::AxisEvidence), + }, + Op::Spectrum { + axis: 0, + operation: SpectrumOperation::Magnitude, + }, + ]) + .map_err(|error| fail(&error))?; + let output = plan.apply(&input.into()).map_err(|error| fail(&error))?; + let source = plotx_io::nmr_view::NmrSource::new(std::sync::Arc::new(output)) + .map_err(|error| fail(&error))?; + Ok(Spectrum { + ppm: source.axes()[0] + .coordinate_values() + .map_err(|error| fail(&error))?, + values: source.trace().map_err(|error| fail(&error))?, + unit: nmr::axis::AxisUnit::Ppm, + hz_per_point: Some(data.spectral_width_hz / target as f64), + observe_freq_mhz: Some(data.observe_freq_mhz), + nucleus: data.nucleus.clone(), + }) +} diff --git a/crates/processing/src/craft/preflight.rs b/crates/processing/src/craft/preflight.rs index a2909efb..31e63aaa 100644 --- a/crates/processing/src/craft/preflight.rs +++ b/crates/processing/src/craft/preflight.rs @@ -1,8 +1,6 @@ use plotx_analysis::peaks::{DetectParams, detect_peaks, estimate_noise}; use plotx_io::{Domain, NmrData}; -use rustfft::FftPlanner; use serde::{Deserialize, Serialize}; -use std::f64::consts::PI; use super::{CraftDerivedPlan, CraftParams, CraftReference, CraftRegionId}; @@ -28,6 +26,7 @@ pub enum CraftIssueCode { NoClearSignal, RegionWithoutClearSignal, DenseSignalWindow, + ProcessingFailure, } #[derive(Clone, Copy, Debug, PartialEq, Eq, Serialize, Deserialize)] @@ -89,6 +88,13 @@ impl CraftInputAssessment { action, }) }; + if let Some(message) = &plan.processing_error { + error( + CraftIssueCode::ProcessingFailure, + message, + CraftIssueAction::CheckAcquisitionMetadata, + ); + } if data.domain != Domain::Time { error( CraftIssueCode::IncompatibleDomain, @@ -177,7 +183,19 @@ impl CraftInputAssessment { { Vec::new() } else { - detect_clear_signals(data, reference, plan.effective_skip_points) + match detect_clear_signals(data, reference, plan.effective_skip_points) { + Ok(signals) => signals, + Err(error) => { + issues.push(CraftAssessmentIssue { + code: CraftIssueCode::ProcessingFailure, + severity: CraftIssueSeverity::Error, + region: None, + message: error.to_string(), + action: CraftIssueAction::CheckAcquisitionMetadata, + }); + Vec::new() + } + } }; if plan.available_points >= 16 && clear_signals.is_empty() { issues.push(warning(CraftIssueCode::NoClearSignal, None, "No clear signal reached the 6σ height and 5σ prominence thresholds; the calculation can run but needs review.", CraftIssueAction::ConfirmWithIndependentEvidence)); @@ -258,7 +276,7 @@ fn regions_outside_bandwidth_or_overlap( { return false; } - let half_ppm = data.spectral_width_hz / (2.0 * data.observe_freq_mhz); + let half_ppm = data.spectral_width_hz / (2.0 * reference.reference_frequency_mhz); let carrier = reference.effective_carrier_ppm(); let lower = carrier - half_ppm; let upper = carrier + half_ppm; @@ -284,28 +302,20 @@ pub(super) fn detect_clear_signals( data: &NmrData, reference: CraftReference, skip: usize, -) -> Vec { - let input = &data.points[skip..]; +) -> Result, super::CraftError> { + let input = data + .points + .get(skip..) + .ok_or(super::CraftError::InvalidInput)?; if input.len() < 3 { - return Vec::new(); - } - let fft_len = input.len().next_power_of_two(); - let mut spectrum = vec![num_complex::Complex64::new(0.0, 0.0); fft_len]; - let duration_s = input.len() as f64 / data.spectral_width_hz; - let matched_line_broadening_hz = 1.0 / duration_s.max(f64::MIN_POSITIVE); - for (index, (&sample, output)) in input.iter().zip(&mut spectrum).enumerate() { - let time_s = index as f64 / data.spectral_width_hz; - *output = sample * (-PI * matched_line_broadening_hz * time_s).exp(); + return Ok(Vec::new()); } - FftPlanner::::new() - .plan_fft_forward(fft_len) - .process(&mut spectrum); - let magnitudes = spectrum + let spectrum = super::preview_spectrum(data, reference, skip)?; + let fft_len = spectrum.len(); + let shifted = spectrum + .values .iter() - .map(|value| value.norm()) - .collect::>(); - let shifted = (0..fft_len) - .map(|index| magnitudes[(index + fft_len / 2) % fft_len]) + .map(|value| value.re) .collect::>(); let sigma = estimate_noise(&shifted).max(f64::MIN_POSITIVE); let xs = (0..fft_len).map(|index| index as f64).collect::>(); @@ -326,16 +336,12 @@ pub(super) fn detect_clear_signals( max_count: Some(64), }, ); - peaks + Ok(peaks .into_iter() - .map(|peak| { - let frequency_hz = (peak.index as f64 / fft_len as f64 - 0.5) * data.spectral_width_hz; - CraftSignalSuggestion { - chemical_shift_ppm: reference.effective_carrier_ppm() - + frequency_hz / data.observe_freq_mhz, - height_sigma: shifted[peak.index] / sigma, - prominence_sigma: peak.prominence / sigma, - } + .map(|peak| CraftSignalSuggestion { + chemical_shift_ppm: spectrum.ppm[peak.index], + height_sigma: shifted[peak.index] / sigma, + prominence_sigma: peak.prominence / sigma, }) - .collect() + .collect()) } diff --git a/crates/processing/src/craft/regions.rs b/crates/processing/src/craft/regions.rs index c5d30017..0efe72ec 100644 --- a/crates/processing/src/craft/regions.rs +++ b/crates/processing/src/craft/regions.rs @@ -47,8 +47,8 @@ pub(super) fn build_modeling_windows( let requested: Vec<(CraftRegion, f64, f64)> = if params.regions.is_empty() { let selection = CraftRegion::new( CraftRegionId(0), - effective_carrier_ppm - half_sw / data.observe_freq_mhz, - effective_carrier_ppm + half_sw / data.observe_freq_mhz, + effective_carrier_ppm - half_sw / reference.reference_frequency_mhz, + effective_carrier_ppm + half_sw / reference.reference_frequency_mhz, ); vec![(selection, -half_sw, half_sw)] } else { @@ -60,8 +60,8 @@ pub(super) fn build_modeling_windows( .map(|region| { ( region, - (region.start_ppm - effective_carrier_ppm) * data.observe_freq_mhz, - (region.end_ppm - effective_carrier_ppm) * data.observe_freq_mhz, + (region.start_ppm - effective_carrier_ppm) * reference.reference_frequency_mhz, + (region.end_ppm - effective_carrier_ppm) * reference.reference_frequency_mhz, ) }) .collect() @@ -76,8 +76,8 @@ pub(super) fn build_modeling_windows( requested_cores.push(( CraftRegion::new( selection.id, - effective_carrier_ppm + start / data.observe_freq_mhz, - effective_carrier_ppm + end / data.observe_freq_mhz, + effective_carrier_ppm + start / reference.reference_frequency_mhz, + effective_carrier_ppm + end / reference.reference_frequency_mhz, ), start, end, @@ -98,8 +98,8 @@ pub(super) fn build_modeling_windows( let signal_hz = clear_signals .iter() .filter_map(|signal| { - let frequency = - (signal.chemical_shift_ppm - effective_carrier_ppm) * data.observe_freq_mhz; + let frequency = (signal.chemical_shift_ppm - effective_carrier_ppm) + * reference.reference_frequency_mhz; let weight = signal.prominence_sigma.max(f64::MIN_POSITIVE); (frequency.is_finite() && weight.is_finite() diff --git a/crates/processing/src/craft/resolution.rs b/crates/processing/src/craft/resolution.rs index b6e1d4d0..1a1b1872 100644 --- a/crates/processing/src/craft/resolution.rs +++ b/crates/processing/src/craft/resolution.rs @@ -138,6 +138,8 @@ pub struct CraftDerivedPlan { pub reconstruction_points: usize, pub resolved_regions: Vec, pub modeling_windows: Vec, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub processing_error: Option, } pub fn resolve_craft_invocation( @@ -232,7 +234,7 @@ pub fn resolve_craft_invocation( } fn full_bandwidth_region(data: &NmrData, reference: CraftReference) -> Option { - let half_width_ppm = data.spectral_width_hz / (2.0 * data.observe_freq_mhz); + let half_width_ppm = data.spectral_width_hz / (2.0 * reference.reference_frequency_mhz); let carrier = reference.effective_carrier_ppm(); (half_width_ppm.is_finite() && carrier.is_finite()).then(|| { CraftRegion::new( @@ -302,6 +304,7 @@ fn derive_plan( }; let resolved_regions = params.regions.clone(); let mut modeling_windows = Vec::new(); + let mut processing_error = None; if data.spectral_width_hz.is_finite() && data.spectral_width_hz > 0.0 && data.observe_freq_mhz.is_finite() @@ -310,10 +313,16 @@ fn derive_plan( && params.profile.modeling_bandwidth_hz().is_finite() { let filter_input = available_points.min(fit_points.saturating_add(params.fir_filter_taps)); - let clear_signals = detect_clear_signals(data, reference, effective_skip_points); - for window in - build_modeling_windows(data, params, reference, &clear_signals).unwrap_or_default() - { + let windows = detect_clear_signals(data, reference, effective_skip_points) + .and_then(|signals| build_modeling_windows(data, params, reference, &signals)); + let windows = match windows { + Ok(windows) => windows, + Err(error) => { + processing_error = Some(error.to_string()); + Vec::new() + } + }; + for window in windows { let modeled_bandwidth_hz = window.modeling_band_hz.1 - window.modeling_band_hz.0; let mut decimation = (data.spectral_width_hz / (2.0 * modeled_bandwidth_hz).max(f64::MIN_POSITIVE)) @@ -345,6 +354,7 @@ fn derive_plan( reconstruction_points, resolved_regions, modeling_windows, + processing_error, } } diff --git a/crates/processing/src/craft/stability.rs b/crates/processing/src/craft/stability.rs index f006e36d..8692edde 100644 --- a/crates/processing/src/craft/stability.rs +++ b/crates/processing/src/craft/stability.rs @@ -15,7 +15,7 @@ pub(super) fn stability_diagnostics( reference: CraftReference, data: &NmrData, ) -> CraftStabilityDiagnostics { - let delta_ppm = (0.01_f64).max(8.0 / data.observe_freq_mhz.max(f64::MIN_POSITIVE)); + let delta_ppm = (0.01_f64).max(8.0 / reference.reference_frequency_mhz.max(f64::MIN_POSITIVE)); let mut perturbations = vec![("original".to_owned(), selections.to_vec())]; for (name, start_delta, end_delta) in [ ("shift left", -delta_ppm, -delta_ppm), @@ -55,7 +55,7 @@ pub(super) fn stability_diagnostics( } let carrier = reference.effective_carrier_ppm(); - let half_ppm = data.spectral_width_hz / (2.0 * data.observe_freq_mhz); + let half_ppm = data.spectral_width_hz / (2.0 * reference.reference_frequency_mhz); let lower = carrier - half_ppm; let upper = carrier + half_ppm; let mut skipped = Vec::new(); diff --git a/crates/processing/src/craft_tests.rs b/crates/processing/src/craft_tests.rs index c735e035..4c057d02 100644 --- a/crates/processing/src/craft_tests.rs +++ b/crates/processing/src/craft_tests.rs @@ -233,14 +233,14 @@ fn overlapping_requested_regions_are_rejected_as_ambiguous() { #[test] fn reference_maps_displayed_regions_and_reported_shifts_without_changing_frequency() { let input = data(&[(120.0, 5.0, 0.2, 2.0)], 4096, 2_000.0); - let reference = CraftReference::new(input.carrier_ppm, 0.15); + let reference = CraftReference::new(input.carrier_ppm, input.observe_freq_mhz, 0.15); let params = CraftParams { regions: vec![CraftRegion::new(CraftRegionId(7), 0.38, 0.40)], fir_filter_taps: 127, ..CraftParams::default() }; - let clear_signals = preflight::detect_clear_signals(&input, reference, 0); + let clear_signals = preflight::detect_clear_signals(&input, reference, 0).unwrap(); let regions = build_modeling_windows(&input, ¶ms, reference, &clear_signals).unwrap(); assert_eq!(regions.len(), 1); assert!( @@ -308,7 +308,7 @@ fn modeling_windows_are_independent_while_components_preserve_region_identity() }; let reference = CraftReference::acquisition(&input); - let clear_signals = preflight::detect_clear_signals(&input, reference, 0); + let clear_signals = preflight::detect_clear_signals(&input, reference, 0).unwrap(); let windows = build_modeling_windows(&input, ¶ms, reference, &clear_signals).unwrap(); assert_eq!(windows.len(), 2); assert!( @@ -393,7 +393,7 @@ fn rejects_non_finite_reference() { &input, &resolve_craft_invocation( &input, - CraftReference::new(input.carrier_ppm, f64::NAN), + CraftReference::new(input.carrier_ppm, input.observe_freq_mhz, f64::NAN), &CraftParamOverrides::default(), None, ), @@ -411,7 +411,7 @@ fn rejects_reference_for_a_different_acquisition_carrier() { &input, &resolve_craft_invocation( &input, - CraftReference::new(input.carrier_ppm + 0.1, 0.0), + CraftReference::new(input.carrier_ppm + 0.1, input.observe_freq_mhz, 0.0), &CraftParamOverrides::default(), None, ), diff --git a/crates/processing/src/fft.rs b/crates/processing/src/fft.rs deleted file mode 100644 index fe2b83dc..00000000 --- a/crates/processing/src/fft.rs +++ /dev/null @@ -1,425 +0,0 @@ -use crate::{Apodization, AxisPipeline, Spectrum, StepKind, TimeTrace}; -use num_complex::Complex64; -use plotx_io::{Domain, NmrData}; -use rustfft::FftPlanner; - -/// Transform an FID into an *unphased* frequency-domain [`Spectrum`]: apply the -/// pipeline's enabled apodization windows and zero-fill, run the forward FFT -/// (removing the digital-filter group delay unless `group_delay_correct` is -/// false), `fftshift`, and build a ppm axis. Phase and other frequency-domain -/// steps are a separate cheap stage ([`crate::reapply`]). -pub fn transform_base(data: &NmrData, pipe: &AxisPipeline, group_delay_correct: bool) -> Spectrum { - let n_raw = data.len(); - if n_raw == 0 { - return Spectrum { - ppm: Vec::new(), - values: Vec::new(), - hz_per_point: 0.0, - observe_freq_mhz: data.observe_freq_mhz, - nucleus: data.nucleus.clone(), - }; - } - - if data.domain == Domain::Frequency { - let n = data.len(); - let sw = data.spectral_width_hz; - let hz_per_point = sw / n as f64; - let obs = data.observe_freq_mhz.max(f64::MIN_POSITIVE); - let half = n as f64 / 2.0; - return Spectrum { - ppm: (0..n) - .map(|i| data.carrier_ppm + (i as f64 - half) * hz_per_point / obs) - .collect(), - values: data.points.clone(), - hz_per_point, - observe_freq_mhz: data.observe_freq_mhz, - nucleus: data.nucleus.clone(), - }; - } - - let dt = data.dwell_s(); - let mut buf = apply_time_steps(data.points.clone(), pipe, dt); - let n = buf.len(); - - if data.domain == Domain::Time { - let mut planner = FftPlanner::::new(); - let fft = planner.plan_fft_forward(n); - fft.process(&mut buf); - if group_delay_correct { - remove_group_delay(&mut buf, data.group_delay); - } - } - - let shifted = fftshift(&buf); - - let sw = data.spectral_width_hz; - let hz_per_point = if n > 0 { sw / n as f64 } else { 0.0 }; - let obs = data.observe_freq_mhz.max(f64::MIN_POSITIVE); - let half = n as f64 / 2.0; - let ppm: Vec = (0..n) - .map(|i| { - let offset_hz = (i as f64 - half) * hz_per_point; - data.carrier_ppm + offset_hz / obs - }) - .collect(); - - Spectrum { - ppm, - values: shifted, - hz_per_point, - observe_freq_mhz: data.observe_freq_mhz, - nucleus: data.nucleus.clone(), - } -} - -/// Apply the enabled time-domain prefix without inventing an FFT. Callers use -/// this when the typed pipeline finishes in the time domain. -pub fn transform_time(data: &NmrData, pipe: &AxisPipeline) -> TimeTrace { - let dt = data.dwell_s(); - let values = apply_time_steps(data.points.clone(), pipe, dt); - TimeTrace { - time_s: (0..values.len()).map(|index| index as f64 * dt).collect(), - values, - nucleus: data.nucleus.clone(), - source: data.source.clone(), - } -} - -/// Apply the enabled time-domain prefix exactly in recipe order. -/// -/// Keeping this one kernel for time output and FFT input means adding/removing -/// FFT changes only the domain transition: it cannot silently reorder windows -/// or collapse multiple zero-fill steps. -pub(crate) fn apply_time_steps( - mut values: Vec, - pipe: &AxisPipeline, - dt: f64, -) -> Vec { - for step in pipe.steps.iter().filter(|step| step.enabled) { - match step.kind { - StepKind::Apodize(window) => apply_apodization(&mut values, window, dt), - StepKind::ZeroFill(fill) => { - let target = fill.target(values.len()); - values.resize(target, Complex64::new(0.0, 0.0)); - } - StepKind::Fft => break, - StepKind::Phase(_) - | StepKind::Baseline(_) - | StepKind::Reference(_) - | StepKind::Magnitude - | StepKind::Smooth(_) - | StepKind::Normalize(_) - | StepKind::Bin(_) - | StepKind::Reverse - | StepKind::Invert => {} - } - } - values -} - -pub(crate) fn time_step_output_len(mut len: usize, pipe: &AxisPipeline) -> usize { - for step in pipe.steps.iter().filter(|step| step.enabled) { - match step.kind { - StepKind::ZeroFill(fill) => len = fill.target(len), - StepKind::Fft => break, - _ => {} - } - } - len -} - -/// Apodize a FID in place over its populated samples. `t = i·dt` seconds, with -/// `dt` the sample interval; `dt` is unused by the point-index windows. -pub(crate) fn apply_apodization(buf: &mut [Complex64], apo: Apodization, dt: f64) { - let n = buf.len(); - match apo { - Apodization::None => {} - Apodization::CosineBell => { - if n <= 1 { - return; - } - let denom = (n - 1) as f64; - for (i, c) in buf.iter_mut().enumerate() { - *c *= (std::f64::consts::FRAC_PI_2 * i as f64 / denom).cos(); - } - } - Apodization::Exponential { lb_hz } => { - let k = std::f64::consts::PI * lb_hz; - for (i, c) in buf.iter_mut().enumerate() { - *c *= (-k * (i as f64 * dt)).exp(); - } - } - Apodization::Gaussian { lb_hz, gb_hz } => { - let a = std::f64::consts::PI * lb_hz; - // 4·ln2 maps the Gaussian's frequency FWHM onto its time-domain width. - let g = (std::f64::consts::PI * gb_hz).powi(2) / (4.0 * std::f64::consts::LN_2); - for (i, c) in buf.iter_mut().enumerate() { - let t = i as f64 * dt; - *c *= (a * t - g * t * t).exp(); - } - } - } -} - -// A group delay is a circular shift of the FID origin by `delay` samples, which -// by the shift theorem appears as a linear phase ramp. Use signed FFT-bin -// frequencies here: for a fractional delay, treating the upper half as positive -// frequencies puts the phase wrap at DC after `fftshift`, creating a visible -// discontinuity in the real spectrum. With signed bins the unavoidable wrap is -// at the Nyquist boundary instead. -fn remove_group_delay(spectrum: &mut [Complex64], delay: f64) { - if delay == 0.0 || !delay.is_finite() { - return; - } - let n = spectrum.len(); - if n == 0 { - return; - } - let phase_per_bin = std::f64::consts::TAU * delay / n as f64; - let negative_start = n.div_ceil(2); - for (m, c) in spectrum.iter_mut().enumerate() { - let signed_bin = if m < negative_start { - m as f64 - } else { - m as f64 - n as f64 - }; - *c *= Complex64::from_polar(1.0, phase_per_bin * signed_bin); - } -} - -fn fftshift(v: &[Complex64]) -> Vec { - let n = v.len(); - let mid = n.div_ceil(2); // pivot for both even and odd N - let mut out = Vec::with_capacity(n); - out.extend_from_slice(&v[mid..]); - out.extend_from_slice(&v[..mid]); - out -} - -#[cfg(test)] -mod tests { - use super::*; - use crate::{ - Apodization, AxisPipeline, ProcessingStep, StepId, StepKind, StepSource, ZeroFill, - }; - use plotx_io::Domain; - use std::f64::consts::TAU; - - // A detached test recipe: no dataset owns it, so numbering its own steps - // 0..n is enough to keep them distinguishable. - fn pipe(apo: Option, zf: ZeroFill) -> AxisPipeline { - let kinds = apo - .map(StepKind::Apodize) - .into_iter() - .chain([StepKind::ZeroFill(zf), StepKind::Fft]); - AxisPipeline { - steps: kinds - .enumerate() - .map(|(index, kind)| { - ProcessingStep::new(StepId::new(index as u64), kind, StepSource::User) - }) - .collect(), - } - } - - fn decaying_sinusoid( - npoints: usize, - spectral_width_hz: f64, - observe_freq_mhz: f64, - carrier_ppm: f64, - shift_ppm: f64, - group_delay: f64, - ) -> NmrData { - let dt = 1.0 / spectral_width_hz; - let freq_hz = (shift_ppm - carrier_ppm) * observe_freq_mhz; - let points = (0..npoints) - .map(|k| { - let t = k as f64 * dt; - let decay = (-t / 1.0).exp(); - Complex64::from_polar(decay, TAU * freq_hz * t) - }) - .collect(); - NmrData { - points, - domain: Domain::Time, - spectral_width_hz, - observe_freq_mhz, - carrier_ppm, - nucleus: "1H".into(), - source: "test".into(), - group_delay, - } - } - - #[test] - fn single_peak_lands_at_expected_ppm() { - let data = decaying_sinusoid(4096, 4000.0, 400.0, 0.0, 2.0, 0.0); - let s = transform_base(&data, &pipe(None, ZeroFill::None), true); - - let (idx, _) = s - .real() - .iter() - .enumerate() - .max_by(|a, b| a.1.partial_cmp(b.1).unwrap()) - .unwrap(); - let peak_ppm = s.ppm[idx]; - assert!( - (peak_ppm - 2.0).abs() < 0.05, - "peak found at {peak_ppm} ppm, expected ~2.0" - ); - } - - #[test] - fn group_delay_is_removed() { - let ideal = decaying_sinusoid(1024, 4000.0, 400.0, 0.0, 2.0, 0.0); - let n = ideal.len(); - let d = 7usize; - // Right-shift the FID by `d` points (a leading group delay), tag it. - let mut delayed = ideal.clone(); - delayed.points = (0..n).map(|k| ideal.points[(k + n - d) % n]).collect(); - delayed.group_delay = d as f64; - - let raw = pipe(None, ZeroFill::None); - let a = transform_base(&ideal, &raw, true).real(); - let b = transform_base(&delayed, &raw, true).real(); - let max_err = a - .iter() - .zip(&b) - .map(|(x, y)| (x - y).abs()) - .fold(0.0f64, f64::max); - assert!(max_err < 1e-9, "group delay not removed: max_err={max_err}"); - } - - #[test] - fn fractional_group_delay_uses_signed_fft_frequencies() { - let n = 16usize; - let delay = 3.25; - let negative_start = n.div_ceil(2); - let phase_per_bin = std::f64::consts::TAU * delay / n as f64; - let mut delayed: Vec = (0..n) - .map(|m| { - let signed_bin = if m < negative_start { - m as f64 - } else { - m as f64 - n as f64 - }; - Complex64::from_polar(1.0, -phase_per_bin * signed_bin) - }) - .collect(); - - remove_group_delay(&mut delayed, delay); - - assert!( - delayed - .iter() - .all(|value| (*value - Complex64::new(1.0, 0.0)).norm() < 1e-12), - "fractional delay correction must not introduce a phase jump at DC" - ); - } - - #[test] - fn fftshift_moves_dc_to_center() { - let v: Vec = (0..8).map(|i| Complex64::new(i as f64, 0.0)).collect(); - let s = fftshift(&v); - assert_eq!(s[4], Complex64::new(0.0, 0.0)); - } - - #[test] - fn zero_fill_target_never_shrinks() { - assert_eq!(ZeroFill::None.target(3000), 3000); - assert_eq!(ZeroFill::Factor(1).target(3000), 4096); - assert_eq!(ZeroFill::Factor(2).target(3000), 8192); - assert_eq!(ZeroFill::Size(1000).target(3000), 3000); - assert_eq!(ZeroFill::Size(9000).target(3000), 9000); - } - - #[test] - fn zero_fill_interpolates_without_moving_the_peak() { - let data = decaying_sinusoid(4096, 4000.0, 400.0, 0.0, 2.0, 0.0); - let raw = transform_base(&data, &pipe(None, ZeroFill::None), true); - let filled = transform_base(&data, &pipe(None, ZeroFill::Factor(2)), true); - assert_eq!(filled.len(), 8192); - assert!(filled.len() > raw.len()); - - let peak_ppm = |s: &Spectrum| { - let (i, _) = s - .real() - .iter() - .enumerate() - .max_by(|a, b| a.1.partial_cmp(b.1).unwrap()) - .unwrap(); - s.ppm[i] - }; - assert!((peak_ppm(&raw) - 2.0).abs() < 0.05); - assert!((peak_ppm(&filled) - 2.0).abs() < 0.05); - } - - #[test] - fn time_steps_keep_recipe_order_with_or_without_fft() { - let data = NmrData { - points: vec![Complex64::new(1.0, 0.0); 3], - domain: Domain::Time, - spectral_width_hz: 1000.0, - observe_freq_mhz: 400.0, - carrier_ppm: 0.0, - nucleus: "1H".into(), - source: "ordered time steps".into(), - group_delay: 0.0, - }; - let kinds = [ - StepKind::ZeroFill(ZeroFill::Size(5)), - StepKind::Apodize(Apodization::CosineBell), - StepKind::ZeroFill(ZeroFill::Factor(1)), - StepKind::Fft, - ]; - let spectral = AxisPipeline { - steps: kinds - .into_iter() - .enumerate() - .map(|(index, kind)| { - ProcessingStep::new(StepId::new(index as u64), kind, StepSource::User) - }) - .collect(), - }; - let mut temporal = spectral.clone(); - temporal.steps.last_mut().unwrap().enabled = false; - - let trace = transform_time(&data, &temporal); - let spectrum = transform_base(&data, &spectral, true); - assert_eq!(trace.values.len(), 8); - assert_eq!(spectrum.values.len(), trace.values.len()); - assert!((trace.values[2].re - std::f64::consts::FRAC_1_SQRT_2).abs() < 1e-12); - } - - #[test] - fn exponential_window_broadens_the_line() { - let data = decaying_sinusoid(4096, 4000.0, 400.0, 0.0, 2.0, 0.0); - let sharp = transform_base(&data, &pipe(None, ZeroFill::None), true); - let broad = transform_base( - &data, - &pipe( - Some(Apodization::Exponential { lb_hz: 20.0 }), - ZeroFill::None, - ), - true, - ); - let fwhm = |s: &Spectrum| { - let re = s.real(); - let (peak_i, &peak) = re - .iter() - .enumerate() - .max_by(|a, b| a.1.partial_cmp(b.1).unwrap()) - .unwrap(); - let half = peak / 2.0; - let count = re.iter().filter(|&&v| v >= half).count(); - let _ = peak_i; - count - }; - assert!( - fwhm(&broad) > fwhm(&sharp), - "exponential window should broaden: sharp={} broad={}", - fwhm(&sharp), - fwhm(&broad) - ); - } -} diff --git a/crates/processing/src/fft2.rs b/crates/processing/src/fft2.rs deleted file mode 100644 index 9c6fafaa..00000000 --- a/crates/processing/src/fft2.rs +++ /dev/null @@ -1,553 +0,0 @@ -use crate::fft::{apply_time_steps, time_step_output_len}; -use crate::phase::apply_slice; -use crate::{AxisMeta, Params2D, Spectrum2D, StackSpectrum}; -use num_complex::Complex64; -use plotx_io::{Domain, NmrData2D, QuadMode}; -use rustfft::FftPlanner; - -fn sample_interval(sw_hz: f64) -> f64 { - if sw_hz != 0.0 { 1.0 / sw_hz } else { 0.0 } -} - -/// Transform a 2D FID into an *unphased* frequency-domain [`Spectrum2D`]: FFT -/// along the direct (F2) axis of every row, quadrature recombination + FFT along -/// the indirect (F1) axis, `fftshift` on both, and ppm axes from the per-dimension -/// spectral widths and observe frequencies. Each axis's window and zero-fill are -/// applied before its FFT; phase is a separate cheap stage ([`reapply_phase_2d`]). -pub fn transform(data: &NmrData2D, params: &Params2D) -> Spectrum2D { - transform_cancellable(data, params, &|| false).expect("non-cancelling transform") -} - -/// Cooperative-cancellation variant used by the desktop compute service. The -/// callback is checked between FFT rows/columns, which bounds cancellation -/// latency without adding synchronization inside rustfft itself. -pub fn transform_cancellable( - data: &NmrData2D, - params: &Params2D, - cancelled: &impl Fn() -> bool, -) -> Option { - let cols = data.cols; - let rows = data.rows; - if cols == 0 || rows == 0 { - return Some(empty(data)); - } - if data.domain == Domain::Frequency { - return Some(Spectrum2D { - f2_ppm: ppm_axis( - cols, - data.direct.spectral_width_hz, - data.direct.observe_freq_mhz, - data.direct.carrier_ppm, - ), - f1_ppm: ppm_axis( - rows, - data.indirect.spectral_width_hz, - data.indirect.observe_freq_mhz, - data.indirect.carrier_ppm, - ), - f2_domain: Domain::Frequency, - f1_domain: Domain::Frequency, - data: data.data.clone(), - f2_size: cols, - f1_size: rows, - direct: AxisMeta::from(&data.direct), - indirect: AxisMeta::from(&data.indirect), - source: data.source.clone(), - }); - } - - let f2_domain = params - .f2 - .output_domain(data.domain) - .expect("live F2 pipeline is domain-valid"); - let f1_domain = params - .f1 - .output_domain(data.domain) - .expect("live F1 pipeline is domain-valid"); - let mut planner = FftPlanner::::new(); - - let f2_dt = sample_interval(data.direct.spectral_width_hz); - let f2_n = time_step_output_len(cols, ¶ms.f2); - let f2_fft = (data.domain == Domain::Time && f2_domain == Domain::Frequency) - .then(|| planner.plan_fft_forward(f2_n)); - let mut rows_ft: Vec> = Vec::with_capacity(rows); - for r in 0..rows { - if cancelled() { - return None; - } - let mut buf = apply_time_steps(data.row(r).to_vec(), ¶ms.f2, f2_dt); - if let Some(fft) = &f2_fft { - fft.process(&mut buf); - remove_group_delay(&mut buf, data.direct.group_delay); - buf = fftshift(&buf); - } - rows_ft.push(buf); - } - - // For NUS the acquired increments are reconstructed onto the full grid - // first; without a user-supplied schedule no (mirrored/aliased) spectrum is - // produced — the app surfaces a prompt to enter the sampling list. - let t1_rows = match build_t1_rows(data, &rows_ft, f2_n, &mut planner) { - Some(rows) => rows, - None => return Some(empty(data)), - }; - if cancelled() { - return None; - } - let f1_inc = t1_rows.len(); - let f1_dt = sample_interval(data.indirect.spectral_width_hz); - let f1_n = time_step_output_len(f1_inc, ¶ms.f1); - let f1_fft = (data.domain == Domain::Time && f1_domain == Domain::Frequency) - .then(|| planner.plan_fft_forward(f1_n)); - - let mut out = vec![Complex64::new(0.0, 0.0); f1_n * f2_n]; - let mut col: Vec = Vec::with_capacity(f1_n); - for c in 0..f2_n { - if cancelled() { - return None; - } - col.clear(); - for row in t1_rows.iter() { - col.push(row[c]); - } - col = apply_time_steps(std::mem::take(&mut col), ¶ms.f1, f1_dt); - if let Some(fft) = &f1_fft { - fft.process(&mut col); - col = fftshift(&col); - } - for (k, v) in col.iter().copied().enumerate() { - out[k * f2_n + c] = v; - } - } - - let f2_ppm = coordinate_axis(f2_domain, f2_n, &data.direct); - let f1_ppm = coordinate_axis(f1_domain, f1_n, &data.indirect); - - Some(Spectrum2D { - f2_ppm, - f1_ppm, - f2_domain, - f1_domain, - data: out, - f2_size: f2_n, - f1_size: f1_n, - direct: AxisMeta::from(&data.direct), - indirect: AxisMeta::from(&data.indirect), - source: data.source.clone(), - }) -} - -/// Apply the per-axis phase `(phase0, phase1, pivot_frac)` to an unphased -/// [`Spectrum2D`] from [`transform`], in place on a clone. F2 phase depends only -/// on the column (direct index), F1 phase only on the row (indirect index); both -/// are computed on the display (fftshifted) grid, exactly like the 1D kernel. -pub fn reapply_phase_2d(base: &Spectrum2D, f2: (f64, f64, f64), f1: (f64, f64, f64)) -> Spectrum2D { - reapply_phase_2d_cancellable(base, f2, f1, &|| false).expect("non-cancelling phase pass") -} - -pub fn reapply_phase_2d_cancellable( - base: &Spectrum2D, - f2: (f64, f64, f64), - f1: (f64, f64, f64), - cancelled: &impl Fn() -> bool, -) -> Option { - let mut out = base.clone(); - let nr = out.f1_size; - let nc = out.f2_size; - if nr == 0 || nc == 0 { - return Some(out); - } - let d1 = (nr - 1).max(1) as f64; - let d2 = (nc - 1).max(1) as f64; - let (f2p0, f2p1, f2piv) = f2; - let (f1p0, f1p1, f1piv) = f1; - for r in 0..nr { - if cancelled() { - return None; - } - let phi1 = f1p0 + f1p1 * (r as f64 / d1 - f1piv); - for c in 0..nc { - let phi2 = f2p0 + f2p1 * (c as f64 / d2 - f2piv); - out.data[r * nc + c] *= Complex64::from_polar(1.0, -(phi1 + phi2)); - } - } - Some(out) -} - -/// Apply the direct-axis phase `(phase0, phase1, pivot_frac)` to every trace of -/// an unphased stack. -pub fn reapply_phase_stack(base: &StackSpectrum, f2: (f64, f64, f64)) -> StackSpectrum { - reapply_phase_stack_cancellable(base, f2, &|| false).expect("non-cancelling phase pass") -} - -pub fn reapply_phase_stack_cancellable( - base: &StackSpectrum, - f2: (f64, f64, f64), - cancelled: &impl Fn() -> bool, -) -> Option { - let mut out = base.clone(); - let (p0, p1, piv) = f2; - for t in out.traces.iter_mut() { - if cancelled() { - return None; - } - apply_slice(t, p0, p1, piv); - } - Some(out) -} - -/// Pseudo-2D processing: Fourier transform only the direct dimension, keeping -/// each increment as its own 1D spectrum for a stacked display. No indirect FFT -/// or quadrature recombination is applied — the indirect axis is a parameter -/// array (gradient strength, relaxation delay, …), not a frequency. -pub fn stack(data: &NmrData2D, params: &Params2D) -> StackSpectrum { - stack_cancellable(data, params, &|| false).expect("non-cancelling stack transform") -} - -pub fn stack_cancellable( - data: &NmrData2D, - params: &Params2D, - cancelled: &impl Fn() -> bool, -) -> Option { - let cols = data.cols; - let rows = data.rows; - if cols == 0 || rows == 0 { - let direct_domain = params.f2.output_domain(data.domain).unwrap_or(data.domain); - return Some(StackSpectrum { - ppm: Vec::new(), - direct_domain, - traces: Vec::new(), - direct: AxisMeta::from(&data.direct), - source: data.source.clone(), - }); - } - let direct_domain = params - .f2 - .output_domain(data.domain) - .expect("live direct-axis pipeline is domain-valid"); - let f2_dt = sample_interval(data.direct.spectral_width_hz); - let f2_n = time_step_output_len(cols, ¶ms.f2); - let mut planner = FftPlanner::::new(); - let fft = (data.domain == Domain::Time && direct_domain == Domain::Frequency) - .then(|| planner.plan_fft_forward(f2_n)); - - // Unphased traces; the absorptive phase is derived by `reapply_phase_stack`. - let mut traces = Vec::with_capacity(rows); - for r in 0..rows { - if cancelled() { - return None; - } - let mut buf = apply_time_steps(data.row(r).to_vec(), ¶ms.f2, f2_dt); - if let Some(fft) = &fft { - fft.process(&mut buf); - remove_group_delay(&mut buf, data.direct.group_delay); - buf = fftshift(&buf); - } - traces.push(buf); - } - - let ppm = coordinate_axis(direct_domain, f2_n, &data.direct); - - Some(StackSpectrum { - ppm, - direct_domain, - traces, - direct: AxisMeta::from(&data.direct), - source: data.source.clone(), - }) -} - -// Assemble the complex t1 interferogram rows from the F2-transformed stored -// rows, applying the indirect conjugation that fixes the F1 frequency sense. -// Returns `None` only for a NUS dataset with no user-supplied schedule, so the -// caller withholds the spectrum instead of showing a wrong reconstruction. -fn build_t1_rows( - data: &NmrData2D, - rows_ft: &[Vec], - f2_n: usize, - planner: &mut FftPlanner, -) -> Option>> { - if let Some(nus) = &data.nus { - let schedule = nus.schedule.as_ref()?; - return Some(crate::nus::reconstruct_rows( - rows_ft, - nus.echo_antiecho, - schedule, - nus.grid, - f2_n, - data.indirect_conjugate, - crate::nus::DEFAULT_IST_ITERS, - planner, - )); - } - let f1_inc = f1_increments(data.rows, data.quad); - let rows: Vec> = (0..f1_inc) - .map(|k| { - (0..f2_n) - .map(|c| { - let v = combine(rows_ft, k, c, data.quad); - if data.indirect_conjugate { v.conj() } else { v } - }) - .collect() - }) - .collect(); - Some(rows) -} - -/// Number of complex indirect increments the F1 transform actually receives. -pub fn f1_increments(rows: usize, quad: QuadMode) -> usize { - match quad { - QuadMode::Complex => rows, - QuadMode::States | QuadMode::StatesTppi | QuadMode::EchoAntiecho => rows / 2, - } -} - -// Build the complex t1 sample at increment `k`, F2 point `c`, from the -// F2-transformed rows, per the indirect-dimension quadrature scheme. Each stored -// row is already a complex F2 spectrum, so the cosine/sine channels combine -// directly without needing the F2 phase. -fn combine(rows_ft: &[Vec], k: usize, c: usize, quad: QuadMode) -> Complex64 { - match quad { - QuadMode::Complex => rows_ft[k][c], - QuadMode::States => rows_ft[2 * k][c] + Complex64::i() * rows_ft[2 * k + 1][c], - QuadMode::StatesTppi => { - let sign = if (k & 1) == 0 { 1.0 } else { -1.0 }; - (rows_ft[2 * k][c] + Complex64::i() * rows_ft[2 * k + 1][c]) * sign - } - // Echo/anti-echo each select a single coherence pathway, so one row of - // the pair is already a clean phase-modulated t1 series; magnitude mode - // needs no further recombination. - QuadMode::EchoAntiecho => rows_ft[2 * k + 1][c], - } -} - -fn ppm_axis(n: usize, sw_hz: f64, obs_mhz: f64, carrier_ppm: f64) -> Vec { - let hz_per_point = if n > 0 { sw_hz / n as f64 } else { 0.0 }; - let obs = obs_mhz.max(f64::MIN_POSITIVE); - let half = n as f64 / 2.0; - (0..n) - .map(|i| carrier_ppm + (i as f64 - half) * hz_per_point / obs) - .collect() -} - -fn coordinate_axis(domain: Domain, count: usize, dim: &plotx_io::Dim) -> Vec { - match domain { - Domain::Time => { - let dwell = dim.dwell_s(); - (0..count).map(|index| index as f64 * dwell).collect() - } - Domain::Frequency => ppm_axis( - count, - dim.spectral_width_hz, - dim.observe_freq_mhz, - dim.carrier_ppm, - ), - } -} - -/// Estimate a zero-order `(phase0, phase1)` that makes the highest-energy -/// trace's tallest peak purely absorptive-positive: `phase0 = arg(peak)`, -/// `phase1 = 0`. Applied uniformly, this phases the dominant resonance — the one -/// the user reads for a relaxation/diffusion fit — exactly, with its real part -/// carrying the signal and its imaginary (dispersive) part nulled. First-order -/// correction is deliberately skipped: with truncation ringing or spinning -/// sidebands a single ramp cannot phase every peak and tends to spoil the main -/// one. `None` for an empty stack. Seeded into `f2.phase0` at load. -pub fn absorptive_phase(traces: &[Vec]) -> Option<(f64, f64)> { - let energy = |t: &[Complex64]| t.iter().map(|c| c.norm_sqr()).sum::(); - let reference = traces.iter().filter(|t| !t.is_empty()).max_by(|a, b| { - energy(a) - .partial_cmp(&energy(b)) - .unwrap_or(std::cmp::Ordering::Equal) - })?; - let peak = reference.iter().max_by(|a, b| { - a.norm() - .partial_cmp(&b.norm()) - .unwrap_or(std::cmp::Ordering::Equal) - })?; - if peak.norm() <= f64::MIN_POSITIVE { - return None; - } - // arg(peak) rotates the peak onto the positive real axis (apply_phase rotates - // by e^{-iφ}), so its absorptive lobe points up. - Some((peak.arg(), 0.0)) -} - -fn remove_group_delay(spectrum: &mut [Complex64], delay: f64) { - if delay == 0.0 || !delay.is_finite() { - return; - } - let n = spectrum.len(); - if n == 0 { - return; - } - let k = std::f64::consts::TAU * delay / n as f64; - for (m, c) in spectrum.iter_mut().enumerate() { - *c *= Complex64::from_polar(1.0, k * m as f64); - } -} - -fn fftshift(v: &[Complex64]) -> Vec { - let n = v.len(); - let mid = n.div_ceil(2); - let mut out = Vec::with_capacity(n); - out.extend_from_slice(&v[mid..]); - out.extend_from_slice(&v[..mid]); - out -} - -fn empty(data: &NmrData2D) -> Spectrum2D { - Spectrum2D { - f2_ppm: Vec::new(), - f1_ppm: Vec::new(), - f2_domain: data.domain, - f1_domain: data.domain, - data: Vec::new(), - f2_size: 0, - f1_size: 0, - direct: AxisMeta::from(&data.direct), - indirect: AxisMeta::from(&data.indirect), - source: data.source.clone(), - } -} - -#[cfg(test)] -mod tests { - use super::*; - use crate::Params2D; - use plotx_io::Dim; - use std::f64::consts::TAU; - - fn dim(sw: f64, obs: f64, nucleus: &str) -> Dim { - Dim { - spectral_width_hz: sw, - observe_freq_mhz: obs, - carrier_ppm: 0.0, - nucleus: nucleus.into(), - group_delay: 0.0, - } - } - - // A single 2D frequency: phase-modulated e^{iΩ2 t2}·e^{iΩ1 t1} with decay, so - // a forward FFT in each dimension lands one magnitude peak at (f1, f2). - fn single_peak_2d(f2_ppm: f64, f1_ppm: f64) -> NmrData2D { - let (cols, rows) = (256usize, 128usize); - let direct = dim(4000.0, 400.0, "1H"); - let indirect = dim(2000.0, 100.0, "13C"); - let dt2 = 1.0 / direct.spectral_width_hz; - let dt1 = 1.0 / indirect.spectral_width_hz; - let f2_hz = f2_ppm * direct.observe_freq_mhz; - let f1_hz = f1_ppm * indirect.observe_freq_mhz; - let mut data = Vec::with_capacity(rows * cols); - for k in 0..rows { - let t1 = k as f64 * dt1; - for j in 0..cols { - let t2 = j as f64 * dt2; - let decay = (-t2 / 0.3 - t1 / 0.3).exp(); - data.push(Complex64::from_polar( - decay, - TAU * (f2_hz * t2 + f1_hz * t1), - )); - } - } - NmrData2D { - data, - rows, - cols, - domain: Domain::Time, - direct, - indirect, - quad: QuadMode::Complex, - indirect_conjugate: false, - experiment: None, - pseudo_axis: None, - diffusion: None, - nus: None, - source: "synthetic 2D".into(), - } - } - - #[test] - fn ft_places_peak_at_expected_shifts() { - let data = single_peak_2d(2.0, 1.0); - let s = transform(&data, &Params2D::default()); - assert_eq!((s.f1_size, s.f2_size), (128, 256)); - - let mag = s.magnitude(); - let (mut best, mut br, mut bc) = (f32::MIN, 0, 0); - for r in 0..s.f1_size { - for c in 0..s.f2_size { - let v = mag[r * s.f2_size + c]; - if v > best { - best = v; - br = r; - bc = c; - } - } - } - // Tolerances are one frequency bin: ~0.04 ppm (F2), ~0.16 ppm (F1). - assert!((s.f2_ppm[bc] - 2.0).abs() < 0.05, "F2 at {}", s.f2_ppm[bc]); - assert!((s.f1_ppm[br] - 1.0).abs() < 0.2, "F1 at {}", s.f1_ppm[br]); - } - - #[test] - fn reapply_phase_makes_peak_absorptive() { - let data = single_peak_2d(2.0, 1.0); - let base = transform(&data, &Params2D::default()); - - let mag = base.magnitude(); - let (mut best, mut idx) = (f32::MIN, 0usize); - for (i, &v) in mag.iter().enumerate() { - if v > best { - best = v; - idx = i; - } - } - let arg = base.data[idx].arg(); - - // Rotating F2 by the peak's argument lands it on the positive real axis: - // real part carries the magnitude, imaginary part nulls. - let phased = reapply_phase_2d(&base, (arg, 0.0, 0.0), (0.0, 0.0, 0.0)); - let peak = phased.data[idx]; - assert!((peak.re - base.data[idx].norm()).abs() < 1e-6); - assert!(peak.im.abs() < 1e-6); - } - - #[test] - fn stack_keeps_one_spectrum_per_increment() { - let data = single_peak_2d(2.0, 1.0); - let s = stack(&data, &Params2D::default()); - assert_eq!(s.increments(), data.rows); - assert_eq!(s.ppm.len(), data.cols); - let peak_ppm = |trace: &[Complex64]| { - let (mut best, mut bi) = (f64::MIN, 0); - for (i, c) in trace.iter().enumerate() { - if c.norm() > best { - best = c.norm(); - bi = i; - } - } - s.ppm[bi] - }; - assert!((peak_ppm(&s.traces[0]) - 2.0).abs() < 0.05); - assert!((peak_ppm(&s.traces[10]) - 2.0).abs() < 0.05); - } - - #[test] - fn stack_does_not_fft_imported_frequency_rows_again() { - let mut data = single_peak_2d(2.0, 1.0); - data.domain = Domain::Frequency; - data.rows = 2; - data.cols = 3; - data.data = (0..6) - .map(|value| Complex64::new(value as f64, -(value as f64))) - .collect(); - let params = Params2D::frequency_domain(crate::Preset2D::Dosy); - - let result = stack(&data, ¶ms); - - assert_eq!(result.direct_domain, Domain::Frequency); - assert_eq!(result.traces[0], data.row(0)); - assert_eq!(result.traces[1], data.row(1)); - } -} diff --git a/crates/processing/src/lib.rs b/crates/processing/src/lib.rs index 2f6f6563..f3b191b7 100644 --- a/crates/processing/src/lib.rs +++ b/crates/processing/src/lib.rs @@ -1,37 +1,43 @@ -//! Signal processing over [`plotx_io::NmrData`]: FID → FFT → phase → baseline. +//! PlotX processing recipes, native NMR execution, and domain-specific analyses. pub mod align; pub mod arithmetic; -pub mod autophase; -pub mod baseline; -pub mod cleanup; pub mod craft; -pub mod fft; -pub mod fft2; -pub mod nus; +pub mod nmr_bridge; +pub mod nmr_execution; mod output; -pub mod phase; -mod preview; pub mod slice; pub mod timeseries; pub mod xps; pub mod xrd; pub use output::{Processed1D, TimeTrace}; -pub use preview::{Preview, process_up_to}; pub use slice::{ProjectionMode, Slice1D, SliceKind}; use num_complex::Complex64; use plotx_io::Domain; +/// Labels for scientific coordinates supported by NMR display views. +pub fn axis_unit_label(unit: Option) -> &'static str { + use nmr::axis::AxisUnit; + match unit { + Some(AxisUnit::Ppm) => "ppm", + Some(AxisUnit::Hertz) => "Hz", + Some(AxisUnit::Second) => "s", + Some(AxisUnit::TeslaPerMeter) => "T/m", + _ => "", + } +} + #[derive(Debug, Clone)] pub struct Spectrum { - /// Chemical-shift axis in ppm, ordered low → high index. The reversed NMR + /// Spectral coordinates in `unit`. The reversed NMR /// display (high ppm on the left) is a rendering concern, not applied here. pub ppm: Vec, pub values: Vec, - pub hz_per_point: f64, - pub observe_freq_mhz: f64, + pub unit: nmr::axis::AxisUnit, + pub hz_per_point: Option, + pub observe_freq_mhz: Option, pub nucleus: String, } @@ -66,6 +72,15 @@ impl Spectrum { self.points(DisplayMode::Real) } + /// Average spacing for UI bounds; numerical validation belongs to nmr. + pub fn coordinate_spacing(&self) -> Option { + if self.ppm.len() < 2 { + return None; + } + let step = (self.ppm.last()? - self.ppm.first()?).abs() / (self.ppm.len() - 1) as f64; + (step.is_finite() && step > 0.0).then_some(step) + } + pub fn ppm_bounds(&self) -> (f64, f64) { let mut lo = f64::INFINITY; let mut hi = f64::NEG_INFINITY; @@ -195,11 +210,7 @@ impl PhaseParams { pivot_frac: 0.0, auto: None, }; - /// Entropy recovers real first-order phase (tens-to-hundreds of degrees) while - /// staying clean on single peaks and under noise, and — once large spectra are - /// downsampled by peak-preserving pooling rather than plain striding (see - /// `autophase::decimate`) — phases real 13C data without spurious negative - /// peaks. See the ground-truth and large-spectrum tests in `tests.rs`. + /// Library entropy estimation with its versioned scientific quality contract. pub const AUTO: Self = Self { auto: Some(AutoPhaseMethod::Entropy), ..Self::MANUAL_ZERO @@ -678,100 +689,6 @@ impl AxisPipeline { } } -pub use fft::transform_base; - -/// Apply one frequency-domain step to an already transformed spectrum. -pub fn apply_freq_step(spec: &mut Spectrum, kind: &StepKind) { - match kind { - StepKind::Phase(p) => { - let (p0, p1, piv) = match p.auto { - Some(m) => auto_phase(spec, m), - None => (p.phase0, p.phase1, p.pivot_frac), - }; - phase::apply_with_pivot(spec, p0, p1, piv); - } - StepKind::Baseline(m) => baseline::apply(spec, *m), - StepKind::Reference(r) => { - let delta = r.target_ppm - r.at_ppm; - for p in &mut spec.ppm { - *p += delta; - } - } - StepKind::Magnitude => { - for c in &mut spec.values { - *c = Complex64::new(c.norm(), 0.0); - } - } - StepKind::Smooth(m) => cleanup::smooth(spec, *m), - StepKind::Normalize(m) => cleanup::normalize(spec, *m), - StepKind::Bin(p) => cleanup::bin(spec, *p), - StepKind::Reverse => cleanup::reverse(spec), - StepKind::Invert => cleanup::invert(spec), - StepKind::Apodize(_) | StepKind::ZeroFill(_) | StepKind::Fft => {} - } -} - -/// Cheap stage: apply the enabled frequency-domain steps in list order to an -/// unphased `base` from [`transform_base`], producing the display spectrum. -pub fn reapply(base: &Spectrum, pipe: &AxisPipeline) -> Spectrum { - let mut spec = base.clone(); - for step in &pipe.steps { - if step.enabled && !step.kind.at_or_before_fft() { - apply_freq_step(&mut spec, &step.kind); - } - } - spec -} - -pub fn transform_output_base( - data: &plotx_io::NmrData, - pipe: &AxisPipeline, - group_delay_correct: bool, -) -> Result { - match pipe.output_domain(data.domain)? { - Domain::Time => Ok(Processed1D::Time(fft::transform_time(data, pipe))), - Domain::Frequency => Ok(Processed1D::Frequency(transform_base( - data, - pipe, - group_delay_correct, - ))), - } -} - -pub fn reapply_output(base: &Processed1D, pipe: &AxisPipeline) -> Processed1D { - match base { - Processed1D::Time(trace) => Processed1D::Time(trace.clone()), - Processed1D::Frequency(spectrum) => Processed1D::Frequency(reapply(spectrum, pipe)), - } -} - -pub fn process_output( - data: &plotx_io::NmrData, - pipe: &AxisPipeline, - group_delay_correct: bool, -) -> Result { - transform_output_base(data, pipe, group_delay_correct).map(|base| reapply_output(&base, pipe)) -} - -/// Full 1D pipeline, preserving whether the recipe ends in time or frequency. -/// -/// Callers that specifically require a spectrum must inspect the returned -/// [`Processed1D`] instead of turning a valid time-domain output into a panic. -pub fn process( - data: &plotx_io::NmrData, - pipe: &AxisPipeline, - group_delay_correct: bool, -) -> Result { - process_output(data, pipe, group_delay_correct) -} - -/// Compute a phase `(phase0, phase1, pivot_frac)` from the spectrum itself, per -/// the chosen [`AutoPhaseMethod`]. The ramp pivots at the tallest peak so the -/// on-plot handle is consistent across methods. See [`autophase`] for the rules. -pub fn auto_phase(spec: &Spectrum, method: AutoPhaseMethod) -> (f64, f64, f64) { - autophase::compute(&spec.values, method) -} - fn time_side(pipe: &AxisPipeline) -> Vec<(StepKind, bool)> { pipe.steps .iter() @@ -782,7 +699,7 @@ fn time_side(pipe: &AxisPipeline) -> Vec<(StepKind, bool)> { /// Whether moving from `a` to `b` requires re-running the FFT: true iff the /// at-or-before-FFT subsequence (kinds, params, enabled, order) differs, or the -/// group-delay flags differ. Frequency-only edits need only a cheap [`reapply`]. +/// group-delay flags differ. Frequency-only edits need only a cached-base library pass. pub fn needs_retransform(a: &AxisPipeline, b: &AxisPipeline, gd_a: bool, gd_b: bool) -> bool { gd_a != gd_b || time_side(a) != time_side(b) } diff --git a/crates/processing/src/nmr_bridge.rs b/crates/processing/src/nmr_bridge.rs new file mode 100644 index 00000000..c32acba0 --- /dev/null +++ b/crates/processing/src/nmr_bridge.rs @@ -0,0 +1,568 @@ +//! Recipe compilation and staged estimation through the nmr public API. + +use crate::{Apodization, AxisPipeline, PhaseParams, StepId, StepKind, ZeroFill}; +use nmr::acquisition::GroupDelayState; +use nmr::axis::AxisDomain; +use nmr::processing::{ + DelaySource, DigitalFilterCorrection, FourierTransform, PhaseCorrection, ProcessingError, + ProcessingErrorCode, ProcessingOperation as Op, ProcessingOptions, ProcessingPlan, + SpectrumOperation, Window, +}; +use nmr::{Dataset, ExecutionContext, dataset::DescriptorRef}; +use std::{collections::BTreeSet, sync::Arc}; + +#[path = "nmr_bridge_phase.rs"] +mod phase; +pub use phase::{PhaseReport, RepresentativeTrace}; + +pub struct RecipeExecution { + pub dataset: Arc, + /// Estimation provenance for the owning recipe. Series history records the + /// shared explicit correction; this report also identifies its representative. + pub phases: Vec, +} + +#[derive(Debug, thiserror::Error)] +pub enum RecipeError { + #[error("invalid recipe: {0}")] + Invalid(String), + #[error("NMR processing failed at step {step:?}: {source}")] + Library { + step: Option, + #[source] + source: ProcessingError, + }, +} + +impl RecipeError { + pub fn is_cancelled(&self) -> bool { + matches!(self, Self::Library { source, .. } if source.code() == ProcessingErrorCode::Cancelled) + } +} + +#[derive(Clone, Copy, Debug)] +pub enum RecipeRange { + All, + /// Time prefix including the enabled FFT; cache this result before phasing. + Base, + /// Frequency suffix applied to an already cached base. + Frequency, + /// Preview through this stable step; disabled steps remain skipped. + Through(StepId), +} + +/// An invocation override, not a second persisted source of delay state. +#[derive(Clone, Copy, Debug)] +pub enum DelayPolicy { + Disabled, + AxisEvidence, + Explicit(f64), +} + +/// Bound to immutable input so a prepared recipe cannot accidentally execute +/// against a replacement dataset. Local operation positions never escape as IDs. +pub struct CompiledRecipe { + input: Arc, + segments: Vec, + step_ids: Vec, +} + +enum Segment { + Setup(ProcessingPlan), + Plan { + plan: ProcessingPlan, + ids: Vec, + }, + Phase { + axis: usize, + params: PhaseParams, + id: StepId, + }, +} + +impl CompiledRecipe { + /// A deterministic prefix for the library NUS executor. Estimators require + /// their staged input and cannot be flattened into this prefix. + pub fn deterministic_plan(&self) -> Result { + let mut operations = Vec::new(); + for segment in &self.segments { + match segment { + Segment::Setup(plan) | Segment::Plan { plan, .. } => { + operations.extend_from_slice(plan.operations()) + } + Segment::Phase { .. } => { + return Err(RecipeError::Invalid( + "NUS direct prefix cannot contain phase estimation".into(), + )); + } + } + } + ProcessingPlan::new(operations) + .map_err(|source| RecipeError::Library { step: None, source }) + } + pub fn step_id(&self, local_index: usize) -> Option { + self.step_ids.get(local_index).copied() + } + + pub fn execute( + &self, + options: ProcessingOptions, + context: &mut ExecutionContext<'_>, + ) -> Result, RecipeError> { + self.execute_with_report(options, context) + .map(|result| result.dataset) + } + + pub fn execute_with_report( + &self, + options: ProcessingOptions, + context: &mut ExecutionContext<'_>, + ) -> Result { + context + .check_cancelled() + .map_err(|error| self.locate(error.into()))?; + let mut output = Arc::clone(&self.input); + let mut phases = Vec::new(); + for segment in &self.segments { + output = Arc::new(match segment { + Segment::Setup(plan) => plan + .apply_with_context(&output, options, context) + .map_err(|source| RecipeError::Library { step: None, source })?, + Segment::Plan { plan, ids } => plan + .apply_with_context(&output, options, context) + .map_err(|source| RecipeError::Library { + step: source.step_index().and_then(|i| ids.get(i).copied()), + source, + })?, + Segment::Phase { axis, params, id } => { + let result = if let Some(method) = params.auto { + phase::apply_auto( + &output, + *axis, + *id, + phase_method(method), + options, + context, + ) + .map(|(dataset, report)| { + phases.push(report); + dataset + }) + } else { + apply_phase(&output, *axis, *params, options, context) + }; + result.map_err(|source| RecipeError::Library { + step: Some(*id), + source, + })? + } + }); + } + Ok(RecipeExecution { + dataset: output, + phases, + }) + } + + fn locate(&self, source: ProcessingError) -> RecipeError { + RecipeError::Library { + step: source.step_index().and_then(|index| self.step_id(index)), + source, + } + } +} + +pub fn compile( + input: Arc, + pipeline: &AxisPipeline, + axis: usize, + delay: DelayPolicy, + range: RecipeRange, +) -> Result { + let mut seen = BTreeSet::new(); + if pipeline.steps.iter().any(|step| !seen.insert(step.id)) { + return Err(RecipeError::Invalid("duplicate StepId".into())); + } + let end = match range { + RecipeRange::Through(id) => pipeline + .steps + .iter() + .position(|step| step.id == id) + .map(|index| index + 1) + .ok_or_else(|| RecipeError::Invalid(format!("preview step {id:?} does not exist")))?, + _ => pipeline.steps.len(), + }; + let (mut points, mut domain) = match input.descriptor() { + DescriptorRef::Raw(descriptor) => descriptor + .axes() + .get(axis) + .map(|axis| (axis.points(), axis.domain())), + DescriptorRef::Processed(descriptor) => descriptor + .axes() + .get(axis) + .map(|axis| (axis.points(), axis.domain())), + _ => None, + } + .ok_or_else(|| RecipeError::Invalid(format!("axis {axis} does not exist")))?; + let mut ops = Vec::new(); + let mut ids = Vec::new(); + let mut segments = Vec::new(); + if !matches!(range, RecipeRange::Frequency) + && let Some(raw) = input.as_raw() + { + let decoding: Vec<_> = raw + .descriptor() + .axes() + .iter() + .enumerate() + .filter_map(|(axis, value)| { + matches!( + value.kind(), + nmr::raw::RawAxisKind::Indirect(nmr::raw::IndirectComponents::Encoded(_)) + ) + .then_some(Op::ComponentTransform { axis }) + }) + .collect(); + if !decoding.is_empty() { + segments + .push(Segment::Setup(ProcessingPlan::new(decoding).map_err( + |source| RecipeError::Library { step: None, source }, + )?)); + } + } + let mut step_ids = Vec::new(); + for step in &pipeline.steps[..end] { + if !step.enabled { + continue; + } + match range { + RecipeRange::Frequency if step.kind.at_or_before_fft() => continue, + RecipeRange::Base if !step.kind.at_or_before_fft() => break, + _ => {} + } + let required = match step.kind.input_domain() { + plotx_io::Domain::Time => AxisDomain::Time, + plotx_io::Domain::Frequency => AxisDomain::Frequency, + }; + if domain != required { + return Err(RecipeError::Invalid(format!( + "step {:?} ({}) requires {required:?}, found {domain:?}", + step.id, + step.kind.label() + ))); + } + let operation = match step.kind { + StepKind::Apodize(Apodization::None) | StepKind::ZeroFill(ZeroFill::None) => None, + StepKind::Apodize(Apodization::CosineBell) => Some(Op::Window { + axis, + window: Window::SineBell { + offset: 0.5, + end: 1.0, + power: 1.0, + first_point_scale: 1.0, + }, + }), + StepKind::Apodize(Apodization::Exponential { lb_hz }) => Some(Op::Window { + axis, + window: Window::Exponential { lb_hz }, + }), + StepKind::Apodize(Apodization::Gaussian { lb_hz, gb_hz }) => Some(Op::Window { + axis, + window: Window::lorentz_to_gauss(lb_hz, gb_hz).map_err(|source| { + RecipeError::Library { + step: Some(step.id), + source, + } + })?, + }), + StepKind::ZeroFill(fill) => { + points = zero_fill_target(fill, points).map_err(|source| RecipeError::Library { + step: Some(step.id), + source, + })?; + Some(Op::ZeroFill { + axis, + zero_fill: nmr::processing::ZeroFill::new(points).map_err(|source| { + RecipeError::Library { + step: Some(step.id), + source, + } + })?, + }) + } + StepKind::Fft => { + domain = AxisDomain::Frequency; + Some(Op::FourierTransform { + axis, + transform: FourierTransform::default(), + }) + } + StepKind::Phase(params) => { + flush_plan(&mut segments, &mut ops, &mut ids)?; + segments.push(Segment::Phase { + axis, + params, + id: step.id, + }); + step_ids.push(step.id); + None + } + StepKind::Baseline(method) => Some(spectrum_op( + axis, + SpectrumOperation::Baseline(baseline(method)), + )), + StepKind::Magnitude => Some(spectrum_op(axis, SpectrumOperation::Magnitude)), + StepKind::Reference(reference) => Some(spectrum_op( + axis, + SpectrumOperation::Reference { + delta_ppm: reference.target_ppm - reference.at_ppm, + }, + )), + StepKind::Smooth(method) => Some(spectrum_op( + axis, + match method { + crate::SmoothMethod::MovingAverage { window } => { + SpectrumOperation::MovingAverage { + window: usize::from(window), + } + } + crate::SmoothMethod::SavitzkyGolay { window, poly_order } => { + SpectrumOperation::SavitzkyGolay { + window: usize::from(window), + order: usize::from(poly_order), + } + } + }, + )), + StepKind::Normalize(method) => Some(spectrum_op( + axis, + SpectrumOperation::Normalize(match method { + crate::NormalizeMethod::MaxPeak => nmr::processing::Normalization::MaxPeak, + crate::NormalizeMethod::TotalArea => { + nmr::processing::Normalization::TotalArea { + singleton_width: None, + } + } + crate::NormalizeMethod::Constant { divisor } => { + nmr::processing::Normalization::Constant(divisor) + } + }), + )), + StepKind::Bin(bin) => Some(spectrum_op( + axis, + SpectrumOperation::Bin { + width: bin.width, + aggregation: match bin.method { + crate::BinMethod::Sum => nmr::processing::BinAggregation::Sum, + crate::BinMethod::Mean => nmr::processing::BinAggregation::Mean, + }, + }, + )), + StepKind::Reverse => Some(spectrum_op(axis, SpectrumOperation::Reverse)), + StepKind::Invert => Some(spectrum_op(axis, SpectrumOperation::Invert)), + }; + if let Some(operation) = operation { + ops.push(operation); + ids.push(step.id); + step_ids.push(step.id); + } + if matches!(step.kind, StepKind::Fft) { + if let Some(correction) = + delay_correction(&input, axis, delay).map_err(|source| RecipeError::Library { + step: Some(step.id), + source, + })? + { + ops.push(Op::DigitalFilterCorrection { axis, correction }); + ids.push(step.id); + step_ids.push(step.id); + } + let has_reference = input + .as_raw() + .and_then(|raw| raw.descriptor().axes().get(axis)) + .is_some_and(|axis| axis.chemical_shift_reference().is_some()) + || input + .as_processed() + .and_then(|processed| processed.axis_evidence(axis)) + .is_some_and(|evidence| evidence.chemical_shift_reference().is_some()); + if has_reference { + ops.push(Op::ResolveFrequencyFrame { + axis, + frame: nmr::processing::FrequencyFrame::Ppm( + nmr::processing::ReferenceSource::AxisEvidence, + ), + }); + ids.push(step.id); + step_ids.push(step.id); + } + if matches!(range, RecipeRange::Base) { + break; + } + } + } + flush_plan(&mut segments, &mut ops, &mut ids)?; + Ok(CompiledRecipe { + input, + segments, + step_ids, + }) +} + +fn flush_plan( + segments: &mut Vec, + ops: &mut Vec, + ids: &mut Vec, +) -> Result<(), RecipeError> { + if !ops.is_empty() { + let plan = ProcessingPlan::new(std::mem::take(ops)) + .map_err(|source| RecipeError::Library { step: None, source })?; + segments.push(Segment::Plan { + plan, + ids: std::mem::take(ids), + }); + } + Ok(()) +} + +fn spectrum_op(axis: usize, operation: SpectrumOperation) -> Op { + Op::Spectrum { axis, operation } +} + +fn baseline(method: crate::BaselineMethod) -> nmr::processing::RealBaseline { + match method { + crate::BaselineMethod::Offset => nmr::processing::RealBaseline::Offset, + crate::BaselineMethod::Polynomial { order } => nmr::processing::RealBaseline::Polynomial { + order: usize::from(order), + }, + crate::BaselineMethod::AsymmetricLeastSquares { + smoothness, + asymmetry, + iterations, + } => nmr::processing::RealBaseline::Asls { + lambda: smoothness, + asymmetry, + iterations: usize::from(iterations), + }, + } +} + +fn phase_method(method: crate::AutoPhaseMethod) -> nmr::processing::PhaseMethod { + match method { + crate::AutoPhaseMethod::AbsorptivePeak => nmr::processing::PhaseMethod::AbsorptivePeak, + crate::AutoPhaseMethod::Entropy => nmr::processing::PhaseMethod::Entropy, + crate::AutoPhaseMethod::NegativeMinimization => { + nmr::processing::PhaseMethod::NegativeMinimization + } + crate::AutoPhaseMethod::PeakRegression => nmr::processing::PhaseMethod::PeakRegression, + crate::AutoPhaseMethod::RobustConsensus => nmr::processing::PhaseMethod::RobustConsensus, + } +} + +fn apply_phase( + input: &Dataset, + axis: usize, + params: PhaseParams, + options: ProcessingOptions, + context: &mut ExecutionContext<'_>, +) -> Result { + // A previous bin may have changed the length; resolve the endpoint convention + // from the actual intermediate descriptor, never the original input shape. + let points = input + .as_processed() + .and_then(|p| p.descriptor().axes().get(axis)) + .ok_or(ProcessingError::InvalidParameter("phase input axis"))? + .points(); + ProcessingPlan::new(vec![Op::PhaseCorrection { + axis, + correction: manual_phase(params, points)?, + }])? + .apply_with_context(input, options, context) +} + +fn zero_fill_target(fill: ZeroFill, points: usize) -> Result { + match fill { + ZeroFill::None => Ok(points), + ZeroFill::Size(size) => Ok(size.max(points)), + ZeroFill::Factor(factor) => points + .checked_next_power_of_two() + .and_then(|base| { + 1usize + .checked_shl(u32::from(factor.saturating_sub(1))) + .and_then(|multiplier| base.checked_mul(multiplier)) + }) + .ok_or(ProcessingError::SizeOverflow), + } +} + +/// PlotX uses exp(-i phi), radians and i/(N-1). nmr uses exp(+i phi), +/// degrees and i/N. Pivot is an index fraction regardless of coordinate direction. +fn manual_phase(params: PhaseParams, points: usize) -> Result { + if !params.pivot_frac.is_finite() || !(0.0..=1.0).contains(¶ms.pivot_frac) { + return Err(ProcessingError::InvalidParameter("phase pivot")); + } + if points == 1 { + PhaseCorrection::new( + (-params.phase0 + params.phase1 * params.pivot_frac).to_degrees(), + 0.0, + 0.0, + ) + } else { + let scale = points as f64 / (points - 1) as f64; + PhaseCorrection::new( + -params.phase0.to_degrees(), + -params.phase1.to_degrees() * scale, + params.pivot_frac / scale, + ) + } +} + +fn delay_correction( + input: &Dataset, + axis: usize, + policy: DelayPolicy, +) -> Result, ProcessingError> { + use nmr::processed::ProcessedGroupDelay; + let state = input + .as_raw() + .and_then(|raw| raw.descriptor().axes().get(axis)) + .map(|axis| match axis.group_delay() { + GroupDelayState::NotApplicable => ProcessedGroupDelay::NotApplicable, + GroupDelayState::Pending(delay) => ProcessedGroupDelay::Pending(delay), + _ => ProcessedGroupDelay::Unknown, + }) + .or_else(|| { + input + .as_processed()? + .axis_evidence(axis) + .map(|e| e.group_delay()) + }); + let correction = match policy { + DelayPolicy::Disabled => return Ok(None), + DelayPolicy::Explicit(0.0) => { + // Explicitly disabling correction is different from certifying unknown + // hardware delay as zero. nmr requires established zero evidence. + DigitalFilterCorrection::AcknowledgeZeroDelayV1 + } + DelayPolicy::Explicit(value) => { + DigitalFilterCorrection::FrequencyDomainPhaseRampV1(DelaySource::Explicit(value)) + } + DelayPolicy::AxisEvidence => match state { + Some(ProcessedGroupDelay::NotApplicable | ProcessedGroupDelay::Corrected { .. }) => { + return Ok(None); + } + Some(ProcessedGroupDelay::Pending(delay)) if delay.delay_points() == 0.0 => { + DigitalFilterCorrection::AcknowledgeZeroDelayV1 + } + _ => DigitalFilterCorrection::FrequencyDomainPhaseRampV1(DelaySource::AxisEvidence), + }, + }; + Ok(Some(correction)) +} + +#[cfg(test)] +#[path = "nmr_bridge_tests.rs"] +mod tests; + +#[cfg(test)] +#[path = "nmr_bridge_staged_tests.rs"] +mod staged_tests; diff --git a/crates/processing/src/nmr_bridge_phase.rs b/crates/processing/src/nmr_bridge_phase.rs new file mode 100644 index 00000000..4b3fd61b --- /dev/null +++ b/crates/processing/src/nmr_bridge_phase.rs @@ -0,0 +1,195 @@ +//! Representative selection is application policy; estimation and rotation belong to nmr. + +use super::*; +use nmr::processing::PhaseMethod; + +#[derive(Clone, Debug)] +pub struct PhaseReport { + pub step: StepId, + pub axis: usize, + pub points: usize, + pub display_pivot: usize, + pub method: PhaseMethod, + pub correction: PhaseCorrection, + pub objective: f64, + pub evaluations: usize, + pub input: nmr::provenance::CanonicalDatasetDigests, + pub representative: Option, +} + +impl PhaseReport { + /// Convert the library's exp(+i phase), i/N convention to the recipe's + /// exp(-i phase), i/(N-1) convention without estimating another correction. + pub fn recipe_parameters(&self) -> (f64, f64, f64) { + let scale = if self.points > 1 { + (self.points - 1) as f64 / self.points as f64 + } else { + 1.0 + }; + let phase1 = -self.correction.p1_degrees().to_radians() * scale; + let pivot = if self.points > 1 { + self.display_pivot as f64 / (self.points - 1) as f64 + } else { + 0.0 + }; + let phase0 = -self.correction.p0_degrees().to_radians() + + phase1 * (pivot - self.correction.pivot_fraction() / scale); + (phase0, phase1, pivot) + } +} + +/// Logical selection bound to `PhaseReport::input`, not application collection positions. +#[derive(Clone, Debug)] +pub struct RepresentativeTrace { + pub removed_axis: usize, + pub index: usize, + pub component: usize, + pub input: nmr::provenance::CanonicalDatasetDigests, +} + +pub(super) fn apply_auto( + input: &Dataset, + axis: usize, + step: StepId, + method: PhaseMethod, + options: ProcessingOptions, + context: &mut ExecutionContext<'_>, +) -> Result<(Dataset, PhaseReport), ProcessingError> { + let processed = input + .as_processed() + .ok_or(ProcessingError::InvalidParameter( + "automatic phase requires processed input", + ))?; + let axes = processed.descriptor().axes(); + if axes.len() == 1 { + let estimate = method + .prepare(input, axis, options)? + .estimate_with_context(context)?; + let output = estimate.apply_with_context(input, options, context)?; + let mut pivot = 0; + let mut peak = -1.0_f64; + for point in 0..axes[0].points() { + if point % 4096 == 0 { + context.check_cancelled()?; + } + let re = processed + .data() + .get(&[point], &[0]) + .map_err(|_| ProcessingError::InvalidParameter("phase pivot"))?; + let im = processed + .data() + .get(&[point], &[1]) + .map_err(|_| ProcessingError::InvalidParameter("phase pivot"))?; + if re.hypot(im) > peak { + pivot = point; + peak = re.hypot(im); + } + } + return Ok((output, report(input, axis, step, &estimate, None, pivot))); + } + if axes.len() != 2 || axis >= 2 { + return Err(ProcessingError::InvalidParameter("automatic phase axis")); + } + // Choose the trace through the strongest Cartesian component. Selecting by + // peak amplitude avoids overflow from summing squared, unscaled samples and + // also handles an exactly zero real plane in hypercomplex input. + let other = 1 - axis; + let shared_pair = axes.iter().any(|axis| { + matches!( + axis.component_basis(), + nmr::processed::ComponentBasis::SharedComplex { .. } + ) + }); + let data = processed.data(); + let mut peak = -1.0_f64; + let mut selection = (0, 0); + let mut display_pivot = 0; + for index in 0..axes[other].points() { + context.check_cancelled()?; + for component in 0..axes[other].component_count() { + for point in 0..axes[axis].points() { + if point % 4096 == 0 { + context.check_cancelled()?; + } + let mut coordinates = [0; 2]; + coordinates[axis] = point; + coordinates[other] = index; + for channel in 0..axes[axis].component_count() { + let mut components = [0; 2]; + components[axis] = channel; + components[other] = component; + let value = data.get(&coordinates, &components).map_err(|_| { + ProcessingError::InvalidParameter("representative phase component") + })?; + if value.abs() > peak { + peak = value.abs(); + // Shared fields form one complex trace even when its + // imaginary field is strongest; Slice selects that pair at zero. + selection = (index, if shared_pair { 0 } else { component }); + display_pivot = point; + } + } + } + } + } + let representative = ProcessingPlan::new(vec![spectrum_op( + other, + SpectrumOperation::Slice { + index: selection.0, + component: selection.1, + }, + )])? + .apply_with_context(input, options, context)?; + let phase = method + .prepare(&representative, 0, options)? + .estimate_with_context(context)?; + let output = ProcessingPlan::new(vec![Op::PhaseCorrection { + axis, + correction: phase.correction(), + }])? + .apply_with_context(input, options, context)?; + let representative = RepresentativeTrace { + removed_axis: other, + index: selection.0, + component: selection.1, + input: representative.canonical_digests(), + }; + Ok(( + output, + report( + input, + axis, + step, + &phase, + Some(representative), + display_pivot, + ), + )) +} + +fn report( + input: &Dataset, + axis: usize, + step: StepId, + phase: &nmr::processing::PhaseEstimate, + representative: Option, + display_pivot: usize, +) -> PhaseReport { + PhaseReport { + step, + axis, + points: input + .as_processed() + .expect("phase input is processed") + .descriptor() + .axes()[axis] + .points(), + display_pivot, + method: phase.method(), + correction: phase.correction(), + objective: phase.objective(), + evaluations: phase.evaluations(), + input: input.canonical_digests(), + representative, + } +} diff --git a/crates/processing/src/nmr_bridge_staged_tests.rs b/crates/processing/src/nmr_bridge_staged_tests.rs new file mode 100644 index 00000000..914d947b --- /dev/null +++ b/crates/processing/src/nmr_bridge_staged_tests.rs @@ -0,0 +1,302 @@ +use super::*; +use crate::{AutoPhaseMethod, BaselineMethod, BinMethod, BinParams, ProcessingStep, StepSource}; +use nmr::Complex64; +use nmr::axis::{AxisCoordinates, AxisRole, AxisUnit}; +use nmr::processed::{ + ComponentBasis, ProcessedAxis, ProcessedDataset, ProcessedOrigin, ProcessedProvenance, +}; + +fn spectrum(values: Vec) -> Arc { + Arc::new( + ProcessedDataset::from_complex_trace( + ProcessedAxis::new( + AxisRole::Signal, + AxisDomain::Frequency, + Some(AxisUnit::Ppm), + values.len(), + AxisCoordinates::Uniform { + start: 0.0, + step: 1.0, + }, + ComponentBasis::Cartesian, + ) + .unwrap(), + values, + ProcessedProvenance::new(ProcessedOrigin::Unknown, vec![]).unwrap(), + ) + .unwrap() + .into(), + ) +} + +fn pipe(kinds: impl IntoIterator) -> AxisPipeline { + AxisPipeline { + steps: kinds + .into_iter() + .enumerate() + .map(|(i, kind)| { + ProcessingStep::new(StepId::new(71 + i as u64 * 3), kind, StepSource::User) + }) + .collect(), + } +} + +fn run(input: Arc, pipeline: &AxisPipeline) -> Result, RecipeError> { + compile(input, pipeline, 0, DelayPolicy::Disabled, RecipeRange::All)? + .execute(ProcessingOptions::new(), &mut ExecutionContext::default()) +} + +#[test] +fn bin_then_manual_phase_resolves_the_actual_output_length() { + let params = PhaseParams { + phase0: 0.4, + phase1: 1.2, + pivot_frac: 0.3, + auto: None, + }; + let result = run( + spectrum(vec![Complex64::new(1.0, 2.0); 5]), + &pipe([ + StepKind::Bin(BinParams { + width: 2.0, + method: BinMethod::Mean, + }), + StepKind::Phase(params), + ]), + ) + .unwrap(); + let processed = result.as_processed().unwrap(); + assert_eq!( + processed.descriptor().axes()[0] + .coordinate_iter() + .unwrap() + .collect::>(), + [0.5, 2.5, 4.0] + ); + for i in 0..3 { + let expected = Complex64::new(1.0, 2.0) + * Complex64::from_polar(1.0, -(0.4 + 1.2 * (i as f64 / 2.0 - 0.3))); + let actual = Complex64::new( + processed.data().get(&[i], &[0]).unwrap(), + processed.data().get(&[i], &[1]).unwrap(), + ); + assert!((actual - expected).norm() < 1e-13); + } +} + +#[test] +fn estimator_errors_keep_the_recipe_identity_and_leave_the_input_unchanged() { + let input = spectrum(vec![Complex64::new(1.0, 2.0); 8]); + let digest = input.canonical_digests(); + let pipeline = pipe([ + StepKind::Invert, + StepKind::Phase(PhaseParams { + auto: Some(AutoPhaseMethod::PeakRegression), + ..PhaseParams::MANUAL_ZERO + }), + StepKind::Baseline(BaselineMethod::Offset), + ]); + let error = run(Arc::clone(&input), &pipeline).unwrap_err(); + assert!( + matches!(error, RecipeError::Library { step: Some(id), .. } if id == pipeline.steps[1].id) + ); + assert_eq!(input.canonical_digests(), digest); +} + +#[test] +fn estimator_and_later_segments_replay_and_survive_an_offline_snapshot() { + let input = spectrum( + (0..128) + .map(|i| { + let d = (i as f64 - 64.0) / 3.0; + Complex64::new(1.0, d) / (1.0 + d * d) * Complex64::from_polar(1.0, 0.7) + }) + .collect(), + ); + let pipeline = pipe([ + StepKind::Phase(PhaseParams { + auto: Some(AutoPhaseMethod::AbsorptivePeak), + ..PhaseParams::MANUAL_ZERO + }), + StepKind::Phase(PhaseParams { + phase0: 0.2, + ..PhaseParams::MANUAL_ZERO + }), + StepKind::Baseline(BaselineMethod::Offset), + StepKind::Invert, + ]); + let output = run(Arc::clone(&input), &pipeline).unwrap(); + let history = output + .as_processed() + .unwrap() + .provenance() + .history() + .unwrap(); + let replay = history + .replay(&[input.as_ref()], ProcessingOptions::new()) + .unwrap(); + assert_eq!(replay.canonical_digests(), output.canonical_digests()); + let mut bytes = Vec::new(); + plotx_io::nmr_bridge::snapshot::write( + &output, + &mut bytes, + Default::default(), + &mut ExecutionContext::default(), + ) + .unwrap(); + let restored = plotx_io::nmr_bridge::snapshot::read( + &mut bytes.as_slice(), + Default::default(), + &mut ExecutionContext::default(), + ) + .unwrap(); + assert_eq!(restored.canonical_digests(), output.canonical_digests()); + let after = run(restored, &pipe([StepKind::Invert])).unwrap(); + for (a, b) in after + .as_dense_processed() + .unwrap() + .samples() + .iter() + .zip(output.as_dense_processed().unwrap().samples()) + { + assert_eq!(*a, -*b); + } +} + +#[test] +fn axis_magnitude_retains_the_other_cartesian_component() { + let axis = || { + ProcessedAxis::new( + AxisRole::Signal, + AxisDomain::Frequency, + Some(AxisUnit::Ppm), + 1, + AxisCoordinates::Explicit(vec![1.0]), + ComponentBasis::Cartesian, + ) + .unwrap() + }; + let input = Arc::new( + ProcessedDataset::from_dense_samples( + nmr::processed::ProcessedDescriptor::new(vec![axis(), axis()]).unwrap(), + vec![3.0, 4.0, 5.0, 12.0], + ProcessedProvenance::new(ProcessedOrigin::Unknown, vec![]).unwrap(), + ) + .unwrap() + .into(), + ); + let output = compile( + input, + &pipe([StepKind::Magnitude]), + 1, + DelayPolicy::Disabled, + RecipeRange::All, + ) + .unwrap() + .execute(ProcessingOptions::new(), &mut ExecutionContext::default()) + .unwrap(); + assert_eq!( + output + .as_processed() + .unwrap() + .descriptor() + .component_counts(), + [2, 1] + ); + assert_eq!(output.as_dense_processed().unwrap().samples(), [5.0, 13.0]); + let rotated = run( + output, + &pipe([StepKind::Phase(PhaseParams { + phase0: std::f64::consts::FRAC_PI_2, + ..PhaseParams::MANUAL_ZERO + })]), + ) + .unwrap(); + let values = rotated.as_dense_processed().unwrap().samples(); + assert!((values[0] - 13.0).abs() < 1e-12); + assert!((values[1] + 5.0).abs() < 1e-12); +} + +#[test] +fn series_phase_uses_one_representative_and_preserves_parameter_coordinates() { + let parameter = ProcessedAxis::new( + AxisRole::ArrayParameter, + AxisDomain::Parameter, + Some(AxisUnit::Second), + 3, + AxisCoordinates::Explicit(vec![0.003, 0.001, 0.001]), + ComponentBasis::Scalar, + ) + .unwrap(); + let signal = ProcessedAxis::new( + AxisRole::Signal, + AxisDomain::Frequency, + Some(AxisUnit::Ppm), + 8, + AxisCoordinates::Uniform { + start: 4.0, + step: -0.5, + }, + ComponentBasis::Cartesian, + ) + .unwrap(); + let samples: Vec = [1.0, 3.0, -2.0] + .into_iter() + .flat_map(|scale| { + (0..8).flat_map(move |point| { + let value = Complex64::from_polar(scale * if point == 3 { 10.0 } else { 0.1 }, 0.7); + [value.re, value.im] + }) + }) + .collect(); + let input = Arc::new( + ProcessedDataset::from_dense_samples( + nmr::processed::ProcessedDescriptor::new(vec![parameter, signal]).unwrap(), + samples, + ProcessedProvenance::new(ProcessedOrigin::Unknown, vec![]).unwrap(), + ) + .unwrap() + .into(), + ); + let output = compile( + Arc::clone(&input), + &pipe([StepKind::Phase(PhaseParams { + auto: Some(AutoPhaseMethod::AbsorptivePeak), + ..PhaseParams::MANUAL_ZERO + })]), + 1, + DelayPolicy::Disabled, + RecipeRange::All, + ) + .unwrap() + .execute_with_report(ProcessingOptions::new(), &mut ExecutionContext::default()) + .unwrap(); + let report = &output.phases[0]; + assert_eq!(report.step, StepId::new(71)); + assert_eq!(report.method, nmr::processing::PhaseMethod::AbsorptivePeak); + assert_eq!(report.input, input.canonical_digests()); + let selection = report.representative.as_ref().unwrap(); + assert_eq!( + (selection.removed_axis, selection.index, selection.component), + (0, 1, 0) + ); + let processed = output.dataset.as_processed().unwrap(); + let replay = processed + .provenance() + .history() + .unwrap() + .replay(&[input.as_ref()], ProcessingOptions::new()) + .unwrap(); + assert_eq!( + replay.canonical_digests(), + output.dataset.canonical_digests() + ); + assert_eq!( + processed.descriptor().axes()[0].coordinates(), + &AxisCoordinates::Explicit(vec![0.003, 0.001, 0.001]) + ); + for (row, expected) in [10.0, 30.0, -20.0].into_iter().enumerate() { + assert!((processed.data().get(&[row, 3], &[0, 0]).unwrap() - expected).abs() < 1e-12); + assert!(processed.data().get(&[row, 3], &[0, 1]).unwrap().abs() < 1e-12); + } +} diff --git a/crates/processing/src/nmr_bridge_tests.rs b/crates/processing/src/nmr_bridge_tests.rs new file mode 100644 index 00000000..fed71f51 --- /dev/null +++ b/crates/processing/src/nmr_bridge_tests.rs @@ -0,0 +1,448 @@ +use super::*; +use crate::{ProcessingStep, StepSource}; +use nmr::processed::{ComponentBasis, ProcessedAxis, ProcessedDataset}; +use nmr::raw::{DirectSamples, RawAxis, RawAxisKind, RawDatasetBuilder, RawMetadata}; +use nmr::resource::WorkLedger; +use nmr::{ + Complex64, + axis::{AxisCoordinates, AxisRole, AxisUnit}, +}; + +fn raw(points: usize, delay: GroupDelayState) -> Arc { + let axis = RawAxis::new( + RawAxisKind::Direct(DirectSamples::Complex), + AxisDomain::Time, + Some(AxisUnit::Second), + points, + AxisCoordinates::Uniform { + start: 0.0, + step: 0.001, + }, + ) + .unwrap() + .with_group_delay(delay) + .unwrap(); + Arc::new(Dataset::from_raw( + RawDatasetBuilder::new(vec![axis], RawMetadata::default()) + .unwrap() + .dense( + (0..points) + .map(|index| { + Complex64::from_polar( + 1.0, + std::f64::consts::TAU * index as f64 / points as f64, + ) + }) + .collect(), + ) + .unwrap(), + )) +} + +#[test] +fn processed_time_input_retains_reference_and_zero_filter_evidence_across_snapshot_and_fft() { + let input = plotx_io::nmr_bridge::read( + &std::path::PathBuf::from(env!("CARGO_MANIFEST_DIR")) + .join("../io/tests/fixtures/nmr/bruker-1d"), + &mut ExecutionContext::default(), + ) + .unwrap(); + // A time-domain operation creates processed state before the FFT recipe. + let time = nmr::processing::ProcessingPlan::new(vec![Op::ZeroFill { + axis: 0, + zero_fill: nmr::processing::ZeroFill::new(8).unwrap(), + }]) + .unwrap() + .apply(&input) + .unwrap(); + assert!(time.as_processed().is_some()); + let mut bytes = Vec::new(); + plotx_io::nmr_bridge::snapshot::write( + &time, + &mut bytes, + Default::default(), + &mut ExecutionContext::default(), + ) + .unwrap(); + let restored = plotx_io::nmr_bridge::snapshot::read( + &mut bytes.as_slice(), + Default::default(), + &mut ExecutionContext::default(), + ) + .unwrap(); + let output = compile( + restored, + &pipeline([StepKind::Fft]), + 0, + DelayPolicy::AxisEvidence, + RecipeRange::All, + ) + .unwrap() + .execute(ProcessingOptions::new(), &mut ExecutionContext::default()) + .unwrap(); + let source = plotx_io::nmr_view::NmrSource::new(output.clone()).unwrap(); + assert_eq!(source.axes()[0].unit, Some(AxisUnit::Ppm)); + assert_eq!(source.reference_frequency_mhz(0), Some(400.0)); + assert!(matches!( + output + .as_processed() + .unwrap() + .axis_evidence(0) + .unwrap() + .group_delay(), + nmr::processed::ProcessedGroupDelay::Corrected { + delay_points: 0.0, + .. + } + )); +} + +fn spectrum(points: usize, descending: bool) -> Arc { + let axis = ProcessedAxis::new( + AxisRole::Signal, + AxisDomain::Frequency, + Some(AxisUnit::Ppm), + points, + AxisCoordinates::Uniform { + start: 4.0, + step: if descending { -0.1 } else { 0.1 }, + }, + ComponentBasis::Cartesian, + ) + .unwrap(); + Arc::new(Dataset::from_processed( + ProcessedDataset::from_complex_trace( + axis, + vec![Complex64::new(1.0, 2.0); points], + nmr::processed::ProcessedProvenance::new( + nmr::processed::ProcessedOrigin::Unknown, + vec![], + ) + .unwrap(), + ) + .unwrap(), + )) +} + +fn pipeline(kinds: impl IntoIterator) -> AxisPipeline { + AxisPipeline { + steps: kinds + .into_iter() + .enumerate() + .map(|(index, kind)| { + ProcessingStep::new(StepId::new(100 + index as u64 * 3), kind, StepSource::User) + }) + .collect(), + } +} + +fn execute( + input: Arc, + pipe: &AxisPipeline, + range: RecipeRange, +) -> Result, RecipeError> { + compile(input, pipe, 0, DelayPolicy::Disabled, range)? + .execute(ProcessingOptions::new(), &mut ExecutionContext::default()) +} + +#[test] +fn phase_preserves_radians_sign_endpoint_pivot_singleton_and_axis_direction() { + let params = PhaseParams { + phase0: 0.7, + phase1: -1.4, + pivot_frac: 0.37, + auto: None, + }; + for points in [1, 2, 5, 6] { + for descending in [false, true] { + let output = execute( + spectrum(points, descending), + &pipeline([StepKind::Phase(params)]), + RecipeRange::All, + ) + .unwrap(); + let data = output.as_dense_processed().unwrap(); + for index in 0..points { + let phi = params.phase0 + + params.phase1 + * (index as f64 / (points - 1).max(1) as f64 - params.pivot_frac); + let expected = Complex64::new(1.0, 2.0) * Complex64::from_polar(1.0, -phi); + let actual = Complex64::new( + data.get(&[index], &[0]).unwrap(), + data.get(&[index], &[1]).unwrap(), + ); + assert!( + (actual - expected).norm() < 2e-14, + "{points}/{index}/{descending}" + ); + } + } + } +} + +#[test] +fn fft_matches_analytic_tone_and_integer_center_for_odd_and_even_lengths() { + for points in [1, 5, 6] { + let output = execute( + raw(points, GroupDelayState::NotApplicable), + &pipeline([StepKind::Fft]), + RecipeRange::All, + ) + .unwrap(); + let processed = output.as_processed().unwrap(); + let axis = &processed.descriptor().axes()[0]; + for index in 0..points { + let q = index as isize - (points / 2) as isize; + assert!( + (axis.coordinate(index).unwrap() - q as f64 * 1000.0 / points as f64).abs() < 1e-12 + ); + let expected = if points == 1 || q == 1 { + points as f64 + } else { + 0.0 + }; + assert!((processed.data().get(&[index], &[0]).unwrap() - expected).abs() < 1e-12); + assert!(processed.data().get(&[index], &[1]).unwrap().abs() < 1e-12); + } + } +} + +#[test] +fn reordered_windows_and_zero_fills_use_current_length_and_preview_identity() { + let pipe = pipeline([ + StepKind::ZeroFill(ZeroFill::Size(5)), + StepKind::Apodize(Apodization::CosineBell), + StepKind::ZeroFill(ZeroFill::Factor(1)), + StepKind::Fft, + ]); + let input = raw(3, GroupDelayState::NotApplicable); + let preview = execute( + Arc::clone(&input), + &pipe, + RecipeRange::Through(pipe.steps[2].id), + ) + .unwrap(); + let data = preview.as_dense_processed().unwrap(); + assert_eq!(data.shape(), &[8]); + let expected = Complex64::from_polar( + std::f64::consts::FRAC_1_SQRT_2, + 4.0 * std::f64::consts::PI / 3.0, + ); + assert!((data.get(&[2], &[0]).unwrap() - expected.re).abs() < 1e-14); + assert!((data.get(&[2], &[1]).unwrap() - expected.im).abs() < 1e-14); + assert_eq!(data.get(&[7], &[0]).unwrap(), 0.0); + let output = execute(input, &pipe, RecipeRange::All).unwrap(); + assert_eq!(output.as_dense_processed().unwrap().shape(), &[8]); +} + +#[test] +fn split_cache_matches_full_recipe_and_shares_work_budget() { + let pipe = pipeline([ + StepKind::Apodize(Apodization::Exponential { lb_hz: 1.0 }), + StepKind::Fft, + StepKind::Phase(PhaseParams { + phase0: 0.3, + ..PhaseParams::MANUAL_ZERO + }), + ]); + let input = raw(8, GroupDelayState::NotApplicable); + let whole = execute(Arc::clone(&input), &pipe, RecipeRange::All).unwrap(); + let mut ledger = WorkLedger::processing_default(); + let mut context = ExecutionContext::new(&mut ledger); + let base = compile(input, &pipe, 0, DelayPolicy::Disabled, RecipeRange::Base) + .unwrap() + .execute(ProcessingOptions::new(), &mut context) + .unwrap(); + let output = compile( + base, + &pipe, + 0, + DelayPolicy::Disabled, + RecipeRange::Frequency, + ) + .unwrap() + .execute(ProcessingOptions::new(), &mut context) + .unwrap(); + assert_eq!(whole.as_dense_processed(), output.as_dense_processed()); + assert!(ledger.used() > 0); +} + +#[test] +fn disabling_fft_preserves_time_output_and_disabled_preview_endpoint() { + let mut pipe = pipeline([ + StepKind::Apodize(Apodization::Exponential { lb_hz: 2.0 }), + StepKind::Fft, + ]); + pipe.steps[1].enabled = false; + let result = execute( + raw(8, GroupDelayState::NotApplicable), + &pipe, + RecipeRange::Through(pipe.steps[1].id), + ) + .unwrap(); + assert_eq!( + result.as_processed().unwrap().descriptor().axes()[0].domain(), + AxisDomain::Time + ); +} + +#[test] +fn unknown_delay_is_not_zero_and_library_error_maps_to_fft_id() { + let pipe = pipeline([StepKind::Apodize(Apodization::None), StepKind::Fft]); + let input = raw(8, GroupDelayState::Unknown); + let recipe = compile(input, &pipe, 0, DelayPolicy::AxisEvidence, RecipeRange::All).unwrap(); + assert_eq!(recipe.step_id(0), Some(pipe.steps[1].id)); + assert_eq!(recipe.step_id(1), Some(pipe.steps[1].id)); + let error = recipe + .execute(ProcessingOptions::new(), &mut ExecutionContext::default()) + .unwrap_err(); + assert!(matches!(error, RecipeError::Library { step: Some(id), .. } if id == pipe.steps[1].id)); + assert!(!error.is_cancelled()); +} + +#[test] +fn new_spectrum_steps_execute_without_changing_the_input() { + let input = spectrum(8, false); + let before = input.canonical_digests(); + let output = execute( + Arc::clone(&input), + &pipeline([ + StepKind::Reference(crate::ReferenceParams { + at_ppm: 4.0, + target_ppm: 1.0, + }), + StepKind::Reverse, + StepKind::Invert, + StepKind::Baseline(crate::BaselineMethod::Offset), + ]), + RecipeRange::All, + ) + .unwrap(); + assert_eq!(input.canonical_digests(), before); + let processed = output.as_processed().unwrap(); + assert_eq!(processed.descriptor().axes()[0].coordinate(0).unwrap(), 1.0); + for i in 0..8 { + assert_eq!(processed.data().get(&[i], &[0]).unwrap(), 0.0); + assert_eq!(processed.data().get(&[i], &[1]).unwrap(), -2.0); + } +} + +#[test] +fn cancellation_is_distinct_including_an_empty_recipe_and_budget_is_enforced() { + let token = nmr::CancellationToken::new(); + token.cancel(); + let mut context = ExecutionContext::default().with_cancellation(token); + let input = raw(8, GroupDelayState::NotApplicable); + let recipe = compile( + Arc::clone(&input), + &pipeline([]), + 0, + DelayPolicy::Disabled, + RecipeRange::All, + ) + .unwrap(); + assert!( + recipe + .execute(ProcessingOptions::new(), &mut context) + .unwrap_err() + .is_cancelled() + ); + let mut ledger = WorkLedger::new(0); + let recipe = compile( + input, + &pipeline([StepKind::Fft]), + 0, + DelayPolicy::Disabled, + RecipeRange::All, + ) + .unwrap(); + let error = recipe + .execute( + ProcessingOptions::new(), + &mut ExecutionContext::new(&mut ledger), + ) + .unwrap_err(); + assert!( + matches!(error, RecipeError::Library { source, .. } if source.code() == ProcessingErrorCode::ResourceLimit) + ); +} + +#[test] +fn overflow_and_stale_preview_ids_are_recoverable() { + let input = raw(8, GroupDelayState::NotApplicable); + assert!( + compile( + Arc::clone(&input), + &pipeline([StepKind::ZeroFill(ZeroFill::Factor(255))]), + 0, + DelayPolicy::Disabled, + RecipeRange::All + ) + .is_err() + ); + assert!( + compile( + input, + &pipeline([StepKind::Fft]), + 0, + DelayPolicy::Disabled, + RecipeRange::Through(StepId::new(17)) + ) + .is_err() + ); +} + +#[test] +fn magnitude_uses_library_scalar_projection_without_inventing_an_imaginary_channel() { + let output = execute( + spectrum(5, false), + &pipeline([StepKind::Magnitude]), + RecipeRange::All, + ) + .unwrap(); + let data = output.as_processed().unwrap(); + assert_eq!(data.descriptor().component_counts(), [1]); + assert!( + data.data() + .samples() + .iter() + .all(|value| (*value - 5.0f64.sqrt()).abs() < 1e-14) + ); +} + +#[test] +fn explicit_fractional_delay_uses_signed_bins_and_conflicting_evidence_fails() { + let pipe = pipeline([StepKind::Fft]); + let input = raw(5, GroupDelayState::Unknown); + let result = compile( + input, + &pipe, + 0, + DelayPolicy::Explicit(0.25), + RecipeRange::All, + ) + .unwrap() + .execute(ProcessingOptions::new(), &mut ExecutionContext::default()) + .unwrap(); + let expected = Complex64::from_polar(5.0, std::f64::consts::TAU * 0.25 / 5.0); + let data = result.as_dense_processed().unwrap(); + assert!((data.get(&[3], &[0]).unwrap() - expected.re).abs() < 1e-12); + assert!((data.get(&[3], &[1]).unwrap() - expected.im).abs() < 1e-12); + let known = raw( + 5, + GroupDelayState::Pending(nmr::raw::PendingGroupDelay::user_constructed(0.5).unwrap()), + ); + let error = compile( + known, + &pipe, + 0, + DelayPolicy::Explicit(0.25), + RecipeRange::All, + ) + .unwrap() + .execute(ProcessingOptions::new(), &mut ExecutionContext::default()) + .unwrap_err(); + assert!( + matches!(error, RecipeError::Library { source, .. } if matches!(source.root_cause(), ProcessingError::DelayEvidenceMismatch)) + ); +} diff --git a/crates/processing/src/nmr_execution.rs b/crates/processing/src/nmr_execution.rs new file mode 100644 index 00000000..48ac967c --- /dev/null +++ b/crates/processing/src/nmr_execution.rs @@ -0,0 +1,113 @@ +//! Application execution retains the full library output beside disposable views. + +use crate::nmr_bridge::{self, DelayPolicy, PhaseReport, RecipeError, RecipeRange}; +use crate::{AxisPipeline, Processed1D, Spectrum, TimeTrace}; +use nmr::axis::{AxisDomain, AxisUnit}; +use nmr::{ExecutionContext, processing::ProcessingOptions}; +use plotx_io::nmr_view::NmrSource; + +#[path = "nmr_execution_2d.rs"] +mod twod; +pub use twod::{NusRequest, Output2D, execute_2d, validate_2d_domains, view_2d}; + +/// One bounded ledger for the complete F2/NUS/F1 task and its frequency suffix. +/// A 512 x 1024 NUS acquisition needs over 40 billion preflight work units; +/// the library default is too small even though its memory use is modest. +pub fn processing_2d_work_ledger() -> nmr::resource::WorkLedger { + nmr::resource::WorkLedger::new(100_000_000_000) +} + +#[derive(Debug, thiserror::Error)] +pub enum ExecutionError { + #[error(transparent)] + Recipe(#[from] RecipeError), + #[error(transparent)] + View(#[from] plotx_io::IoError), +} + +impl ExecutionError { + pub fn is_cancelled(&self) -> bool { + matches!(self, Self::Recipe(error) if error.is_cancelled()) + } +} + +#[derive(Clone, Debug)] +pub struct Output1D { + pub source: NmrSource, + pub view: Processed1D, + pub phases: Vec, +} + +pub fn execute_1d( + source: &NmrSource, + pipeline: &AxisPipeline, + delay: DelayPolicy, + range: RecipeRange, + context: &mut ExecutionContext<'_>, +) -> Result { + let result = nmr_bridge::compile(source.dataset().clone(), pipeline, 0, delay, range)? + .execute_with_report(ProcessingOptions::default(), context)?; + let source = NmrSource::new(result.dataset)?.with_display_label(source.source().to_owned()); + let view = view_1d(&source)?; + Ok(Output1D { + source, + view, + phases: result.phases, + }) +} + +pub fn view_1d(source: &NmrSource) -> Result { + let axis = source.direct_axis()?; + check_view_size(1, axis.points)?; + let coordinates = axis.coordinate_values()?; + let values = source.trace()?; + let nucleus = axis.nucleus.clone().unwrap_or_default(); + match (axis.domain, axis.unit) { + (AxisDomain::Time, Some(AxisUnit::Second)) => Ok(Processed1D::Time(TimeTrace { + time_s: coordinates, + values, + nucleus, + source: source.source().to_owned(), + })), + (AxisDomain::Frequency, Some(unit @ (AxisUnit::Ppm | AxisUnit::Hertz))) => { + Ok(Processed1D::Frequency(Spectrum { + ppm: coordinates, + values, + unit, + hz_per_point: match &axis.coordinates { + nmr::axis::AxisCoordinates::Uniform { step, .. } => { + if unit == AxisUnit::Hertz { + Some(step.abs()) + } else { + source + .reference_frequency_mhz(0) + .map(|frequency| frequency * step.abs()) + } + } + _ => None, + }, + observe_freq_mhz: axis.observe_frequency_mhz(), + nucleus, + })) + } + _ => Err(plotx_io::IoError::NmrConversion( + "NMR display requires coordinates in seconds, hertz or ppm".into(), + )), + } +} + +/// Bound disposable host views separately from the library's scientific output. +/// This includes row vectors, axes, magnitude and the temporary flattening copy. +pub(super) fn check_view_size(rows: usize, cols: usize) -> Result<(), plotx_io::IoError> { + let bytes = rows + .checked_mul(cols) + .and_then(|points| points.checked_mul(40)) + .and_then(|bytes| rows.checked_mul(32)?.checked_add(bytes)) + .and_then(|bytes| cols.checked_mul(8)?.checked_add(bytes)); + if bytes.is_none_or(|bytes| bytes > 512 * 1024 * 1024) { + return Err(plotx_io::IoError::NmrConversion( + "NMR display exceeds the 512 MiB view allocation limit".into(), + )); + } + Ok(()) +} diff --git a/crates/processing/src/nmr_execution_2d.rs b/crates/processing/src/nmr_execution_2d.rs new file mode 100644 index 00000000..e18666ec --- /dev/null +++ b/crates/processing/src/nmr_execution_2d.rs @@ -0,0 +1,280 @@ +//! Two-dimensional execution keeps Cartesian lanes and sparse inputs in nmr. + +use super::{ExecutionError, nmr_bridge}; +use crate::nmr_bridge::{DelayPolicy, PhaseReport, RecipeError, RecipeRange}; +use crate::{AxisMeta, Layout2D, Params2D, Processed2D, Spectrum2D, StackSpectrum}; +use nmr::axis::{AxisDomain, AxisUnit}; +use nmr::processing::{AutoNusSettings, NusSettings, ProcessingOptions}; +use nmr::{Complex64, ExecutionContext}; +use plotx_io::{Domain, IoError, nmr_view::NmrSource}; +use std::sync::Arc; + +#[derive(Clone, Debug)] +pub struct Output2D { + pub source: NmrSource, + pub view: Processed2D, + pub phases: Vec, +} + +#[derive(Clone, Copy, Debug, PartialEq, serde::Serialize, serde::Deserialize)] +pub struct NusRequest { + /// Library iteration ceiling, 1..=2048; exhaustion is an error. + pub max_iterations: usize, + /// An independent noise estimate in the processed F2 spectrum amplitude + /// units. Absence selects library automatic estimation, never zero noise. + pub noise_standard_deviation: Option, +} + +impl Default for NusRequest { + fn default() -> Self { + Self { + max_iterations: 1000, + noise_standard_deviation: None, + } + } +} + +fn library(source: nmr::processing::ProcessingError) -> ExecutionError { + RecipeError::Library { step: None, source }.into() +} + +pub fn validate_2d_domains( + source: &plotx_io::nmr_series::NmrSeriesSource, + params: &Params2D, +) -> Result<(), String> { + for (index, name, pipeline) in [(1, "F2", ¶ms.f2), (0, "F1", ¶ms.f1)] { + if source.source_dataset().axes()[index].domain == AxisDomain::Parameter { + if pipeline.steps.iter().any(|step| step.enabled) { + return Err(format!( + "invalid {name} pipeline: parameter axes cannot accept spectral processing" + )); + } + } else { + pipeline + .output_domain( + source + .input_domain(index) + .map_err(|error| error.to_string())?, + ) + .map_err(|error| format!("invalid {name} pipeline: {error}"))?; + } + } + Ok(()) +} + +pub fn execute_2d( + source: &NmrSource, + params: &Params2D, + delay: DelayPolicy, + range: RecipeRange, + nus: Option, + context: &mut ExecutionContext<'_>, +) -> Result { + if source.axes().len() != 2 { + return Err(RecipeError::Invalid("2D processing requires two axes".into()).into()); + } + if matches!(range, RecipeRange::All) { + let base = execute_2d(source, params, delay, RecipeRange::Base, nus, context)?; + return execute_2d( + &base.source, + params, + DelayPolicy::Disabled, + RecipeRange::Frequency, + None, + context, + ); + } + let options = ProcessingOptions::default(); + let direct = nmr_bridge::compile(source.dataset().clone(), ¶ms.f2, 1, delay, range)?; + let sparse = source + .dataset() + .as_raw() + .is_some_and(|raw| raw.data().is_sparse()); + let mut phases = Vec::new(); + let mut output = + if sparse && params.f2.has_enabled_fft() && !matches!(range, RecipeRange::Frequency) { + let request = nus.unwrap_or_default(); + let plan = direct.deterministic_plan()?; + let prepared = if request.noise_standard_deviation.is_some() { + NusSettings { + max_iterations: request.max_iterations, + noise_standard_deviation: request.noise_standard_deviation, + } + .prepare_with_context(source.dataset(), plan, options, context) + } else { + AutoNusSettings { + max_iterations: request.max_iterations, + } + .prepare_with_context(source.dataset(), plan, options, context) + } + .map_err(library)?; + Arc::new(prepared.execute_with_context(context).map_err(library)?) + } else if sparse && !params.f2.has_enabled_fft() { + // Only an unchanged acquisition can be displayed before reconstruction. + if params.f2.steps.iter().any(|step| step.enabled) { + return Err(RecipeError::Invalid( + "Reconstruct NUS data before applying a time-domain recipe".into(), + ) + .into()); + } + source.dataset().clone() + } else { + let result = direct.execute_with_report(options, context)?; + phases.extend(result.phases); + result.dataset + }; + if params.layout == Layout2D::Ft { + let result = + nmr_bridge::compile(output.clone(), ¶ms.f1, 0, DelayPolicy::Disabled, range)? + .execute_with_report(options, context)?; + output = result.dataset; + phases.extend(result.phases); + } + let source = NmrSource::new(output)?.with_display_label(source.source().to_owned()); + let view = view_2d(&source, params.layout, context)?; + Ok(Output2D { + source, + view, + phases, + }) +} + +fn domain(axis: &plotx_io::nmr_view::NmrAxis) -> Result { + match (axis.domain, axis.unit) { + (AxisDomain::Time, Some(AxisUnit::Second)) => Ok(Domain::Time), + (AxisDomain::Frequency, Some(AxisUnit::Ppm | AxisUnit::Hertz)) => Ok(Domain::Frequency), + _ => Err(IoError::NmrConversion( + "Spectral display requires an axis in seconds, hertz or ppm".into(), + )), + } +} + +pub fn view_2d( + source: &NmrSource, + layout: Layout2D, + context: &mut ExecutionContext<'_>, +) -> Result { + let axes = source.axes(); + if axes.len() != 2 { + return Err(RecipeError::Invalid("2D view requires two axes".into()).into()); + } + let direct_domain = domain(&axes[1])?; + let cols = axes[1].points; + let rows = source + .dataset() + .as_raw() + .and_then(|raw| raw.data().sparse_traces()) + .map_or(axes[0].points, |traces| traces.len()); + super::check_view_size(rows, cols)?; + let direct = AxisMeta { + nucleus: axes[1].nucleus.clone().unwrap_or_default(), + observe_freq_mhz: axes[1].observe_frequency_mhz(), + unit: axes[1].unit, + }; + let ppm = axes[1].coordinate_values()?; + let mut traces = Vec::new(); + let mut magnitudes = Vec::new(); + if let Some(raw) = source.dataset().as_raw() { + let lanes = raw.descriptor().axes()[0].component_lanes(); + let rows = raw + .data() + .sparse_traces() + .map_or(axes[0].points, |traces| traces.len()); + if raw.data().is_sparse() && layout == Layout2D::Ft { + return Err(RecipeError::Invalid( + "Reconstruct the NUS grid before displaying contours".into(), + ) + .into()); + } + for row in 0..rows { + context.check_cancelled().map_err(|e| library(e.into()))?; + let trace = if raw.data().is_sparse() { + raw.read_observation(nmr::raw::ObservationOrdinal::new(row)) + } else { + raw.read_trace(&[row]) + } + .map_err(|error| IoError::NmrConversion(error.to_string()))?; + let samples = trace.samples(); + traces.push(samples[..cols].to_vec()); + for col in 0..cols { + if col % 4096 == 0 { + context + .check_cancelled() + .map_err(|error| library(error.into()))?; + } + let mut magnitude = 0.0_f64; + for lane in 0..lanes { + magnitude = magnitude.hypot(samples[lane * cols + col].norm()); + } + magnitudes.push(magnitude); + } + } + } else { + let data = source + .dataset() + .as_processed() + .ok_or_else(|| RecipeError::Invalid("unsupported 2D representation".into()))?; + let descriptor = data.descriptor(); + let sample = |row, col, indirect, direct| { + data.data() + .get(&[row, col], &[indirect, direct]) + .map_err(|error| IoError::NmrConversion(error.to_string())) + }; + for row in 0..axes[0].points { + context.check_cancelled().map_err(|e| library(e.into()))?; + let mut trace = Vec::with_capacity(cols); + for col in 0..cols { + if col % 4096 == 0 { + context + .check_cancelled() + .map_err(|error| library(error.into()))?; + } + trace.push(Complex64::new( + sample(row, col, 0, 0)?, + if descriptor.axes()[1].component_count() == 2 { + sample(row, col, 0, 1)? + } else { + 0.0 + }, + )); + // A display reduction of every Cartesian field. The canonical + // dataset and any requested magnitude operation stay in nmr. + let mut magnitude = 0.0_f64; + for indirect in 0..descriptor.axes()[0].component_count() { + for direct in 0..descriptor.axes()[1].component_count() { + magnitude = magnitude.hypot(sample(row, col, indirect, direct)?); + } + } + magnitudes.push(magnitude); + } + traces.push(trace); + } + } + let source_label = source.source().to_owned(); + Ok(match layout { + Layout2D::Stack => Processed2D::Stack(Arc::new(StackSpectrum { + ppm, + direct_domain, + traces, + direct, + source: source_label, + })), + Layout2D::Ft => Processed2D::Ft(Arc::new(Spectrum2D { + f2_ppm: ppm, + f1_ppm: axes[0].coordinate_values()?, + f2_domain: direct_domain, + f1_domain: domain(&axes[0])?, + data: traces.into_iter().flatten().collect(), + magnitude_plane: Some(Arc::from(magnitudes)), + f2_size: cols, + f1_size: axes[0].points, + direct, + indirect: AxisMeta { + nucleus: axes[0].nucleus.clone().unwrap_or_default(), + observe_freq_mhz: axes[0].observe_frequency_mhz(), + unit: axes[0].unit, + }, + source: source_label, + })), + }) +} diff --git a/crates/processing/src/nmr_test_execution.rs b/crates/processing/src/nmr_test_execution.rs new file mode 100644 index 00000000..dd213130 --- /dev/null +++ b/crates/processing/src/nmr_test_execution.rs @@ -0,0 +1,94 @@ +//! Execute the historical signal-quality fixtures through the production bridge. +use super::*; +use crate::nmr_bridge::{DelayPolicy, RecipeRange}; +use plotx_io::nmr_view::NmrSource; +fn run( + data: &NmrData, + pipeline: &AxisPipeline, + delay: bool, + range: RecipeRange, +) -> Result { + let source = NmrSource::try_from(data.clone()).map_err(|e| e.to_string())?; + crate::nmr_execution::execute_1d( + &source, + pipeline, + if delay { + DelayPolicy::AxisEvidence + } else { + DelayPolicy::Disabled + }, + range, + &mut nmr::ExecutionContext::default(), + ) + .map(|out| out.view) + .map_err(|e| e.to_string()) +} +pub fn process( + data: &NmrData, + pipeline: &AxisPipeline, + delay: bool, +) -> Result { + run(data, pipeline, delay, RecipeRange::All) +} +pub fn process_output( + data: &NmrData, + pipeline: &AxisPipeline, + delay: bool, +) -> Result { + process(data, pipeline, delay) +} +pub fn transform_base(data: &NmrData, pipeline: &AxisPipeline, delay: bool) -> Spectrum { + match run(data, pipeline, delay, RecipeRange::Base).unwrap() { + Processed1D::Frequency(s) => s, + _ => panic!("test requires FFT"), + } +} +pub fn process_up_to( + data: &NmrData, + pipeline: &AxisPipeline, + delay: bool, + step: StepId, +) -> Processed1D { + run(data, pipeline, delay, RecipeRange::Through(step)).unwrap() +} +pub fn apply_phase(spectrum: &mut Spectrum, method: AutoPhaseMethod) { + let axis = nmr::processed::ProcessedAxis::new( + nmr::axis::AxisRole::Signal, + nmr::axis::AxisDomain::Frequency, + Some(spectrum.unit), + spectrum.len(), + nmr::axis::AxisCoordinates::Explicit(spectrum.ppm.clone()), + nmr::processed::ComponentBasis::Cartesian, + ) + .unwrap(); + let input = nmr::processed::ProcessedDataset::from_complex_trace( + axis, + spectrum.values.clone(), + nmr::processed::ProcessedProvenance::new( + nmr::processed::ProcessedOrigin::Unknown, + Vec::new(), + ) + .unwrap(), + ) + .unwrap(); + let source = NmrSource::new(std::sync::Arc::new(input.into())).unwrap(); + let pipeline = AxisPipeline { + steps: vec![ProcessingStep::new( + StepId::new(0), + StepKind::Phase(PhaseParams { + auto: Some(method), + ..PhaseParams::MANUAL_ZERO + }), + StepSource::User, + )], + }; + let out = crate::nmr_execution::execute_1d( + &source, + &pipeline, + DelayPolicy::Disabled, + RecipeRange::All, + &mut nmr::ExecutionContext::default(), + ) + .unwrap(); + spectrum.values = out.view.as_frequency().unwrap().values.clone(); +} diff --git a/crates/processing/src/nus.rs b/crates/processing/src/nus.rs deleted file mode 100644 index ef4c3804..00000000 --- a/crates/processing/src/nus.rs +++ /dev/null @@ -1,188 +0,0 @@ -//! Non-uniform-sampling (NUS) indirect-axis reconstruction. -//! -//! JEOL echo/anti-echo HSQC (and other P/N experiments) select a single -//! coherence pathway per stored channel, so the two F1 channels are the echo -//! (P) and anti-echo (N) interferograms rather than States cosine/sine. Feeding -//! P/N straight into a `cos + i·sin` assembly places each peak at both ±Ω (the -//! F1 mirror); [`pn_to_shr`] first remaps them to States-Haberkorn-Ruben -//! channels so the assembly resolves a single frequency. -//! -//! Only a subset (M of N nominal) increments are acquired; [`ist`] fills the -//! gaps by iterative soft thresholding: transform, shrink all but the strongest -//! F1 components, inverse-transform, and re-impose the measured samples, so the -//! reconstruction stays consistent with the acquired data while favouring a -//! sparse (peak-like) spectrum. - -use num_complex::Complex64; -use rustfft::FftPlanner; - -/// Convert an echo/anti-echo channel pair `(P, N)` at one F1 increment and F2 -/// point into the complex States t1 sample. `cos = (P + N)/2`, `sin = (P − N)/2i`, -/// and the hypercomplex assembly `cos + i·sin` collapses to a single-frequency -/// sample (no ±Ω mirror). Sign conventions vary between spectrometers; the -/// caller applies the indirect conjugation that fixes the F1 sense. -#[inline] -pub fn pn_to_shr(p: Complex64, n: Complex64) -> Complex64 { - let cos = (p + n) * 0.5; - // sin = (P − N) / (2i) = −i·(P − N)/2. - let sin = (p - n) * Complex64::new(0.0, -0.5); - cos + Complex64::i() * sin -} - -/// Number of IST iterations when the recipe does not specify one. -pub const DEFAULT_IST_ITERS: usize = 100; - -/// Reconstruct one dense length-`grid` complex t1 interferogram from the sparse -/// measured samples by iterative soft thresholding. -/// -/// `measured[j]` holds the acquired sample for grid index `positions[j]`; all -/// other grid points start at zero and are filled by the iteration. `iters` -/// passes shrink the F1 spectrum toward the largest components (threshold -/// decays geometrically), inverse-transform, then restore the measured points. -pub fn ist( - positions: &[usize], - measured: &[Complex64], - grid: usize, - iters: usize, - planner: &mut FftPlanner, -) -> Vec { - let mut x = vec![Complex64::new(0.0, 0.0); grid]; - for (&pos, &m) in positions.iter().zip(measured) { - if pos < grid { - x[pos] = m; - } - } - if grid == 0 || iters == 0 { - return x; - } - let fwd = planner.plan_fft_forward(grid); - let inv = planner.plan_fft_inverse(grid); - let norm = 1.0 / grid as f64; - // Threshold starts just below the strongest component and decays - // geometrically so weaker peaks are admitted progressively, recovering - // essentially the full peak list by the last pass. - let decay = 0.98_f64; - let mut spec = vec![Complex64::new(0.0, 0.0); grid]; - for i in 0..iters { - spec.copy_from_slice(&x); - fwd.process(&mut spec); - let max_amp = spec.iter().map(|c| c.norm()).fold(0.0, f64::max); - if max_amp <= f64::MIN_POSITIVE { - break; - } - let threshold = max_amp * decay.powi(i as i32); - for c in spec.iter_mut() { - let amp = c.norm(); - if amp <= threshold { - *c = Complex64::new(0.0, 0.0); - } else { - // Soft threshold: shrink the surviving magnitude by `threshold`. - *c *= (amp - threshold) / amp; - } - } - inv.process(&mut spec); - for (dst, src) in x.iter_mut().zip(spec.iter()) { - *dst = src * norm; - } - // Data consistency: re-impose the measured samples exactly. - for (&pos, &m) in positions.iter().zip(measured) { - if pos < grid { - x[pos] = m; - } - } - } - x -} - -/// Reconstruct the full `grid × f2_n` complex t1 interferogram grid from the -/// acquired (F2-transformed) rows. For echo/anti-echo the stored rows are P/N -/// pairs remapped by [`pn_to_shr`]; otherwise each stored row is one measured -/// increment. Each F2 column is reconstructed independently along F1 by [`ist`]. -/// `positions` holds the 0-based grid index of each acquired increment. -#[allow(clippy::too_many_arguments)] -pub fn reconstruct_rows( - rows_ft: &[Vec], - echo_antiecho: bool, - positions: &[usize], - grid: usize, - f2_n: usize, - indirect_conjugate: bool, - iters: usize, - planner: &mut FftPlanner, -) -> Vec> { - let acquired = positions.len(); - let mut out = vec![vec![Complex64::new(0.0, 0.0); f2_n]; grid]; - let mut measured = vec![Complex64::new(0.0, 0.0); acquired]; - for c in 0..f2_n { - for (k, m) in measured.iter_mut().enumerate() { - let s = if echo_antiecho { - pn_to_shr(rows_ft[2 * k][c], rows_ft[2 * k + 1][c]) - } else { - rows_ft[k][c] - }; - // The P/N→SHR remap already fixes the F1 sense, so echo/anti-echo takes - // the opposite conjugation to the plain States/phase-modulated path. - *m = if indirect_conjugate ^ echo_antiecho { - s.conj() - } else { - s - }; - } - let col = ist(positions, &measured, grid, iters, planner); - for (g, v) in col.into_iter().enumerate() { - out[g][c] = v; - } - } - out -} - -#[cfg(test)] -mod tests { - use super::*; - use std::f64::consts::TAU; - - #[test] - fn pn_to_shr_resolves_single_frequency() { - // P = e^{+iΩ}, N = e^{-iΩ}: the States assembly of the raw pair peaks at - // both ±Ω, but pn_to_shr yields e^{+iΩ} (a single frequency). - let omega = 0.7; - let p = Complex64::from_polar(1.0, omega); - let n = Complex64::from_polar(1.0, -omega); - let t1 = pn_to_shr(p, n); - assert!((t1 - Complex64::from_polar(1.0, omega)).norm() < 1e-12); - // The naive States mix keeps a mirror term of comparable size. - let naive = p + Complex64::i() * n; - assert!(naive.norm() > 0.5); - } - - #[test] - fn ist_recovers_a_sparse_spectrum() { - // One F1 tone sampled at a NUS subset of a 32-point grid must reconstruct - // to a single peak at the right frequency with the gaps filled. - let grid = 32usize; - let k0 = 5usize; // target frequency bin - let full: Vec = (0..grid) - .map(|t| Complex64::from_polar(1.0, TAU * k0 as f64 * t as f64 / grid as f64)) - .collect(); - let positions = [0usize, 1, 2, 4, 7, 9, 13, 18, 21, 25, 29, 31]; - let measured: Vec = positions.iter().map(|&p| full[p]).collect(); - let mut planner = FftPlanner::::new(); - let recon = ist(&positions, &measured, grid, 200, &mut planner); - - let mut spec = recon.clone(); - planner.plan_fft_forward(grid).process(&mut spec); - let peak = spec - .iter() - .enumerate() - .max_by(|a, b| a.1.norm().partial_cmp(&b.1.norm()).unwrap()) - .unwrap() - .0; - assert_eq!( - peak, k0, - "reconstructed peak lands at the sampled frequency" - ); - for (&p, &m) in positions.iter().zip(measured.iter()) { - assert!((recon[p] - m).norm() < 1e-6); - } - } -} diff --git a/crates/processing/src/phase.rs b/crates/processing/src/phase.rs deleted file mode 100644 index 7aef5213..00000000 --- a/crates/processing/src/phase.rs +++ /dev/null @@ -1,114 +0,0 @@ -use crate::Spectrum; -use num_complex::Complex64; - -pub fn apply(spec: &mut Spectrum, phase0: f64, phase1: f64) { - apply_with_pivot(spec, phase0, phase1, 0.0); -} - -/// Fractional index (`0..=1`) of the largest-magnitude point — a sensible default -/// first-order phase pivot, so the ramp rotates about the tallest peak. Returns -/// `0.0` for an empty or single-point buffer. -pub fn peak_pivot_frac(values: &[Complex64]) -> f64 { - let n = values.len(); - if n < 2 { - return 0.0; - } - let peak = values - .iter() - .enumerate() - .max_by(|(_, a), (_, b)| a.norm().total_cmp(&b.norm())) - .map_or(0, |(i, _)| i); - peak as f64 / (n - 1) as f64 -} - -/// Zeroth- and first-order phase correction in place. The first-order ramp -/// rotates around `pivot_frac` (a `0..=1` fractional index): the phase there is -/// exactly `phase0`. `pivot_frac = 0.0` ramps from the first point. -pub fn apply_with_pivot(spec: &mut Spectrum, phase0: f64, phase1: f64, pivot_frac: f64) { - apply_slice(&mut spec.values, phase0, phase1, pivot_frac); -} - -/// The phase-correction kernel over a raw complex buffer, shared by the 1D -/// [`Spectrum`] path and each dimension of a 2D spectrum. Rotates point `i` by -/// `e^{-iφ}`, `φ = phase0 + phase1·(i/(n−1) − pivot_frac)`. -pub fn apply_slice(buf: &mut [Complex64], phase0: f64, phase1: f64, pivot_frac: f64) { - let n = buf.len(); - if n == 0 { - return; - } - let denom = (n - 1).max(1) as f64; - for (i, c) in buf.iter_mut().enumerate() { - let frac = i as f64 / denom; - let phi = phase0 + phase1 * (frac - pivot_frac); - *c *= Complex64::from_polar(1.0, -phi); - } -} - -#[cfg(test)] -mod tests { - use super::*; - - fn spectrum_from(values: Vec) -> Spectrum { - let n = values.len(); - Spectrum { - ppm: (0..n).map(|i| i as f64).collect(), - values, - hz_per_point: 1.0, - observe_freq_mhz: 400.0, - nucleus: "1H".into(), - } - } - - #[test] - fn zero_phase_is_identity() { - let mut s = spectrum_from(vec![Complex64::new(1.0, 2.0), Complex64::new(-3.0, 0.5)]); - let before = s.values.clone(); - apply(&mut s, 0.0, 0.0); - for (a, b) in before.iter().zip(&s.values) { - assert!((a - b).norm() < 1e-12); - } - } - - #[test] - fn peak_pivot_frac_lands_on_tallest_point() { - let vals = vec![ - Complex64::new(0.1, 0.0), - Complex64::new(0.2, 0.0), - Complex64::new(5.0, 0.0), - Complex64::new(0.3, 0.0), - Complex64::new(0.1, 0.0), - ]; - assert!((peak_pivot_frac(&vals) - 0.5).abs() < 1e-12); - assert_eq!(peak_pivot_frac(&[]), 0.0); - assert_eq!(peak_pivot_frac(&[Complex64::new(9.0, 0.0)]), 0.0); - } - - #[test] - fn phase0_rotates_imag_into_real() { - // (0 + i)·e^(-iπ/2) = 1. - let mut s = spectrum_from(vec![Complex64::new(0.0, 1.0)]); - apply(&mut s, std::f64::consts::FRAC_PI_2, 0.0); - assert!((s.values[0].re - 1.0).abs() < 1e-12); - assert!(s.values[0].im.abs() < 1e-12); - } - - #[test] - fn repivot_preserves_the_phase_curve() { - let mut params = crate::PhaseParams { - phase0: 0.7, - phase1: -1.4, - pivot_frac: 0.2, - auto: None, - }; - let before = [0.0, 0.25, 0.8, 1.0] - .map(|frac| params.phase0 + params.phase1 * (frac - params.pivot_frac)); - - params.repivot(0.75); - - let after = [0.0, 0.25, 0.8, 1.0] - .map(|frac| params.phase0 + params.phase1 * (frac - params.pivot_frac)); - for (before, after) in before.into_iter().zip(after) { - assert!((before - after).abs() < 1e-12); - } - } -} diff --git a/crates/processing/src/preview.rs b/crates/processing/src/preview.rs deleted file mode 100644 index 23339eea..00000000 --- a/crates/processing/src/preview.rs +++ /dev/null @@ -1,68 +0,0 @@ -use crate::{AxisPipeline, Spectrum, StepId, StepKind, apply_freq_step, fft, transform_base}; -use num_complex::Complex64; - -/// An intermediate pipeline output: time-domain before FFT, frequency-domain -/// after an enabled FFT. -#[derive(Debug, Clone)] -pub enum Preview { - Time { fid: Vec, dt: f64 }, - Freq(Spectrum), -} - -/// Run enabled steps until (and including) `stop`. -pub fn process_up_to( - data: &plotx_io::NmrData, - pipe: &AxisPipeline, - group_delay_correct: bool, - stop: StepId, -) -> Preview { - let dt = if data.spectral_width_hz != 0.0 { - 1.0 / data.spectral_width_hz - } else { - 0.0 - }; - let stop_before_fft = pipe - .steps - .iter() - .take_while(|step| !(step.enabled && matches!(step.kind, StepKind::Fft))) - .any(|step| step.id == stop); - - if stop_before_fft || !pipe.has_enabled_fft() { - let mut buf = data.points.clone(); - for step in &pipe.steps { - if step.enabled { - match step.kind { - StepKind::Apodize(apodization) => { - fft::apply_apodization(&mut buf, apodization, dt); - } - StepKind::ZeroFill(zero_fill) => { - let size = zero_fill.target(buf.len()); - buf.resize(size, Complex64::new(0.0, 0.0)); - } - _ => {} - } - } - if step.id == stop { - break; - } - } - return Preview::Time { fid: buf, dt }; - } - - let mut spectrum = transform_base(data, pipe, group_delay_correct); - for step in &pipe.steps { - if step.kind.at_or_before_fft() { - if step.id == stop { - break; - } - continue; - } - if step.enabled { - apply_freq_step(&mut spectrum, &step.kind); - } - if step.id == stop { - break; - } - } - Preview::Freq(spectrum) -} diff --git a/crates/processing/src/slice.rs b/crates/processing/src/slice.rs index e8ae38b4..65872c9d 100644 --- a/crates/processing/src/slice.rs +++ b/crates/processing/src/slice.rs @@ -6,7 +6,7 @@ use num_complex::Complex64; use plotx_io::Domain; -use crate::{Spectrum2D, StackSpectrum}; +use crate::Spectrum2D; /// The orientation of a 1D cut through a true-2D spectrum. `Row`/`Column` name /// the axis the resulting trace runs *along*. @@ -35,7 +35,9 @@ pub struct Slice1D { pub domain: Domain, pub values: Vec, pub nucleus: String, - pub observe_freq_mhz: f64, + pub observe_freq_mhz: Option, + pub reference_freq_mhz: Option, + pub unit: nmr::axis::AxisUnit, /// The fixed-axis coordinate the cut was taken at, for labelling. /// `None` for a projection, which spans the whole axis. pub position: Option, @@ -52,83 +54,6 @@ impl Spectrum2D { pub fn nearest_f1(&self, ppm: f64) -> usize { nearest(&self.f1_ppm, ppm) } - - /// A single row/column cut at a grid index (clamped in range). - pub fn slice(&self, kind: SliceKind, index: usize) -> Slice1D { - match kind { - SliceKind::Row => { - let r = index.min(self.f1_size.saturating_sub(1)); - let start = r * self.f2_size; - Slice1D { - coordinates: self.f2_ppm.clone(), - domain: self.f2_domain, - values: self.data[start..start + self.f2_size].to_vec(), - nucleus: self.direct.nucleus.clone(), - observe_freq_mhz: self.direct.observe_freq_mhz, - position: self.f1_ppm.get(r).copied(), - position_domain: self.f1_domain, - } - } - SliceKind::Column => { - let c = index.min(self.f2_size.saturating_sub(1)); - Slice1D { - coordinates: self.f1_ppm.clone(), - domain: self.f1_domain, - values: (0..self.f1_size).map(|r| self.at(r, c)).collect(), - nucleus: self.indirect.nucleus.clone(), - observe_freq_mhz: self.indirect.observe_freq_mhz, - position: self.f2_ppm.get(c).copied(), - position_domain: self.f2_domain, - } - } - } - } - - /// A whole-axis projection. `kind` names the surviving axis (as for - /// [`Self::slice`]): a `Row` projection collapses F1 to give intensity vs F2, - /// a `Column` projection collapses F2 to give intensity vs F1. - pub fn project(&self, kind: SliceKind, mode: ProjectionMode) -> Slice1D { - match kind { - SliceKind::Row => Slice1D { - coordinates: self.f2_ppm.clone(), - domain: self.f2_domain, - values: (0..self.f2_size) - .map(|c| reduce((0..self.f1_size).map(|r| self.at(r, c)), mode)) - .collect(), - nucleus: self.direct.nucleus.clone(), - observe_freq_mhz: self.direct.observe_freq_mhz, - position: None, - position_domain: self.f1_domain, - }, - SliceKind::Column => Slice1D { - coordinates: self.f1_ppm.clone(), - domain: self.f1_domain, - values: (0..self.f1_size) - .map(|r| reduce((0..self.f2_size).map(|c| self.at(r, c)), mode)) - .collect(), - nucleus: self.indirect.nucleus.clone(), - observe_freq_mhz: self.indirect.observe_freq_mhz, - position: None, - position_domain: self.f2_domain, - }, - } - } -} - -impl StackSpectrum { - /// One increment's direct-dimension 1D trace (clamped in range). - pub fn slice(&self, increment: usize) -> Slice1D { - let i = increment.min(self.increments().saturating_sub(1)); - Slice1D { - coordinates: self.ppm.clone(), - domain: self.direct_domain, - values: self.traces.get(i).cloned().unwrap_or_default(), - nucleus: self.direct.nucleus.clone(), - observe_freq_mhz: self.direct.observe_freq_mhz, - position: None, - position_domain: self.direct_domain, - } - } } fn nearest(axis: &[f64], ppm: f64) -> usize { @@ -139,90 +64,82 @@ fn nearest(axis: &[f64], ppm: f64) -> usize { .unwrap_or(0) } -fn reduce(values: impl Iterator, mode: ProjectionMode) -> Complex64 { - match mode { - ProjectionMode::Sum => values.sum(), - ProjectionMode::Skyline => values.fold(Complex64::new(0.0, 0.0), |best, c| { - if c.norm() > best.norm() { c } else { best } - }), - } +/// A native reduction removes the selected dimension and its explicitly chosen +/// real component. The surviving axis retains all of its Cartesian components. +#[derive(Clone, Copy, Debug)] +pub enum Reduction { + Slice(usize), + Projection(ProjectionMode), } -#[cfg(test)] -mod tests { - use super::*; - use crate::AxisMeta; - - fn spectrum() -> Spectrum2D { - // 2 rows (F1) × 3 cols (F2): row r, col c carries value (r*10 + c). - let (f2_size, f1_size) = (3, 2); - let data = (0..f1_size) - .flat_map(|r| (0..f2_size).map(move |c| Complex64::new((r * 10 + c) as f64, 0.0))) - .collect(); - Spectrum2D { - f2_domain: plotx_io::Domain::Frequency, - f1_domain: plotx_io::Domain::Frequency, - f2_ppm: vec![1.0, 2.0, 3.0], - f1_ppm: vec![10.0, 20.0], - data, - f2_size, - f1_size, - direct: AxisMeta { - nucleus: "1H".into(), - observe_freq_mhz: 400.0, - }, - indirect: AxisMeta { - nucleus: "13C".into(), - observe_freq_mhz: 100.0, - }, - source: "t".into(), - } - } - - #[test] - fn row_slice_is_a_full_f2_trace_at_fixed_f1() { - let s = spectrum(); - let row = s.slice(SliceKind::Row, 1); - assert_eq!(row.coordinates, vec![1.0, 2.0, 3.0]); - assert_eq!(row.domain, Domain::Frequency); - assert_eq!( - row.values.iter().map(|c| c.re).collect::>(), - vec![10.0, 11.0, 12.0] - ); - assert_eq!(row.nucleus, "1H"); - assert_eq!(row.position, Some(20.0)); - assert_eq!(row.position_domain, Domain::Frequency); - } - - #[test] - fn column_slice_is_a_full_f1_trace_at_fixed_f2() { - let s = spectrum(); - let col = s.slice(SliceKind::Column, 2); - assert_eq!(col.coordinates, vec![10.0, 20.0]); - assert_eq!( - col.values.iter().map(|c| c.re).collect::>(), - vec![2.0, 12.0] - ); - assert_eq!(col.nucleus, "13C"); - assert_eq!(col.position, Some(3.0)); - } - - #[test] - fn sum_projection_collapses_the_other_axis() { - let s = spectrum(); - let proj = s.project(SliceKind::Row, ProjectionMode::Sum); - // Column c sums rows: (0+10), (1+11), (2+12). - assert_eq!( - proj.values.iter().map(|c| c.re).collect::>(), - vec![10.0, 12.0, 14.0] - ); - assert_eq!(proj.position, None); - } - - #[test] - fn nearest_index_snaps_to_the_grid() { - let s = spectrum(); - assert_eq!(s.nearest_f2(2.4), 1); - assert_eq!(s.nearest_f1(18.0), 1); +pub fn extract( + source: &plotx_io::nmr_view::NmrSource, + kind: SliceKind, + reduction: Reduction, +) -> Result<(plotx_io::nmr_view::NmrSource, Slice1D), String> { + use crate::Processed1D; + use nmr::processing::{ + ProcessingOperation, ProcessingOptions, ProcessingPlan, SpectrumOperation, + }; + let axis = match kind { + SliceKind::Row => 0, + SliceKind::Column => 1, + }; + let axes = source.axes(); + if axes.len() != 2 { + return Err("Slice extraction requires two axes".into()); } + let operation = match reduction { + Reduction::Slice(index) => SpectrumOperation::Slice { + index, + component: 0, + }, + Reduction::Projection(ProjectionMode::Sum) => SpectrumOperation::Sum { component: 0 }, + Reduction::Projection(ProjectionMode::Skyline) => { + SpectrumOperation::Skyline { component: 0 } + } + }; + let output = ProcessingPlan::new(vec![ProcessingOperation::Spectrum { axis, operation }]) + .and_then(|plan| { + plan.apply_with_context( + source.dataset(), + ProcessingOptions::default(), + &mut nmr::ExecutionContext::default(), + ) + }) + .map_err(|error| error.to_string())?; + let output = plotx_io::nmr_view::NmrSource::new(std::sync::Arc::new(output)) + .map_err(|error| error.to_string())?; + let view = crate::nmr_execution::view_1d(&output).map_err(|error| error.to_string())?; + let (coordinates, domain, values) = match view { + Processed1D::Frequency(s) => (s.ppm, Domain::Frequency, s.values), + Processed1D::Time(t) => (t.time_s, Domain::Time, t.values), + }; + let surviving = &output.axes()[0]; + let position = match reduction { + Reduction::Slice(index) => axes[axis] + .coordinate_values() + .map_err(|error| error.to_string())? + .get(index) + .copied(), + Reduction::Projection(_) => None, + }; + let slice = Slice1D { + coordinates, + domain, + values, + reference_freq_mhz: output.reference_frequency_mhz(0), + nucleus: surviving.nucleus.clone().unwrap_or_default(), + observe_freq_mhz: surviving.observe_frequency_mhz(), + unit: surviving + .unit + .ok_or_else(|| "Slice has no spectral unit".to_owned())?, + position, + position_domain: if axes[axis].domain == nmr::axis::AxisDomain::Time { + Domain::Time + } else { + Domain::Frequency + }, + }; + Ok((output, slice)) } diff --git a/crates/processing/src/tests.rs b/crates/processing/src/tests.rs index 80c959f2..b6c043e9 100644 --- a/crates/processing/src/tests.rs +++ b/crates/processing/src/tests.rs @@ -1,9 +1,12 @@ //! Unit tests for the processing pipeline and 2D transforms. use super::*; +#[path = "nmr_test_execution.rs"] +mod execution; +use execution::*; use plotx_io::{Dim, Domain, NmrData2D, QuadMode}; -fn data2d(exp: Option<&str>, f2_nuc: &str, f1_nuc: &str) -> NmrData2D { +fn data2d(exp: Option<&str>, f2_nuc: &str, f1_nuc: &str) -> plotx_io::nmr_series::NmrSeriesSource { let dim = |nuc: &str| Dim { spectral_width_hz: 1000.0, observe_freq_mhz: 400.0, @@ -12,9 +15,9 @@ fn data2d(exp: Option<&str>, f2_nuc: &str, f1_nuc: &str) -> NmrData2D { group_delay: 0.0, }; NmrData2D { - data: Vec::new(), - rows: 0, - cols: 0, + data: vec![Complex64::new(1.0, 0.0); 4], + rows: 2, + cols: 2, domain: Domain::Time, direct: dim(f2_nuc), indirect: dim(f1_nuc), @@ -26,6 +29,8 @@ fn data2d(exp: Option<&str>, f2_nuc: &str, f1_nuc: &str) -> NmrData2D { nus: None, source: String::new(), } + .try_into() + .unwrap() } #[test] @@ -279,8 +284,9 @@ mod groundtruth { Spectrum { ppm: (0..n).map(|i| i as f64).collect(), values, - hz_per_point: 1.0, - observe_freq_mhz: 400.0, + unit: nmr::axis::AxisUnit::Ppm, + hz_per_point: Some(1.0), + observe_freq_mhz: Some(400.0), nucleus: "1H".into(), } } @@ -318,8 +324,7 @@ mod groundtruth { ) -> f64 { let truth = clean(n, peaks); let mut s = spec(scramble(&truth, a0, a1, noise)); - let (p0, p1, piv) = auto_phase(&s, m); - phase::apply_with_pivot(&mut s, p0, p1, piv); + apply_phase(&mut s, m); residual(&s.values, &truth) } @@ -566,15 +571,15 @@ fn process_up_to_returns_time_then_freq() { steps: vec![apo, fft], }; match process_up_to(&data, &pipe, true, apo_id) { - Preview::Time { fid, dt } => { - assert_eq!(fid.len(), data.len()); - assert!((dt - 1.0 / data.spectral_width_hz).abs() < 1e-12); + Processed1D::Time(trace) => { + assert_eq!(trace.values.len(), data.len()); + assert!((trace.time_s[1] - 1.0 / data.spectral_width_hz).abs() < 1e-12); } _ => panic!("expected time-domain preview"), } assert!(matches!( process_up_to(&data, &pipe, true, fft_id), - Preview::Freq(_) + Processed1D::Frequency(_) )); } diff --git a/crates/processing/src/twod.rs b/crates/processing/src/twod.rs index 76b10474..a90d1c56 100644 --- a/crates/processing/src/twod.rs +++ b/crates/processing/src/twod.rs @@ -8,14 +8,22 @@ use std::sync::Arc; #[derive(Debug, Clone)] pub struct AxisMeta { pub nucleus: String, - pub observe_freq_mhz: f64, + pub observe_freq_mhz: Option, + pub unit: Option, +} + +impl AxisMeta { + pub fn unit_label(&self) -> &'static str { + crate::axis_unit_label(self.unit) + } } impl From<&plotx_io::Dim> for AxisMeta { fn from(d: &plotx_io::Dim) -> Self { Self { nucleus: d.nucleus.clone(), - observe_freq_mhz: d.observe_freq_mhz, + observe_freq_mhz: Some(d.observe_freq_mhz), + unit: Some(nmr::axis::AxisUnit::Ppm), } } } @@ -81,7 +89,7 @@ impl Preset2D { /// Best-guess preset for a dataset from its experiment hint and nuclei. Pseudo-2D /// families (DOSY, relaxation) are matched first; otherwise homo- vs /// heteronuclear is decided from the two axes' nuclei. -pub fn recommend_preset(data: &plotx_io::NmrData2D) -> Preset2D { +pub fn recommend_preset(data: &plotx_io::nmr_series::NmrSeriesSource) -> Preset2D { // A recovered indirect ruler is the strongest signal: some JEOL relaxation // arrays carry no relaxation keyword in the experiment name, but do embed a // delay/gradient `y_acq` axis. Trust it over the hint. @@ -123,7 +131,9 @@ pub fn recommend_preset(data: &plotx_io::NmrData2D) -> Preset2D { if has(&["cosy"]) { return Preset2D::Cosy; } - if data.direct.nucleus == data.indirect.nucleus { + if data.direct.nucleus.is_empty() || data.indirect.nucleus.is_empty() { + Preset2D::Generic + } else if data.direct.nucleus == data.indirect.nucleus { Preset2D::Cosy } else { Preset2D::Hsqc @@ -178,11 +188,11 @@ pub fn needs_retransform_2d(a: &Params2D, b: &Params2D) -> bool { /// indirect one. #[derive(Debug, Clone)] pub struct Spectrum2D { - /// Coordinate values for F2. The historical field name is retained for - /// project-internal compatibility; `f2_domain` decides whether values are - /// ppm or acquisition seconds. + /// Full Cartesian magnitude projected for display, including indirect lanes. + pub magnitude_plane: Option>, + /// Coordinate values for F2, interpreted through `direct.unit`. pub f2_ppm: Vec, - /// Coordinate values for F1; interpreted through `f1_domain`. + /// Coordinate values for F1, interpreted through `indirect.unit`. pub f1_ppm: Vec, pub f2_domain: plotx_io::Domain, pub f1_domain: plotx_io::Domain, @@ -207,7 +217,18 @@ impl Spectrum2D { /// Row-major magnitude grid, `f1_size × f2_size`. pub fn magnitude(&self) -> Vec { - self.data.iter().map(|c| c.norm() as f32).collect() + self.magnitude_plane.as_ref().map_or_else( + || self.data.iter().map(|c| c.norm() as f32).collect(), + |plane| plane.iter().map(|value| *value as f32).collect(), + ) + } + + /// Full Cartesian magnitude at a checked row-major view position. + pub fn magnitude_at(&self, index: usize) -> Option { + self.magnitude_plane.as_ref().map_or_else( + || self.data.get(index).map(|value| value.norm()), + |plane| plane.get(index).copied(), + ) } /// Row-major real (absorption) grid, `f1_size × f2_size`. Meaningful once @@ -225,7 +246,10 @@ impl Spectrum2D { } pub fn max_magnitude(&self) -> f64 { - self.data.iter().map(|c| c.norm()).fold(0.0, f64::max) + self.magnitude_plane.as_ref().map_or_else( + || self.data.iter().map(|c| c.norm()).fold(0.0, f64::max), + |plane| plane.iter().copied().fold(0.0, f64::max), + ) } /// Default first-order phase pivots `(f2_frac, f1_frac)` at the tallest peak. @@ -315,161 +339,6 @@ pub enum Processed2D { Stack(Arc), } -/// Transform a 2D acquisition into an *unphased* frequency-domain result. This -/// is the expensive stage (FFT + window + zero-fill); the app caches it as the -/// `base` and re-derives the phased, display-ready spectrum with [`reapply_2d`]. -pub fn process_2d(data: &plotx_io::NmrData2D, params: &Params2D) -> Processed2D { - process_2d_cancellable(data, params, &|| false).expect("non-cancelling 2D transform") -} - -pub fn process_2d_cancellable( - data: &plotx_io::NmrData2D, - params: &Params2D, - cancelled: &impl Fn() -> bool, -) -> Option { - match params.layout { - Layout2D::Ft => fft2::transform_cancellable(data, params, cancelled) - .map(Arc::new) - .map(Processed2D::Ft), - Layout2D::Stack => fft2::stack_cancellable(data, params, cancelled) - .map(Arc::new) - .map(Processed2D::Stack), - } -} - -/// Cheap stage: apply the enabled frequency-domain steps in `params` to an -/// unphased `base` from [`process_2d`], producing the display-ready spectrum. -/// Baseline steps are not supported for 2D and are ignored. No FFT is run. -pub fn reapply_2d(base: &Processed2D, params: &Params2D) -> Processed2D { - reapply_2d_cancellable(base, params, &|| false).expect("non-cancelling 2D reapply") -} - -pub fn reapply_2d_cancellable( - base: &Processed2D, - params: &Params2D, - cancelled: &impl Fn() -> bool, -) -> Option { - match base { - Processed2D::Ft(s) => reapply_ft(s, params, cancelled) - .map(Arc::new) - .map(Processed2D::Ft), - Processed2D::Stack(s) => reapply_stack(s, params, cancelled) - .map(Arc::new) - .map(Processed2D::Stack), - } -} - -// Reduce an axis pipeline's enabled Phase steps to one `(phase0, phase1, pivot)`: -// stored terms sum, and any auto step contributes the phase from `auto`. -fn axis_phase( - pipe: &AxisPipeline, - auto: impl Fn() -> (f64, f64), - default_pivot: f64, -) -> (f64, f64, f64) { - let (mut p0, mut p1, mut pivot) = (0.0, 0.0, default_pivot); - for step in &pipe.steps { - if !step.enabled { - continue; - } - if let StepKind::Phase(p) = &step.kind { - match p.auto { - Some(_) => { - let (a0, a1) = auto(); - p0 += a0; - p1 += a1; - } - None => { - p0 += p.phase0; - p1 += p.phase1; - pivot = p.pivot_frac; - } - } - } - } - (p0, p1, pivot) -} - -fn has_magnitude(pipe: &AxisPipeline) -> bool { - pipe.steps - .iter() - .any(|s| s.enabled && matches!(s.kind, StepKind::Magnitude)) -} - -fn shift_reference(ppm: &mut [f64], pipe: &AxisPipeline) { - let delta: f64 = pipe - .steps - .iter() - .filter(|s| s.enabled) - .filter_map(|s| match &s.kind { - StepKind::Reference(r) => Some(r.target_ppm - r.at_ppm), - _ => None, - }) - .sum(); - if delta != 0.0 { - for p in ppm.iter_mut() { - *p += delta; - } - } -} - -fn reapply_ft( - s: &Spectrum2D, - params: &Params2D, - cancelled: &impl Fn() -> bool, -) -> Option { - if cancelled() { - return None; - } - let (f2_pivot, f1_pivot) = s.peak_pivot_fracs(); - let peak_arg = s - .data - .iter() - .max_by(|a, b| a.norm().total_cmp(&b.norm())) - .map_or(0.0, |c| c.arg()); - let f2 = axis_phase(¶ms.f2, || (peak_arg, 0.0), f2_pivot); - let f1 = axis_phase(¶ms.f1, || (peak_arg, 0.0), f1_pivot); - let mut out = fft2::reapply_phase_2d_cancellable(s, f2, f1, cancelled)?; - if has_magnitude(¶ms.f2) || has_magnitude(¶ms.f1) { - for row in out.data.chunks_mut(out.f2_size.max(1)) { - if cancelled() { - return None; - } - for c in row { - *c = Complex64::new(c.norm(), 0.0); - } - } - } - shift_reference(&mut out.f2_ppm, ¶ms.f2); - shift_reference(&mut out.f1_ppm, ¶ms.f1); - Some(out) -} - -fn reapply_stack( - s: &StackSpectrum, - params: &Params2D, - cancelled: &impl Fn() -> bool, -) -> Option { - if cancelled() { - return None; - } - let pivot = s.peak_pivot_frac(); - let auto = fft2::absorptive_phase(&s.traces).unwrap_or((0.0, 0.0)); - let f2 = axis_phase(¶ms.f2, || auto, pivot); - let mut out = fft2::reapply_phase_stack_cancellable(s, f2, cancelled)?; - if has_magnitude(¶ms.f2) { - for t in &mut out.traces { - if cancelled() { - return None; - } - for c in t { - *c = Complex64::new(c.norm(), 0.0); - } - } - } - shift_reference(&mut out.ppm, ¶ms.f2); - Some(out) -} - /// Index `i` as a `0..=1` fraction of a `size`-point axis (`0.0` if degenerate). fn frac_of(i: usize, size: usize) -> f64 { if size < 2 { diff --git a/crates/processing/src/xps.rs b/crates/processing/src/xps.rs index c3b2c1a4..582c11db 100644 --- a/crates/processing/src/xps.rs +++ b/crates/processing/src/xps.rs @@ -1,5 +1,6 @@ use crate::{NormalizeMethod, SmoothMethod, StepId, StepSource}; -use num_complex::Complex64; +#[path = "xps_signal.rs"] +mod signal; #[derive(Debug, Clone, PartialEq, serde::Serialize, serde::Deserialize)] pub enum XpsStepKind { @@ -81,18 +82,7 @@ pub fn process_region( ) { return Err("XPS normalization divisor must be finite and non-zero"); } - let mut spectrum = crate::Spectrum { - ppm: energy.clone(), - values: values - .iter() - .map(|value| Complex64::new(*value, 0.0)) - .collect(), - hz_per_point: 1.0, - observe_freq_mhz: 1.0, - nucleus: "XPS".into(), - }; - crate::cleanup::normalize(&mut spectrum, method); - values = spectrum.values.into_iter().map(|value| value.re).collect(); + signal::normalize(&energy, &mut values, method); } } if values.iter().any(|value| !value.is_finite()) { @@ -124,7 +114,7 @@ pub fn estimate_charge_shift( return Err("the C 1s reference region is invalid"); } let edge = 3.min(energy_ev.len() / 4); - let smoothed = crate::cleanup::gaussian_smooth_real(intensity, 3.0) + let smoothed = signal::gaussian_smooth_real(intensity, 3.0) .ok_or("the C 1s reference region cannot be smoothed")?; let index = smoothed[edge..smoothed.len() - edge] .iter() @@ -135,19 +125,8 @@ pub fn estimate_charge_shift( Ok(reference_ev - energy_ev[index]) } -fn smooth_values(energy: &[f64], values: &[f64], method: SmoothMethod) -> Vec { - let mut spectrum = crate::Spectrum { - ppm: energy.to_vec(), - values: values - .iter() - .map(|value| Complex64::new(*value, 0.0)) - .collect(), - hz_per_point: 1.0, - observe_freq_mhz: 1.0, - nucleus: "XPS".into(), - }; - crate::cleanup::smooth(&mut spectrum, method); - spectrum.values.into_iter().map(|value| value.re).collect() +fn smooth_values(_energy: &[f64], values: &[f64], method: SmoothMethod) -> Vec { + signal::smooth(values, method) } #[cfg(test)] diff --git a/crates/processing/src/xps_signal.rs b/crates/processing/src/xps_signal.rs new file mode 100644 index 00000000..66757f69 --- /dev/null +++ b/crates/processing/src/xps_signal.rs @@ -0,0 +1,170 @@ +//! Real-valued XPS intensity processing in binding-energy coordinates. +use crate::{NormalizeMethod, SmoothMethod}; + +/// Gaussian smoothing for real-valued detection helpers that need a stable, +/// symmetric kernel but are not persisted processing steps. +pub fn gaussian_smooth_real(values: &[f64], sigma: f64) -> Option> { + if values.is_empty() + || !sigma.is_finite() + || sigma <= 0.0 + || values.iter().any(|value| !value.is_finite()) + { + return None; + } + let radius = (3.0 * sigma).ceil() as isize; + Some( + (0..values.len()) + .map(|index| { + let mut weighted = 0.0; + let mut total = 0.0; + for offset in -radius..=radius { + let source = + (index as isize + offset).clamp(0, values.len() as isize - 1) as usize; + let weight = (-0.5 * (offset as f64 / sigma).powi(2)).exp(); + weighted += values[source] * weight; + total += weight; + } + weighted / total + }) + .collect(), + ) +} + +fn moving_average(values: &mut Vec, window: usize) { + let n = values.len(); + let w = (window.max(3) | 1).min(if n % 2 == 1 { n } else { n.saturating_sub(1) }); + if n < 3 || w < 3 { + return; + } + let h = w / 2; + let mut out = Vec::with_capacity(n); + for i in 0..n { + let lo = i.saturating_sub(h); + let hi = (i + h + 1).min(n); + let sum: f64 = values[lo..hi].iter().sum(); + out.push(sum / (hi - lo) as f64); + } + *values = out; +} + +/// Least-squares polynomial smoothing: each point is replaced by the value of a +/// degree-`order` polynomial fitted over an odd `window` around it. Edge points +/// reuse the boundary window, evaluated off-center, so a polynomial signal of +/// degree ≤ `order` is reproduced exactly everywhere. +fn savitzky_golay(values: &mut Vec, window: usize, order: usize) { + let n = values.len(); + let w = (window.max(3) | 1).min(if n % 2 == 1 { n } else { n.saturating_sub(1) }); + if n < 3 || w < 3 { + return; + } + let m = order.clamp(1, w - 1) + 1; + let h = w / 2; + let x = |i: usize| i as f64 - h as f64; + + let mut gram = vec![vec![0.0; m]; m]; + for i in 0..w { + let mut powers = vec![1.0; m]; + for k in 1..m { + powers[k] = powers[k - 1] * x(i); + } + for r in 0..m { + for c in 0..m { + gram[r][c] += powers[r] * powers[c]; + } + } + } + let mut gram_inv = vec![vec![0.0; m]; m]; + for k in 0..m { + let mut e = vec![0.0; m]; + e[k] = 1.0; + let Some(col) = plotx_analysis::fit::solve_linear(&gram, &e) else { + return; + }; + for r in 0..m { + gram_inv[r][k] = col[r]; + } + } + let sample_powers: Vec> = (0..w) + .map(|i| { + let mut powers = vec![1.0; m]; + for k in 1..m { + powers[k] = powers[k - 1] * x(i); + } + powers + }) + .collect(); + // projection[k][i]: coefficient k of the fitted polynomial from sample i. + let projection: Vec> = (0..m) + .map(|r| { + sample_powers + .iter() + .map(|powers| (0..m).map(|k| gram_inv[r][k] * powers[k]).sum()) + .collect() + }) + .collect(); + // weights[p][i]: smoothing weights when evaluating at offset p in the window. + let weights: Vec> = (0..w) + .map(|p| { + let mut powers = vec![1.0; m]; + for k in 1..m { + powers[k] = powers[k - 1] * x(p); + } + (0..w) + .map(|i| (0..m).map(|k| powers[k] * projection[k][i]).sum()) + .collect() + }) + .collect(); + + let mut out = Vec::with_capacity(n); + for i in 0..n { + let (start, p) = if i < h { + (0, i) + } else if i + h >= n { + (n - w, i - (n - w)) + } else { + (i - h, h) + }; + let mut acc = 0.0; + for (j, &weight) in weights[p].iter().enumerate() { + acc += values[start + j] * weight; + } + out.push(acc); + } + *values = out; +} + +pub fn normalize(axis: &[f64], values: &mut [f64], method: NormalizeMethod) { + let scale = match method { + NormalizeMethod::MaxPeak => values.iter().map(|c| c.abs()).fold(0.0, f64::max), + NormalizeMethod::TotalArea => values.iter().map(|c| c.abs()).sum::() * axis_step(axis), + NormalizeMethod::Constant { divisor } => divisor, + }; + if scale.is_finite() && scale.abs() > f64::MIN_POSITIVE { + for c in values { + *c /= scale; + } + } +} + +pub fn axis_step(ppm: &[f64]) -> f64 { + if ppm.len() < 2 { + return 1.0; + } + let span = (ppm[ppm.len() - 1] - ppm[0]).abs(); + if span > 0.0 { + span / (ppm.len() - 1) as f64 + } else { + 1.0 + } +} + +pub fn smooth(values: &[f64], method: SmoothMethod) -> Vec { + let mut values = values.to_vec(); + match method { + SmoothMethod::MovingAverage { window } => moving_average(&mut values, window as usize), + SmoothMethod::SavitzkyGolay { window, poly_order } => { + savitzky_golay(&mut values, window as usize, poly_order as usize) + } + } + values +} diff --git a/crates/processing/tests/auto_correction_quality.rs b/crates/processing/tests/auto_correction_quality.rs index 2bb9f659..d01d9dbd 100644 --- a/crates/processing/tests/auto_correction_quality.rs +++ b/crates/processing/tests/auto_correction_quality.rs @@ -6,7 +6,59 @@ //! preserved. use num_complex::Complex64; -use plotx_processing::{AutoPhaseMethod, BaselineMethod, Spectrum, auto_phase, baseline, phase}; +use plotx_processing::{ + AutoPhaseMethod, AxisPipeline, BaselineMethod, PhaseParams, ProcessingStep, Spectrum, StepId, + StepKind, StepSource, nmr_bridge, +}; + +fn apply(spectrum: &mut Spectrum, kind: StepKind) { + let axis = nmr::processed::ProcessedAxis::new( + nmr::axis::AxisRole::Signal, + nmr::axis::AxisDomain::Frequency, + Some(nmr::axis::AxisUnit::Ppm), + spectrum.values.len(), + nmr::axis::AxisCoordinates::Explicit(spectrum.ppm.clone()), + nmr::processed::ComponentBasis::Cartesian, + ) + .unwrap(); + let input = std::sync::Arc::new(nmr::Dataset::from_processed( + nmr::processed::ProcessedDataset::from_complex_trace( + axis, + spectrum.values.clone(), + nmr::processed::ProcessedProvenance::new( + nmr::processed::ProcessedOrigin::Unknown, + vec![], + ) + .unwrap(), + ) + .unwrap(), + )); + let pipeline = AxisPipeline { + steps: vec![ProcessingStep::new(StepId::new(1), kind, StepSource::User)], + }; + let output = nmr_bridge::compile( + input, + &pipeline, + 0, + nmr_bridge::DelayPolicy::Disabled, + nmr_bridge::RecipeRange::All, + ) + .unwrap() + .execute( + nmr::processing::ProcessingOptions::new(), + &mut nmr::ExecutionContext::default(), + ) + .unwrap(); + spectrum.values = output + .as_dense_processed() + .unwrap() + .samples() + .as_chunks::<2>() + .0 + .iter() + .map(|v| Complex64::new(v[0], v[1])) + .collect(); +} const PHASE_POINTS: usize = 512; const BASELINE_POINTS: usize = 640; @@ -16,8 +68,9 @@ fn spectrum(values: Vec) -> Spectrum { Spectrum { ppm: (0..n).map(|i| i as f64).collect(), values, - hz_per_point: 1.0, - observe_freq_mhz: 400.0, + unit: nmr::axis::AxisUnit::Ppm, + hz_per_point: Some(1.0), + observe_freq_mhz: Some(400.0), nucleus: "1H".into(), } } @@ -120,8 +173,13 @@ fn assess_phase_quality(corrected: &[Complex64], reference: &[Complex64]) -> Pha fn robust_phase_quality(phase0: f64, phase1: f64, scale: f64) -> PhaseQuality { let reference = ideal_phase_spectrum(scale); let mut observed = spectrum(inject_phase(&reference, phase0, phase1)); - let correction = auto_phase(&observed, AutoPhaseMethod::RobustConsensus); - phase::apply_with_pivot(&mut observed, correction.0, correction.1, correction.2); + apply( + &mut observed, + StepKind::Phase(PhaseParams { + auto: Some(AutoPhaseMethod::RobustConsensus), + ..PhaseParams::MANUAL_ZERO + }), + ); assess_phase_quality(&observed.values, &reference) } @@ -254,7 +312,7 @@ fn asls_quality(shape: BaselineShape, scale: f64) -> BaselineQuality { } let observed = values.clone(); let mut corrected = spectrum(values); - baseline::apply(&mut corrected, BaselineMethod::AUTO); + apply(&mut corrected, StepKind::Baseline(BaselineMethod::AUTO)); // Baseline correction is a real-channel operation. Treating the imaginary // channel as immutable is part of its public signal-preservation contract. @@ -397,7 +455,7 @@ fn asls_non_target_peak_shapes_remain_numerically_safe() { .map(|value| value.re.abs()) .fold(0.0_f64, f64::max); let mut corrected = spectrum(values); - baseline::apply(&mut corrected, BaselineMethod::AUTO); + apply(&mut corrected, StepKind::Baseline(BaselineMethod::AUTO)); assert!(corrected.values.iter().all(|value| value.re.is_finite())); for (value, expected_imaginary) in corrected.values.iter().zip(&imaginary_before) { diff --git a/crates/processing/tests/nmr_axis_evidence.rs b/crates/processing/tests/nmr_axis_evidence.rs new file mode 100644 index 00000000..04d286e9 --- /dev/null +++ b/crates/processing/tests/nmr_axis_evidence.rs @@ -0,0 +1,99 @@ +use nmr::processed::{ + ComponentBasis, ProcessedAxis, ProcessedData, ProcessedDataset, ProcessedDescriptor, + ProcessedOrigin, ProcessedProvenance, +}; +use nmr::processing::{FrequencyFrame, ProcessingOperation as Op, ProcessingPlan, ReferenceSource}; +use nmr::{ + ExecutionContext, + axis::{AxisCoordinates, AxisDomain, AxisRole, AxisUnit, FrequencyEvidence}, +}; +use plotx_io::{nmr_bridge, nmr_view::NmrSource}; +use plotx_processing::{ + arithmetic::{SpectrumBinaryOp, combine_spectra}, + slice::{Reduction, SliceKind, extract}, +}; +use std::sync::Arc; + +fn two_axes(indirect_reference: f64) -> NmrSource { + let axes = [2, 3] + .into_iter() + .map(|points| { + ProcessedAxis::new( + AxisRole::Signal, + AxisDomain::Frequency, + Some(AxisUnit::Hertz), + points, + AxisCoordinates::Uniform { + start: 0.0, + step: 1.0, + }, + ComponentBasis::Cartesian, + ) + .unwrap() + .with_frequency_evidence(Some(FrequencyEvidence::new(Some(500.005), None).unwrap())) + .unwrap() + }) + .collect(); + let descriptor = ProcessedDescriptor::new(axes).unwrap(); + let data = + ProcessedData::from_descriptor(&descriptor, (1..=24).map(f64::from).collect()).unwrap(); + let input = ProcessedDataset::new( + descriptor, + data, + ProcessedProvenance::new(ProcessedOrigin::Unknown, vec![]).unwrap(), + ) + .unwrap(); + let ops = [indirect_reference, 400.0] + .into_iter() + .enumerate() + .map(|(axis, mhz)| Op::ResolveFrequencyFrame { + axis, + frame: FrequencyFrame::Ppm(ReferenceSource::Explicit( + nmr::raw::ChemicalShiftReference::user_constructed(5.0, mhz).unwrap(), + )), + }) + .collect(); + NmrSource::new(Arc::new( + ProcessingPlan::new(ops) + .unwrap() + .apply(&input.into()) + .unwrap(), + )) + .unwrap() +} + +#[test] +fn column_slice_reindexes_reference_and_binary_output_keeps_a_reference_offline() { + let (a, _) = extract(&two_axes(100.0), SliceKind::Column, Reduction::Slice(1)).unwrap(); + let (b, _) = extract(&two_axes(200.0), SliceKind::Column, Reduction::Slice(1)).unwrap(); + assert_eq!(a.reference_frequency_mhz(0), Some(100.0)); + assert_eq!(a.reference_frequency_mhz(1), None); + assert_eq!(b.reference_frequency_mhz(0), Some(200.0)); + let result = combine_spectra(&a, &b, SpectrumBinaryOp::Add, 1.0).unwrap(); + assert_eq!(result.reference_frequency_mhz(0), Some(100.0)); + assert_eq!(result.axes()[0].observe_frequency_mhz(), Some(500.005)); + assert_eq!( + result.axes()[0].coordinate_values().unwrap(), + a.axes()[0].coordinate_values().unwrap() + ); + let mut bytes = Vec::new(); + nmr_bridge::snapshot::write( + result.dataset(), + &mut bytes, + Default::default(), + &mut ExecutionContext::default(), + ) + .unwrap(); + let restored = NmrSource::new( + nmr_bridge::snapshot::read( + &mut bytes.as_slice(), + Default::default(), + &mut ExecutionContext::default(), + ) + .unwrap(), + ) + .unwrap(); + assert_eq!(restored.reference_frequency_mhz(0), Some(100.0)); + assert_eq!(restored.axes()[0].observe_frequency_mhz(), Some(500.005)); + assert_eq!(restored.trace().unwrap(), result.trace().unwrap()); +} diff --git a/crates/processing/tests/nmr_group_delay.rs b/crates/processing/tests/nmr_group_delay.rs new file mode 100644 index 00000000..a966f985 --- /dev/null +++ b/crates/processing/tests/nmr_group_delay.rs @@ -0,0 +1,104 @@ +//! Preserve the signal inputs and tolerances from the former fft.rs tests. +use nmr::{Complex64, ExecutionContext}; +use plotx_io::{Domain, NmrData, nmr_view::NmrSource}; +use plotx_processing::nmr_bridge::{DelayPolicy, RecipeRange}; +use plotx_processing::{AxisPipeline, ProcessingStep, StepId, StepKind, StepSource}; +use std::f64::consts::TAU; + +fn fid() -> NmrData { + NmrData { + points: (0..1024) + .map(|k| { + let t = k as f64 / 4000.0; + Complex64::from_polar((-t / 1.0).exp(), TAU * 800.0 * t) + }) + .collect(), + domain: Domain::Time, + spectral_width_hz: 4000.0, + observe_freq_mhz: 400.0, + carrier_ppm: 0.0, + nucleus: "1H".into(), + source: "original group-delay regression".into(), + group_delay: 0.0, + } +} + +fn corrected(data: NmrData) -> Vec { + let source = NmrSource::try_from(data).unwrap(); + let pipe = AxisPipeline { + steps: vec![ProcessingStep::new( + StepId::new(0), + StepKind::Fft, + StepSource::User, + )], + }; + plotx_processing::nmr_execution::execute_1d( + &source, + &pipe, + DelayPolicy::AxisEvidence, + RecipeRange::Base, + &mut ExecutionContext::default(), + ) + .unwrap() + .source + .trace() + .unwrap() +} + +#[test] +fn group_delay_is_removed_with_the_original_seven_point_shift_and_tolerance() { + let ideal = fid(); + let n = ideal.points.len(); + let d = 7usize; + let mut delayed = ideal.clone(); + delayed.points = (0..n).map(|k| ideal.points[(k + n - d) % n]).collect(); + delayed.group_delay = d as f64; + let a = corrected(ideal); + let b = corrected(delayed); + let max_err = a + .iter() + .zip(&b) + .map(|(x, y)| (x.re - y.re).abs()) + .fold(0.0f64, f64::max); + assert!(max_err < 1e-9, "group delay not removed: max_err={max_err}"); +} + +#[test] +fn fractional_group_delay_uses_the_original_signed_bins_and_tolerance() { + let n = 16usize; + let delay = 3.25; + let negative_start = n.div_ceil(2); + let phase_per_bin = TAU * delay / n as f64; + let delayed: Vec = (0..n) + .map(|m| { + let signed_bin = if m < negative_start { + m as f64 + } else { + m as f64 - n as f64 + }; + Complex64::from_polar(1.0, -phase_per_bin * signed_bin) + }) + .collect(); + // Independent inverse DFT feeds the original spectrum into the public FFT + // bridge. The expected corrected spectrum remains exactly one at every bin. + let mut input = fid(); + input.points = (0..n) + .map(|k| { + delayed + .iter() + .enumerate() + .map(|(m, value)| { + value * Complex64::from_polar(1.0, TAU * (m * k) as f64 / n as f64) + }) + .sum::() + / n as f64 + }) + .collect(); + input.group_delay = delay; + assert!( + corrected(input) + .iter() + .all(|value| (*value - Complex64::new(1.0, 0.0)).norm() < 1e-12), + "fractional delay correction must not introduce a phase jump at DC" + ); +} diff --git a/crates/processing/tests/nmr_nus.rs b/crates/processing/tests/nmr_nus.rs new file mode 100644 index 00000000..459d98ad --- /dev/null +++ b/crates/processing/tests/nmr_nus.rs @@ -0,0 +1,126 @@ +use nmr::axis::{AxisCoordinates, AxisDomain, AxisUnit}; +use nmr::processing::{ + FourierTransform, NusSettings, ProcessingOperation, ProcessingOptions, ProcessingPlan, +}; +use nmr::raw::*; +use nmr::{Complex64, Dataset, ExecutionContext}; +use plotx_processing::{AxisPipeline, ProcessingStep, StepId, StepKind, StepSource, nmr_bridge}; + +/// An independent 4x3 separable tone, with one missing indirect observation. +#[test] +fn sparse_tone_reconstructs_and_round_trips() -> Result<(), Box> { + let indirect = RawAxis::new( + RawAxisKind::Indirect(IndirectComponents::Cartesian( + ComponentEvidence::user_constructed(), + )), + AxisDomain::Time, + Some(AxisUnit::Second), + 4, + AxisCoordinates::Uniform { + start: 0.0, + step: 0.01, + }, + )?; + let direct = RawAxis::new( + RawAxisKind::Direct(DirectSamples::Complex), + AxisDomain::Time, + Some(AxisUnit::Second), + 3, + AxisCoordinates::Uniform { + start: 0.0, + step: 0.001, + }, + )?; + let indices = [3, 0, 1]; + let coordinates: Vec<_> = indices + .iter() + .map(|i| SamplingCoordinate::new(vec![*i])) + .collect(); + let traces = indices + .iter() + .enumerate() + .map(|(ordinal, i)| { + let angle = std::f64::consts::TAU * *i as f64 / 4.0; + let samples = [angle.cos(), angle.sin()] + .into_iter() + .flat_map(|lane| { + (0..3).map(move |j| { + Complex64::from_polar(lane, std::f64::consts::TAU * j as f64 / 3.0) + }) + }) + .collect(); + SparseTrace::new( + ObservationOrdinal::new(ordinal), + coordinates[ordinal].clone(), + samples, + ) + }) + .collect(); + let input: Dataset = RawDatasetBuilder::new(vec![indirect, direct], RawMetadata::default())? + .sparse(traces, SamplingSchedule::new(vec![4], coordinates)?)? + .into(); + let plan = ProcessingPlan::new(vec![ProcessingOperation::FourierTransform { + axis: 1, + transform: FourierTransform::default(), + }])?; + let mut context = ExecutionContext::default(); + let options = ProcessingOptions::new(); + let prepared = NusSettings { + max_iterations: 1000, + noise_standard_deviation: Some(0.0), + } + .prepare(&input, plan, options)?; + if prepared.measured_indices() != indices { + return Err("observation order changed".into()); + } + let mixed = prepared.execute_with_context(&mut context)?; + let data = mixed + .as_dense_processed() + .ok_or("missing reconstructed samples")?; + for row in 0..4 { + let angle = std::f64::consts::TAU * row as f64 / 4.0; + for (lane, expected) in [3.0 * angle.cos(), 3.0 * angle.sin()] + .into_iter() + .enumerate() + { + if (data.get(&[row, 2], &[lane, 0])? - expected).abs() > 1e-5 { + return Err("NUS tone reconstruction exceeded 1e-5 amplitude error".into()); + } + } + } + let pipeline = AxisPipeline { + steps: vec![ProcessingStep::new( + StepId::new(77), + StepKind::Fft, + StepSource::User, + )], + }; + let frequency = nmr_bridge::compile( + std::sync::Arc::new(mixed), + &pipeline, + 0, + nmr_bridge::DelayPolicy::Disabled, + nmr_bridge::RecipeRange::All, + )? + .execute(options, &mut context)?; + let values = frequency.as_dense_processed().ok_or("missing F1 output")?; + if (values.get(&[3, 2], &[0, 0])? - 12.0).abs() > 1e-5 { + return Err("incorrect 2D peak amplitude".into()); + } + let mut bytes = vec![]; + plotx_io::nmr_bridge::snapshot::write( + &frequency, + &mut bytes, + Default::default(), + &mut context, + )?; + let restored = plotx_io::nmr_bridge::snapshot::read( + &mut bytes.as_slice(), + Default::default(), + &mut context, + )?; + if restored.canonical_digests() != frequency.canonical_digests() { + return Err("NUS snapshot identity changed".into()); + } + Ok(()) +} diff --git a/crates/processing/tests/nmr_operations.rs b/crates/processing/tests/nmr_operations.rs new file mode 100644 index 00000000..fc1d5273 --- /dev/null +++ b/crates/processing/tests/nmr_operations.rs @@ -0,0 +1,138 @@ +//! Every supported recipe operation executes through the production bridge. + +use nmr::processed::{ + ComponentBasis, ProcessedAxis, ProcessedDataset, ProcessedOrigin, ProcessedProvenance, +}; +use nmr::{ + Complex64, Dataset, ExecutionContext, + axis::{AxisCoordinates, AxisDomain, AxisRole, AxisUnit}, +}; +use plotx_processing::nmr_bridge::{self, DelayPolicy, RecipeRange}; +use plotx_processing::{ + Apodization, AutoPhaseMethod, AxisPipeline, BaselineMethod, BinParams, NormalizeMethod, + PhaseParams, ProcessingStep, ReferenceParams, SmoothMethod, StepId, StepKind, StepSource, +}; +use std::{error::Error, path::PathBuf, sync::Arc}; + +#[test] +fn supported_recipe_operations() -> Result<(), Box> { + let root = PathBuf::from(env!("CARGO_MANIFEST_DIR")).join("../io/tests/fixtures/nmr"); + let raw = + plotx_io::nmr_bridge::read(&root.join("bruker-1d"), &mut ExecutionContext::default())?; + let axis = ProcessedAxis::new( + AxisRole::Signal, + AxisDomain::Frequency, + Some(AxisUnit::Ppm), + 128, + AxisCoordinates::Uniform { + start: 1.0, + step: 0.1, + }, + ComponentBasis::Cartesian, + )?; + let spectrum = Arc::new(Dataset::from_processed( + ProcessedDataset::from_complex_trace( + axis, + (0..128) + .map(|i| { + let mut value = Complex64::new(0.0, 0.0); + for (center, height) in [(24.0, 1.0), (67.0, 0.7), (105.0, 0.5)] { + let d = (i as f64 - center) / 2.0; + value += Complex64::new(height, height * d) / (1.0 + d * d); + } + value * Complex64::from_polar(1.0, 0.3 + 0.4 * i as f64 / 127.0) + }) + .collect(), + ProcessedProvenance::new(ProcessedOrigin::Unknown, vec![])?, + )?, + )); + let mut steps = vec![ + StepKind::Apodize(Apodization::Gaussian { + lb_hz: 1.0, + gb_hz: 2.0, + }), + StepKind::Baseline(BaselineMethod::Offset), + StepKind::Baseline(BaselineMethod::Polynomial { order: 2 }), + StepKind::Baseline(BaselineMethod::AUTO), + StepKind::Reference(ReferenceParams { + at_ppm: 1.0, + target_ppm: 2.0, + }), + StepKind::Smooth(SmoothMethod::MovingAverage { window: 3 }), + StepKind::Smooth(SmoothMethod::DEFAULT), + StepKind::Normalize(NormalizeMethod::MaxPeak), + StepKind::Normalize(NormalizeMethod::TotalArea), + StepKind::Normalize(NormalizeMethod::Constant { divisor: 2.0 }), + StepKind::Bin(BinParams::DEFAULT), + StepKind::Reverse, + StepKind::Invert, + ]; + steps.extend( + [ + AutoPhaseMethod::RobustConsensus, + AutoPhaseMethod::AbsorptivePeak, + AutoPhaseMethod::Entropy, + AutoPhaseMethod::NegativeMinimization, + AutoPhaseMethod::PeakRegression, + ] + .map(|method| { + StepKind::Phase(PhaseParams { + auto: Some(method), + ..PhaseParams::MANUAL_ZERO + }) + }), + ); + for (index, kind) in steps.into_iter().enumerate() { + let input = if matches!(kind, StepKind::Apodize(_)) { + &raw + } else { + &spectrum + }; + let label = format!("{kind:?}"); + let pipe = AxisPipeline { + steps: vec![ProcessingStep::new( + StepId::new(index as u64), + kind, + StepSource::User, + )], + }; + let result = nmr_bridge::compile( + Arc::clone(input), + &pipe, + 0, + DelayPolicy::Disabled, + RecipeRange::All, + ) + .and_then(|recipe| { + recipe.execute( + nmr::processing::ProcessingOptions::new(), + &mut ExecutionContext::default(), + ) + }); + result.unwrap_or_else(|error| panic!("{label}: {error}")); + } + Ok(()) +} + +#[test] +fn sparse_preparation_preserves_observation_order() -> Result<(), Box> { + let root = PathBuf::from(env!("CARGO_MANIFEST_DIR")).join("../io/tests/fixtures/nmr"); + let sparse = + plotx_io::nmr_bridge::read(&root.join("bruker-nus"), &mut ExecutionContext::default())?; + let plan = nmr::processing::ProcessingPlan::new(vec![ + nmr::processing::ProcessingOperation::ComponentTransform { axis: 0 }, + nmr::processing::ProcessingOperation::FourierTransform { + axis: 1, + transform: nmr::processing::FourierTransform::default(), + }, + ])?; + // This fixture's arbitrary integer samples are not a sparse-spectrum oracle. + // Check preparation here; analytic reconstruction is verified separately. + let prepared = nmr::processing::NusSettings { + max_iterations: 2048, + noise_standard_deviation: Some(0.0), + } + .prepare(&sparse, plan, nmr::processing::ProcessingOptions::new())?; + assert_eq!(prepared.measured_indices(), [3, 1]); + Ok(()) +} diff --git a/crates/processing/tests/nmr_phase_quality.rs b/crates/processing/tests/nmr_phase_quality.rs new file mode 100644 index 00000000..6cdb5716 --- /dev/null +++ b/crates/processing/tests/nmr_phase_quality.rs @@ -0,0 +1,211 @@ +//! Retained scientific phase gates from src/tests.rs, executed through nmr. +use nmr::axis::{AxisCoordinates, AxisDomain, AxisRole, AxisUnit}; +use nmr::processed::{ + ComponentBasis, ProcessedAxis, ProcessedDataset, ProcessedOrigin, ProcessedProvenance, +}; +use nmr::processing::{PhaseMethod as AutoPhaseMethod, ProcessingOptions}; +use nmr::{Complex64, Dataset, ExecutionContext}; +type Result = std::result::Result>; + +/// Ground-truth auto-phase harness: build a known clean (absorptive) spectrum, +/// scramble it by a known `(phase0, phase1)`, and score how well a method's +/// correction recovers the original real part. `residual` is normalized RMS +/// against the clean spectrum, so 0 is a perfect recovery. These guard against +/// selecting a phase method on a p1=0-only benchmark, where any zero-order-only +/// method scores perfectly for the wrong reason. +mod groundtruth { + use super::*; + + pub fn clean(n: usize, peaks: &[(f64, f64, f64)]) -> Vec { + (0..n) + .map(|i| { + let mut c = Complex64::new(0.0, 0.0); + for &(frac_c, h, w) in peaks { + let d = (i as f64 - (frac_c * (n - 1) as f64).round()) / w; + c += Complex64::new(h / (1.0 + d * d), h * d / (1.0 + d * d)); + } + c + }) + .collect() + } + + pub fn scramble(vals: &[Complex64], a0: f64, a1: f64, noise: f64) -> Vec { + let denom = (vals.len() - 1) as f64; + vals.iter() + .enumerate() + .map(|(i, c)| { + let frac = i as f64 / denom; + let mut v = c * Complex64::from_polar(1.0, a0 + a1 * frac); + if noise > 0.0 { + let h = |k: f64| (((k * 12.9898).sin() * 43758.5453).fract() - 0.5) * 2.0; + v += Complex64::new(noise * h(i as f64), noise * h(i as f64 + 7.0)); + } + v + }) + .collect() + } + + fn residual(recovered: &[Complex64], truth: &[Complex64]) -> f64 { + let num: f64 = recovered + .iter() + .zip(truth) + .map(|(r, t)| (r.re - t.re).powi(2)) + .sum(); + let den: f64 = truth.iter().map(|t| t.re * t.re).sum(); + (num / den).sqrt() + } + + /// Measure recovery at the original sample resolution. + pub fn recover_n( + n: usize, + peaks: &[(f64, f64, f64)], + a0: f64, + a1: f64, + noise: f64, + m: AutoPhaseMethod, + ) -> Result<(f64, f64)> { + let truth = clean(n, peaks); + let axis = ProcessedAxis::new( + AxisRole::Signal, + AxisDomain::Frequency, + Some(AxisUnit::Ppm), + n, + AxisCoordinates::Uniform { + start: 0.0, + step: 1.0, + }, + ComponentBasis::Cartesian, + )?; + let input = Dataset::from_processed(ProcessedDataset::from_complex_trace( + axis, + scramble(&truth, a0, a1, noise), + ProcessedProvenance::new(ProcessedOrigin::Unknown, vec![])?, + )?); + let mut context = ExecutionContext::default(); + let options = ProcessingOptions::new(); + let estimate = m + .prepare(&input, 0, options)? + .estimate_with_context(&mut context)?; + let p1 = -estimate.correction().p1_degrees() * (n - 1) as f64 / n as f64; + let output = estimate.apply_with_context(&input, options, &mut context)?; + let data = output + .as_dense_processed() + .ok_or("missing processed result")?; + let values = (0..n) + .map(|i| Ok(Complex64::new(data.get(&[i], &[0])?, data.get(&[i], &[1])?))) + .collect::>>()?; + Ok((residual(&values, &truth), p1)) + } + + pub fn one() -> Vec<(f64, f64, f64)> { + vec![(0.5, 1.0, 4.0)] + } + pub fn many() -> Vec<(f64, f64, f64)> { + vec![ + (0.15, 1.0, 4.0), + (0.4, 0.7, 4.0), + (0.62, 0.9, 4.0), + (0.86, 0.5, 4.0), + ] + } +} + +#[test] +fn retained_phase_quality_gates() { + use groundtruth::*; + let cases = [ + ( + "AbsorptivePeak-zero", + 1024, + many(), + 0.3, + 0.0, + 0.0, + AutoPhaseMethod::AbsorptivePeak, + 0.05, + ), + ( + "Entropy-first-order", + 1024, + many(), + 0.3, + 3.0, + 0.0, + AutoPhaseMethod::Entropy, + 0.15, + ), + ( + "Entropy-negative-ramp", + 1024, + many(), + -0.5, + -4.5, + 0.0, + AutoPhaseMethod::Entropy, + 0.2, + ), + ( + "Entropy-single", + 1024, + one(), + 0.9, + 0.0, + 0.0, + AutoPhaseMethod::Entropy, + 0.1, + ), + ( + "Entropy-large-narrow", + 32768, + vec![ + (0.15, 1.0, 2.0), + (0.4, 0.7, 2.0), + (0.62, 0.9, 2.0), + (0.86, 0.5, 2.0), + ], + 0.3, + 220f64.to_radians(), + 0.0, + AutoPhaseMethod::Entropy, + 0.2, + ), + ( + "Entropy-90deg", + 1024, + many(), + 2.0, + 90f64.to_radians(), + 0.01, + AutoPhaseMethod::Entropy, + 0.35, + ), + ( + "Entropy-270deg", + 1024, + many(), + 2.0, + 270f64.to_radians(), + 0.01, + AutoPhaseMethod::Entropy, + 0.35, + ), + ( + "Entropy-500deg", + 1024, + many(), + 2.0, + 500f64.to_radians(), + 0.01, + AutoPhaseMethod::Entropy, + 0.35, + ), + ]; + for (label, n, peaks, p0, p1, noise, method, limit) in cases { + let (residual, estimated_p1) = recover_n(n, &peaks, p0, p1, noise, method) + .unwrap_or_else(|error| panic!("{label}: {error}")); + assert!( + residual < limit, + "{label}: residual={residual}, required <{limit}; endpoint p1={estimated_p1}deg" + ); + } +} diff --git a/crates/processing/tests/nmr_prepare_control.rs b/crates/processing/tests/nmr_prepare_control.rs new file mode 100644 index 00000000..cfed9e34 --- /dev/null +++ b/crates/processing/tests/nmr_prepare_control.rs @@ -0,0 +1,107 @@ +//! Host cancellation must reach the production NUS preparation path. +use nmr::axis::{AxisCoordinates, AxisDomain, AxisUnit}; +use nmr::execution::ExecutionStage; +use nmr::raw::*; +use nmr::{CancellationToken, Complex64, ExecutionContext}; +use plotx_io::nmr_view::NmrSource; +use plotx_processing::nmr_bridge::{DelayPolicy, RecipeRange}; +use plotx_processing::nmr_execution::{NusRequest, execute_2d}; +use plotx_processing::{ + AxisPipeline, Layout2D, Params2D, ProcessingStep, StepId, StepKind, StepSource, +}; +use std::sync::Arc; + +fn input() -> NmrSource { + let axis = |kind, points| { + RawAxis::new( + kind, + AxisDomain::Time, + Some(AxisUnit::Second), + points, + AxisCoordinates::Uniform { + start: 0.0, + step: 0.001, + }, + ) + .unwrap() + }; + let axes = vec![ + axis( + RawAxisKind::Indirect(IndirectComponents::Cartesian( + ComponentEvidence::user_constructed(), + )), + 4, + ), + axis(RawAxisKind::Direct(DirectSamples::Complex), 3), + ]; + let coordinates: Vec<_> = [3, 0, 1] + .into_iter() + .map(|i| SamplingCoordinate::new(vec![i])) + .collect(); + let traces = coordinates + .iter() + .enumerate() + .map(|(ordinal, coordinate)| { + SparseTrace::new( + ObservationOrdinal::new(ordinal), + coordinate.clone(), + vec![Complex64::new(1.0, 0.0); 6], + ) + }) + .collect(); + let raw = RawDatasetBuilder::new(axes, RawMetadata::default()) + .unwrap() + .sparse(traces, SamplingSchedule::new(vec![4], coordinates).unwrap()) + .unwrap(); + NmrSource::new(Arc::new(raw.into())).unwrap() +} + +#[test] +fn production_nus_can_cancel_before_and_during_preparation() { + let input = input(); + let params = Params2D { + layout: Layout2D::Ft, + f2: AxisPipeline { + steps: vec![ProcessingStep::new( + StepId::new(1), + StepKind::Fft, + StepSource::User, + )], + }, + f1: AxisPipeline { steps: vec![] }, + }; + for pre_cancelled in [true, false] { + let token = CancellationToken::new(); + let cancel = token.clone(); + if pre_cancelled { + token.cancel(); + } + let mut saw_preparation = false; + let mut progress = |event: nmr::execution::ProgressEvent| { + if event.stage == ExecutionStage::Preflight && event.completed > 0 { + saw_preparation = true; + cancel.cancel(); + } + }; + let mut context = ExecutionContext::default() + .with_cancellation(token) + .with_progress(&mut progress); + let error = execute_2d( + &input, + ¶ms, + DelayPolicy::Disabled, + RecipeRange::Base, + Some(NusRequest { + max_iterations: 1000, + noise_standard_deviation: Some(0.0), + }), + &mut context, + ) + .unwrap_err(); + assert!(error.is_cancelled(), "{error}"); + assert_eq!(context.ledger().used(), 0); + if !pre_cancelled { + assert!(saw_preparation); + } + } +} diff --git a/crates/processing/tests/nmr_shared_complex.rs b/crates/processing/tests/nmr_shared_complex.rs new file mode 100644 index 00000000..06e7e731 --- /dev/null +++ b/crates/processing/tests/nmr_shared_complex.rs @@ -0,0 +1,322 @@ +use nmr::axis::{AxisCoordinates, AxisDomain, AxisUnit}; +use nmr::raw::{ + ComponentEvidence, DirectSamples, IndirectComponents, RawAxis, RawAxisKind, RawDatasetBuilder, + RawMetadata, +}; +use nmr::{Complex64, ExecutionContext}; +use plotx_io::{nmr_bridge::snapshot, nmr_view::NmrSource}; +use plotx_processing::nmr_bridge::{DelayPolicy, RecipeRange}; +use plotx_processing::{ + AutoPhaseMethod, AxisPipeline, Layout2D, Params2D, PhaseParams, Processed2D, ProcessingStep, + StepId, StepKind, StepSource, nmr_execution, +}; +use std::{f64::consts::TAU, sync::Arc}; + +fn shared_tone() -> NmrSource { + let axis = |kind, points| { + RawAxis::new( + kind, + AxisDomain::Time, + Some(AxisUnit::Second), + points, + AxisCoordinates::Uniform { + start: 0.0, + step: 1.0 / 32.0, + }, + ) + .unwrap() + }; + let input = RawDatasetBuilder::new( + vec![ + axis( + RawAxisKind::Indirect(IndirectComponents::SharedComplex { + conjugated: true, + evidence: ComponentEvidence::user_constructed(), + }), + 8, + ), + axis(RawAxisKind::Direct(DirectSamples::Complex), 16), + ], + RawMetadata::default(), + ) + .unwrap() + .dense( + (0..8) + .flat_map(|row| { + (0..16).map(move |col| { + Complex64::from_polar(1.0, TAU * (-row as f64 / 8.0 + 3.0 * col as f64 / 16.0)) + }) + }) + .collect::>(), + ) + .unwrap(); + NmrSource::new(Arc::new(input.into())).unwrap() +} + +#[test] +fn shared_auto_phase_selects_the_pair_when_the_imaginary_field_is_strongest() { + let input = nmr_execution::execute_2d( + &shared_tone(), + &Params2D { + layout: Layout2D::Ft, + f2: pipeline(0, 1.0), + f1: pipeline(2, 0.0), + }, + DelayPolicy::Disabled, + RecipeRange::All, + None, + &mut ExecutionContext::default(), + ) + .unwrap(); + for axis in 0..2 { + let pipe = AxisPipeline { + steps: vec![ProcessingStep::new( + StepId::new(9), + StepKind::Phase(PhaseParams { + auto: Some(AutoPhaseMethod::AbsorptivePeak), + ..PhaseParams::MANUAL_ZERO + }), + StepSource::User, + )], + }; + let output = plotx_processing::nmr_bridge::compile( + input.source.dataset().clone(), + &pipe, + axis, + DelayPolicy::Disabled, + RecipeRange::Frequency, + ) + .unwrap() + .execute_with_report(Default::default(), &mut ExecutionContext::default()) + .unwrap(); + let report = &output.phases[0]; + assert_eq!(report.representative.as_ref().unwrap().component, 0); + let processed = output.dataset.as_processed().unwrap(); + let re = processed.data().get(&[5, 11], &[0, 0]).unwrap(); + let im = processed.data().get(&[5, 11], &[0, 1]).unwrap(); + assert!((re - 128.0).abs() < 1e-10); + assert!(im.abs() < 1e-10); + let (phase0, phase1, pivot_frac) = report.recipe_parameters(); + let mut manual = pipe.clone(); + manual.steps[0].kind = StepKind::Phase(PhaseParams { + phase0, + phase1, + pivot_frac, + auto: None, + }); + let replayed = plotx_processing::nmr_bridge::compile( + input.source.dataset().clone(), + &manual, + axis, + DelayPolicy::Disabled, + RecipeRange::Frequency, + ) + .unwrap() + .execute(Default::default(), &mut ExecutionContext::default()) + .unwrap(); + for (actual, expected) in replayed + .as_processed() + .unwrap() + .data() + .samples() + .iter() + .zip(processed.data().samples()) + { + assert!((actual - expected).abs() < 1e-10); + } + } +} + +fn pipeline(start: u64, phase: f64) -> AxisPipeline { + AxisPipeline { + steps: vec![ + ProcessingStep::new(StepId::new(start), StepKind::Fft, StepSource::User), + ProcessingStep::new( + StepId::new(start + 1), + StepKind::Phase(PhaseParams { + phase0: phase, + ..PhaseParams::MANUAL_ZERO + }), + StepSource::User, + ), + ], + } +} + +#[test] +fn shared_pair_keeps_phase_capability_signed_peak_and_magnitude_through_snapshot() { + let input = shared_tone(); + assert!(input.has_imaginary(0)); + assert!(input.has_imaginary(1)); + assert!(!input.has_imaginary(2)); + let output = nmr_execution::execute_2d( + &input, + &Params2D { + layout: Layout2D::Ft, + f2: pipeline(0, 0.2), + f1: pipeline(2, 0.5), + }, + DelayPolicy::Disabled, + RecipeRange::All, + None, + &mut ExecutionContext::default(), + ) + .unwrap(); + assert!(output.source.has_imaginary(0)); + assert!(output.source.has_imaginary(1)); + assert_eq!( + output + .source + .dataset() + .as_processed() + .unwrap() + .descriptor() + .component_counts(), + [1, 2] + ); + let Processed2D::Ft(view) = &output.view else { + panic!("expected frequency plane"); + }; + assert_eq!(view.f1_ppm[5], 4.0); + assert_eq!(view.f2_ppm[11], 6.0); + // F1's imaginary orientation reverses its phase rotation on the stored pair. + let expected = Complex64::from_polar(128.0, 0.5 - 0.2); + assert!((view.data[5 * 16 + 11] - expected).norm() < 1e-10); + assert!(view.data[3 * 16 + 11].norm() < 1e-10); + let magnitude = view.magnitude_plane.as_ref().unwrap(); + assert!((magnitude[5 * 16 + 11] - 128.0).abs() < 1e-10); + assert!(magnitude[3 * 16 + 11] < 1e-10); + + let mut bytes = Vec::new(); + snapshot::write( + output.source.dataset(), + &mut bytes, + Default::default(), + &mut ExecutionContext::default(), + ) + .unwrap(); + let restored = NmrSource::new( + snapshot::read( + &mut bytes.as_slice(), + Default::default(), + &mut ExecutionContext::default(), + ) + .unwrap(), + ) + .unwrap(); + assert!(restored.has_imaginary(0)); + assert!(restored.has_imaginary(1)); + let Processed2D::Ft(restored_view) = + nmr_execution::view_2d(&restored, Layout2D::Ft, &mut ExecutionContext::default()).unwrap() + else { + panic!("expected restored frequency plane"); + }; + assert_eq!(restored_view.data, view.data); + assert_eq!(restored_view.magnitude_plane, view.magnitude_plane); + assert_eq!(restored_view.f1_ppm, view.f1_ppm); + assert_eq!(restored_view.f2_ppm, view.f2_ppm); +} + +#[test] +fn shared_reference_and_slices_retain_complex_values_and_axis_coordinates() { + use nmr::processing::{ + FrequencyFrame, PhaseMethod, ProcessingOperation as Op, ProcessingPlan, ReferenceSource, + }; + use plotx_processing::{ReferenceParams, SliceKind, slice::Reduction}; + + let output = nmr_execution::execute_2d( + &shared_tone(), + &Params2D { + layout: Layout2D::Ft, + f2: pipeline(0, 0.2), + f1: pipeline(2, 0.5), + }, + DelayPolicy::Disabled, + RecipeRange::All, + None, + &mut ExecutionContext::default(), + ) + .unwrap(); + let input = ProcessingPlan::new( + [100.0, 400.0] + .into_iter() + .enumerate() + .map(|(axis, mhz)| Op::ResolveFrequencyFrame { + axis, + frame: FrequencyFrame::Ppm(ReferenceSource::Explicit( + nmr::raw::ChemicalShiftReference::user_constructed(4.0, mhz).unwrap(), + )), + }) + .collect(), + ) + .unwrap() + .apply(output.source.dataset()) + .unwrap(); + let input = NmrSource::new(Arc::new(input)).unwrap(); + let original = input.dataset().canonical_digests(); + for (axis, kind, index) in [(0, SliceKind::Column, 11), (1, SliceKind::Row, 5)] { + let pipe = AxisPipeline { + steps: vec![ProcessingStep::new( + StepId::new(4), + StepKind::Reference(ReferenceParams { + at_ppm: 1.0, + target_ppm: 1.1, + }), + StepSource::User, + )], + }; + let shifted = plotx_processing::nmr_bridge::compile( + input.dataset().clone(), + &pipe, + axis, + DelayPolicy::Disabled, + RecipeRange::Frequency, + ) + .unwrap() + .execute(Default::default(), &mut ExecutionContext::default()) + .unwrap(); + let shifted = NmrSource::new(shifted).unwrap(); + assert_eq!( + shifted.dataset().as_processed().unwrap().data().samples(), + input.dataset().as_processed().unwrap().data().samples() + ); + for a in 0..2 { + let delta = if a == axis { 0.1 } else { 0.0 }; + for (before, after) in input.axes()[a] + .coordinate_values() + .unwrap() + .iter() + .zip(shifted.axes()[a].coordinate_values().unwrap()) + { + assert!((after - before - delta).abs() < 1e-12); + } + } + let (slice_source, slice) = + plotx_processing::slice::extract(&shifted, kind, Reduction::Slice(index)).unwrap(); + assert!(slice_source.has_imaginary(0)); + assert_eq!( + slice.coordinates, + shifted.axes()[axis].coordinate_values().unwrap() + ); + assert_eq!(slice.reference_freq_mhz, Some([100.0, 400.0][axis])); + let data = shifted.dataset().as_processed().unwrap().data(); + for (point, actual) in slice.values.iter().enumerate() { + let coordinates = if axis == 0 { + [point, index] + } else { + [index, point] + }; + let expected = Complex64::new( + data.get(&coordinates, &[0, 0]).unwrap(), + data.get(&coordinates, &[0, 1]).unwrap(), + ); + assert_eq!(*actual, if axis == 0 { expected.conj() } else { expected }); + } + PhaseMethod::AbsorptivePeak + .prepare(slice_source.dataset(), 0, Default::default()) + .unwrap() + .estimate() + .unwrap(); + assert_eq!(input.dataset().canonical_digests(), original); + } +} diff --git a/docs/src/content/docs/guides/importing-data.md b/docs/src/content/docs/guides/importing-data.md index e6881c0b..29b8e400 100644 --- a/docs/src/content/docs/guides/importing-data.md +++ b/docs/src/content/docs/guides/importing-data.md @@ -10,9 +10,9 @@ no conversion step is needed. | Format | Extension | Notes | | --- | --- | --- | -| JEOL Delta | `.jdf` | 1D, 2D, and pseudo-2D (DOSY / T1 / T2) | -| Bruker TopSpin | `fid` / `ser` directories | 1D and 2D | -| Varian/Agilent VnmrJ | `.fid` directory | Raw time-domain 1D and conventional 2D | +| JEOL Delta | `.jdf` | Raw and processed 1D/2D and parameter series; experimental support | +| Bruker TopSpin | `fid` / `ser` / `pdata` | Raw and processed 1D/2D; NUS support is experimental | +| Varian/Agilent VnmrJ | `.fid` directory | Raw 1D, 2D with `phase=[1,2]`, or a series varying one parameter | | Waters MassLynx RAW | `.raw` directory | Validated low-resolution runs, including SQD2 data | | SCIEX legacy WIFF | `.wiff` + `.wiff.scan` | Single- and multi-sample legacy runs; both files must remain together | | Rigaku powder XRD | `.rasx`, FI `.raw`, RAS_RAW `.txt` | Diffraction pattern, acquisition metadata, and attenuation when available | @@ -46,6 +46,45 @@ CasaXPS `.txt` files are recognized from their structured header, not from the extension alone. Other `.txt` files continue through table import. See the [XPS workflow](/guides/xps/) for energy-axis and fitting details. +## Follow import progress + +Scientific data files and acquisition folders load one dataset at a time in the +background, so you can continue working. Completed datasets appear on the board +without changing your current page or selection. The status bar shows the current +file and success/failure counts; review failures in the diagnostic history. +Additional imports wait for the current batch. Opening, closing, or creating a +project cancels unfinished imports for the previous project. + +NMR import includes default processing. NUS reconstruction can take substantially +longer than reading the file. Project files, table previews, ZIP files, and imports +with a manually supplied sampling table use separate import workflows. + +## Supplying a missing NMR sampling table + +Use this option for supported 2D Bruker NUS or JEOL acquisitions that sampled only +part of the indirect grid. You need the original sampling table and acquisition +files with enough grid and calibration information to check it. + +1. Choose **File → Import NMR with Sampling Table…** (also available in the + command palette), then select the Bruker `ser` or JEOL `.jdf` file. +2. Enter the table's source or an explanation, and the full original indirect + grid size, including unsampled points. +3. Enter **Lanes per observation**: the number of component records acquired at + each listed indirect point. Use the acquisition's value, not the number of + points in the table. +4. Select **Zero-based** or **One-based** to match the original table. Enter one + indirect index per line, in acquisition order. Each line represents all lanes + for that observation; keep repeated observations. +5. Click **Validate and import**. If validation fails, check the reported mismatch + against the acquisition records. Conflicts with an existing sampling list, + incorrect observation or lane counts, and missing grid or calibration information + prevent import. + +Vendor files are not modified. Save the project to retain the table and its source; +reopening the project does not require the original files. Repeated coordinates +can be imported but currently prevent [NUS reconstruction](/guides/processing/#reconstruct-a-non-uniformly-sampled-spectrum). +For scripts, see the [CLI declaration format](/reference/cli/#sampling-declarations). + ## Varian/Agilent VnmrJ To import a raw 1D or conventional 2D acquisition, choose **Open Folder…** and @@ -53,9 +92,11 @@ select its `.fid` directory. You can instead choose **Open File…** and select the `fid` file inside. Keep the `fid` and `procpar` files together in the same directory. -Processed spectra, 3D or 4D experiments, imaging, pseudo-2D experiments, -non-uniform sampling, and other arrayed experiments are not supported. See -[File formats](/reference/file-formats/) for compatibility details. +You can also import a series varying one ungrouped parameter. Supported 2D data +requires `phase=[1,2]`. Grouped or multiple parameter arrays, other phase orders, +Varian NUS, processed spectra, and more than two dimensions are unsupported. +See [File formats](/reference/file-formats/#nmr-data-and-projects) for calibration +requirements and format limitations. ## mzML diff --git a/docs/src/content/docs/guides/processing.md b/docs/src/content/docs/guides/processing.md index f2f8d882..ef6d35a0 100644 --- a/docs/src/content/docs/guides/processing.md +++ b/docs/src/content/docs/guides/processing.md @@ -18,7 +18,7 @@ charge correction is shared by all regions at one measurement position. See ## A typical 1D spectrum -A newly imported time-domain 1D dataset already carries the standard pipeline — +A time-domain 1D dataset with known digital-filter delay carries the standard pipeline — apodization, zero filling, FFT, phase correction, and baseline correction, in that order — with automatic phasing enabled. In most cases the spectrum on screen is immediately usable, and a session touches at most three things: @@ -40,6 +40,72 @@ order they are processed. A dataset that arrives already transformed is marked **Imported spectrum** and has no time-domain steps and no FFT: PlotX does not invent an FID for data it never acquired. +**Reset to default** restores the import settings. Raw data with unknown +filter delay remains uncorrected; imported spectra have no FFT, and spectra +containing only real values have no phase-correction step. + +## Check calibration before analysis + +Check the axis units before choosing an analysis: FIDs use seconds, while spectra +use Hz or ppm. Missing calibration is not treated as zero. + +- **Reference** requires a ppm axis and shifts coordinates without changing intensities. +- **CRAFT** requires a complex FID with known spectral width, observe frequency, + chemical-shift reference, and digital-filter delay. +- **DOSY maps** require a frequency-domain series calibrated in ppm. +- **Multiplet analysis** requires a ppm spectrum and a known chemical-shift reference + frequency to report coupling constants in Hz. + +The observe frequency and chemical-shift reference frequency serve different +purposes; do not substitute one for the other when interpreting ppm-to-Hz +conversions. For imported Bruker processed spectra, `SF` provides the reference +for converting ppm intervals to Hz. It does not supply missing acquisition +frequency or carrier information, or confirm that digital-filter correction was applied. + +## Reconstruct a non-uniformly sampled spectrum + +Non-uniform sampling (NUS) records only selected points along the indirect time +axis. PlotX uses the sampling table to reconstruct the full grid before the F1 +FFT produces a 2D spectrum. The grid size need not be a power of two. + +1. Import a supported Bruker NUS or JEOL acquisition with its sampling table. + If the table is missing, restore the vendor companion files or use + [Import NMR with Sampling Table](/guides/importing-data/#supplying-a-missing-nmr-sampling-table). +2. Allow the default processing to finish. PlotX estimates noise automatically; + you do not need to select a noise region or enter a noise value. +3. Check the status bar for errors before interpreting the result. If reconstruction + fails during import, the original observations remain available, but their + display is **not a reconstructed spectrum**. If a later processing change + fails, the last successful display remains visible. + +### Noise and convergence settings + +The automatic noise estimate uses the spectrum after the current F2 processing +steps and updates when those steps change. It requires at least 48 observations +and 32 F2 frequency points, or 32 observations and 128 points for an estimate +checked against held-out observations. Meeting these sizes alone does not +ensure a reliable estimate; additional noise-quality checks must also pass. + +If you have an independently determined noise standard deviation, enable +**Override automatic noise estimate** under **Non-uniform sampling**. Enter it +in the amplitude units of the spectrum after F2 processing. A value of 0 means +noiseless input, not automatic estimation. Update the value if you change F2 +processing in a way that changes the noise scale. + +Set the maximum iteration count between 1 and 2048. Reaching this limit without +convergence reports an error. Keep the F2 and F1 FFT steps enabled to obtain +both frequency axes. Noise settings are saved in the project but are not +transferred to other acquisitions by reusable processing recipes. + +### Input limitations + +Reconstruction requires a sampling table without repeated coordinates and +enough real and imaginary signal information to reconstruct the indirect axis. +Not all acquisition arrangements are supported; an unsupported arrangement +reports an error. Repeated observations can +be imported and saved, but cannot currently be reconstructed; do not delete +repeats from the sampling table to bypass this restriction. + ## Where processing lives Processing opens as a card at the upper right of the canvas — from the @@ -122,6 +188,13 @@ Processing card's ⋮ menu, applied before the pipeline. It governs 1D and 2D data alike: switch it off on a 2D dataset and the direct dimension is left uncorrected too. +For Bruker data, a nonnegative `GRPDLY` supplies the delay, including zero. +If it is missing or -1, PlotX uses supported `DSPFVS`/`DECIM` settings to determine +the delay. Otherwise the delay remains unknown and raw data initially appears +as an FID. To view an uncorrected spectrum, turn off **Group-delay correction** +under **Advanced** and enable FFT. Review the result for filter-related distortion; +disabling correction does not establish the delay required by CRAFT. + ## Apodization Click the **Apodize** step to open its settings. All of them are shown at once, @@ -190,6 +263,15 @@ automatic method each row says which switch to flip first. Through [Automation](/guides/automation/) these values keep their own units: phase angles in radians, the pivot as a fraction. +For 2D data, the selected automatic method estimates a correction from the trace +containing the strongest real or imaginary signal and applies the same correction +across the series, preserving relative row signs. If it fails, review the error +and choose another method or adjust the phase manually. + +For JEOL COSY, both F2 and F1 support phase correction and Reference. An extracted +row follows F2; a column follows F1. Both retain real and imaginary values for +further phase correction. + ## Baseline correction Baseline correction is off by default. Enable the step when your spectrum diff --git a/docs/src/content/docs/reference/cli.md b/docs/src/content/docs/reference/cli.md index 978547fc..302816f0 100644 --- a/docs/src/content/docs/reference/cli.md +++ b/docs/src/content/docs/reference/cli.md @@ -18,9 +18,9 @@ use the in-app Automation window, which runs the same workflows. ## Inspect and process data ```sh -plotx-cli inspect [--json] +plotx-cli inspect [--json] [--sampling-declaration ] plotx-cli craft --output [--region ]... [--expected-ratio ]... -plotx-cli process --scheme --output [--format svg|pdf|png|tiff|jpeg] +plotx-cli process --scheme --output [--format svg|pdf|png|tiff|jpeg] [--sampling-declaration ] ``` `inspect` detects, loads, and describes one supported dataset; `--json` emits a @@ -30,6 +30,17 @@ protocol name. For XPS it reports measurement, region and point counts, the region names, and how many regions have a binding-energy axis or remain kinetic-only. +For NMR, `inspect` describes the source data without processing it. Bruker +experiment directories prefer raw data; select a processed file explicitly to +inspect that spectrum. If a directory is ambiguous, specify a file or processing +directory. For NUS data, the reported shape includes unsampled grid points; it +is not the number of acquired observations. A combination of time, frequency, +or parameter axes is reported as `domain: "mixed"`. + +Check reported warnings before processing. Missing calibration or digital-filter +delay remains unknown; see [NMR format limitations](/reference/file-formats/#nmr-data-and-projects). +Processing failures identify the failing step and return a nonzero exit code. + `process` is the convenience path for a single import, one [processing recipe](/guides/templates/), and one figure export. When `--format` is omitted, the format is inferred from the output file's @@ -45,6 +56,14 @@ report containing each input's fitted components, per-region coherent amplitudes an amplitude ratio when exactly two regions are supplied, diagnostics, and quality checks. Wide selections are still reported under the regions you gave. +CRAFT requires known spectral width, observe frequency, chemical-shift reference, +and digital-filter delay. Missing information produces a failed entry in the report. +The ppm reference frequency is retained separately from the observe frequency; +the report's `chemical_shift_reference.reference_frequency_mhz` defines Hz-to-ppm +conversion. FFT cross-check magnitudes depend on the modeling interval, +exponential window, and zero filling. Use them to assess the fit; use the reported +coherent amplitudes for region amplitude ratios. + When exactly two regions are supplied, repeat `--expected-ratio` once per input to compare the measured ratio with a reference value. The report records the relative error and passes the check when it is within 5%. `all_succeeded` tells @@ -54,6 +73,55 @@ selected region, or reaches a diagnostic limit. Treat a false quality result as an indication that the data needs scientific review, even when the command completed. +## Sampling declarations + +`inspect` and `process` accept `--sampling-declaration sampling.json` for a +supported 2D Bruker NUS or JEOL acquisition with only part of the indirect grid sampled. The JSON file is limited +to 8 MiB. Supply every field explicitly; for example: + +```json +{ + "assertion_id": "my-sampling-table-1", + "source": "user-provided sampling table from experiment notes", + "grid_shape": [4], + "coordinates": [[4], [2]], + "index_base": "one", + "component_counts": [2] +} +``` + +Set the fields from the original acquisition records: + +| Field | Value to supply | +| --- | --- | +| `assertion_id` | An identifier for this sampling declaration. | +| `source` | Where the table came from, such as an acquisition log. | +| `grid_shape` | The full indirect grid size, including unsampled points, as a one-element array. | +| `coordinates` | One indirect index per observation, each in its own array, in acquisition order. | +| `index_base` | `"zero"` for indices starting at 0, or `"one"` for indices starting at 1. | +| `component_counts` | The number of component records (lanes) per observation, as a one-element array. | + +The example describes a four-point grid with two lanes per observation, sampled +at one-based indices 4 then 2. Each coordinate row represents all lanes for that +observation. Preserve acquisition order and repeated observations. Repeats can +be imported, but currently prevent NUS reconstruction. + +PlotX checks the declaration against the acquisition. Missing grid or calibration +information, incorrect lane or observation counts, and conflicts with existing +sampling lists cause an error. This option does not support Varian data, +processed spectra, or acquisitions outside the supported 2D layouts. + +For example, save the declaration as `sampling.json`, then inspect the acquisition: + +```sh +plotx-cli inspect experiment/ser --sampling-declaration sampling.json --json +``` + +The workflow tool `data.import` accepts the same JSON object in its optional +`sampling_declaration` parameter. The declaration must be valid for every path +in that import node; use separate nodes for different tables. Saving the project +retains the declaration without changing the vendor files. + ## Run a workflow ```sh diff --git a/docs/src/content/docs/reference/file-formats.md b/docs/src/content/docs/reference/file-formats.md index 81142a20..1cfe88e8 100644 --- a/docs/src/content/docs/reference/file-formats.md +++ b/docs/src/content/docs/reference/file-formats.md @@ -59,19 +59,46 @@ A `.plotxproc` file stores one processing pipeline, without any data — save a recipe once and apply it to a whole series of similar experiments, on any machine. See [Recipes and templates](/guides/templates/). +## NMR data and projects + +PlotX supports 1D and 2D NMR, including series with a parameter axis. +Choose the input that matches the data you want to work with: + +- **Bruker:** select an experiment directory for raw data, or a spectrum inside + `pdata` for processed data. If a directory contains ambiguous candidates, + select a specific file or processing directory. States 2D (`FnMODE=4`) is supported. + Processed spectra retain the real and imaginary components present in the files. +- **JEOL:** select a `.jdf` file containing a supported raw or processed acquisition. +- **JCAMP-DX:** ordinary XYDATA spectra with one intensity per coordinate, + version 5.00 or 5.01, are supported with Hz or ppm axes. LINK, NTUPLES, and + peak tables are not supported. + +Missing nucleus, frequency-reference, or digital-filter-delay information remains +unknown. Hz spectra can be displayed directly, but some analyses need additional +calibration. Raw data with unknown filter delay initially appears as an FID; +see [Group-delay correction](/guides/processing/#group-delay-correction) to view +an uncorrected spectrum, and [calibration requirements](/guides/processing/#check-calibration-before-analysis) +before choosing an analysis. Unknown coordinates or unsupported data arrangements +produce an import error or diagnostic. + +JEOL and Bruker NUS support is experimental and does not cover every instrument +or acquisition setting. Check import diagnostics and compare results with a trusted +reference before relying on an unfamiliar acquisition type. + +Saving a `.plotx` project retains the imported NMR components, sampling order, +source information, warnings, and processing settings. Reopening recomputes spectra +from the saved settings without requiring the original vendor files. + ## Varian/Agilent VnmrJ raw NMR -PlotX imports raw time-domain 1D and conventional 2D acquisitions. Select the -`.fid` directory or the `fid` file inside it; the `fid` and `procpar` files must -both be present in that directory. A `.fid` directory name by itself is not -enough to identify a dataset. - -The importer accepts the common 16-bit integer, 32-bit integer, and 32-bit -floating-point sample formats, including conventional States 2D data. -Processed spectra, 3D or 4D experiments, imaging, pseudo-2D experiments, -non-uniform sampling, and arrayed parameters other than phase are not -supported. The import also stops if the recorded dimensions do not match the -data. +Select a `.fid` directory containing both `fid` and `procpar`, or the `fid` file +inside it. Supported acquisitions are raw 1D, a series varying one ungrouped +parameter, and 2D with `phase=[1,2]`. Samples may be big-endian 16-bit integers, +32-bit integers, or 32-bit floating-point values. + +Grouped or multiple parameter arrays, other phase orders, Varian NUS, more than +two dimensions, and processed spectra are unsupported. Import stops if the +recorded dimensions or component arrangement do not match the data. ## SCIEX legacy WIFF diff --git a/docs/src/content/docs/zh-cn/guides/importing-data.md b/docs/src/content/docs/zh-cn/guides/importing-data.md index 2d73c24f..1e7d6860 100644 --- a/docs/src/content/docs/zh-cn/guides/importing-data.md +++ b/docs/src/content/docs/zh-cn/guides/importing-data.md @@ -9,9 +9,9 @@ PlotX 直接读取厂商 LC–MS、NMR、XPS、AFM 与电生理格式，无需 | 格式 | 扩展名 | 说明 | | --- | --- | --- | -| JEOL Delta | `.jdf` | 1D、2D 及伪 2D（DOSY / T1 / T2） | -| Bruker TopSpin | `fid` / `ser` 目录 | 1D 与 2D | -| Varian/Agilent VnmrJ | `.fid` 目录 | 原始时域 1D 与常规 2D | +| JEOL Delta | `.jdf` | 原始与已处理的 1D、2D 及参数系列；实验性支持 | +| Bruker TopSpin | `fid` / `ser` / `pdata` | 原始及已处理的 1D、2D；NUS 为实验性支持 | +| Varian/Agilent VnmrJ | `.fid` 目录 | 原始 1D、`phase=[1,2]` 的 2D，或单参数变化系列 | | Waters MassLynx RAW | `.raw` 目录 | 已验证的低分辨率数据，包括 SQD2 数据 | | SCIEX legacy WIFF | `.wiff` + `.wiff.scan` | 支持单样本与多样本 legacy 数据；两个文件必须放在一起 | | Rigaku 粉末 XRD | `.rasx`、FI `.raw`、RAS_RAW `.txt` | 衍射图样、采集元数据，以及文件提供的衰减系数 | @@ -40,14 +40,44 @@ PlotX 直接读取厂商 LC–MS、NMR、XPS、AFM 与电生理格式，无需 CasaXPS `.txt` 按结构头内容识别，而不是只看扩展名；其他 `.txt` 仍进入表格导入。 能量轴与拟合细节见 [XPS 工作流](/zh-cn/guides/xps/)。 +## 查看导入进度 + +科学数据文件与采集目录在后台逐个加载，期间可以继续操作。完成的数据集会出现在 +画板上，不会切换当前页面或选择。状态栏显示当前文件及成功、失败数量；错误可在 +诊断历史中查看。追加导入会排在当前批次之后。打开、关闭或新建项目会取消旧项目 +尚未完成的导入。 + +NMR 导入包含默认处理，NUS 重建可能比读取文件耗时长得多。项目文件、表格预览、 +ZIP 文件和手动补录采样表的导入使用各自的导入流程。 + +## 补录缺失的 NMR 采样表 + +此选项适用于受支持的二维 Bruker NUS 或只采集了部分间接网格点的 JEOL 数据。 +请准备原始采样表，以及含有足够网格与校准信息、可用于核对采样表的采集文件。 + +1. 选择 **File → Import NMR with Sampling Table…**（命令面板中也可搜索）， + 再选择 Bruker `ser` 或 JEOL `.jdf` 文件。 +2. 填写采样表来源或说明，以及原始间接轴的完整网格点数（包括未采样点）。 +3. 填写 **Lanes per observation**：每个所列间接点采集的分量记录数。 + 请按采集设置填写，不是填写采样表的点数。 +4. 根据原表选择 **Zero-based**（从 0 开始）或 **One-based**（从 1 开始）。 + 按采集顺序每行填写一个间接索引；每行代表该次观测的全部分量，并保留重复观测。 +5. 点击 **Validate and import**。校验失败时，根据提示与采集记录核对。 + 与已有采样表冲突、观测或分量数量不符、缺少网格或校准信息，都会阻止导入。 + +导入不会修改厂商文件。保存项目后，采样表及其来源会一并保留，重开无需原文件。 +重复坐标可以导入，但目前不能进行 [NUS 重建](/zh-cn/guides/processing/#重建非均匀采样谱)。 +脚本用法见 [CLI 声明格式](/zh-cn/reference/cli/#采样声明)。 + ## Varian/Agilent VnmrJ 要导入原始 1D 或常规 2D 采集，请选择 **Open Folder…** 并选中 `.fid` 目录。也可以选择 **Open File…**，再选中目录内的 `fid` 文件。请将 `fid` 和 `procpar` 保持在同一目录中。 -暂不支持处理后的谱图、3D 或 4D 实验、成像、伪 2D 实验、非均匀采样及 -其他数组实验。兼容性详情见[文件格式](/zh-cn/reference/file-formats/)。 +也支持仅改变一个非分组参数的系列。二维数据要求 `phase=[1,2]`。分组或多个参数数组、 +其他 phase 顺序、Varian NUS、已处理谱和超过二维的数据不受支持。 +校准要求与格式限制见[文件格式](/zh-cn/reference/file-formats/#nmr-数据与项目)。 ## mzML diff --git a/docs/src/content/docs/zh-cn/guides/processing.md b/docs/src/content/docs/zh-cn/guides/processing.md index 111ed1b5..a8290ae9 100644 --- a/docs/src/content/docs/zh-cn/guides/processing.md +++ b/docs/src/content/docs/zh-cn/guides/processing.md @@ -14,7 +14,7 @@ XPS 为每个谱区使用独立的有序 recipe，而不是 NMR 管线。recipe ## 典型的 1D 谱 -新导入的时域 1D 数据集已带有标准管线——切趾、零填充、FFT、相位校正、基线 +数字滤波延迟已知的时域 1D 数据集带有标准管线——切趾、零填充、FFT、相位校正、基线 校正，按此顺序——并默认启用自动相位。多数情况下屏幕上的谱图立即可用， 一次会话最多只需调整三处： @@ -30,6 +30,53 @@ XPS 为每个谱区使用独立的有序 recipe，而不是 NMR 管线。recipe **Imported spectrum**，没有时域步骤，也没有 FFT——PlotX 不会为它没有采集过的 自由感应衰减凭空造一个 FID。 +**Reset to default** 恢复导入时的设置。滤波延迟未知的原始数据保持未校正； +已导入的频谱不添加 FFT，仅含实部的频谱不添加相位校正步骤。 + +## 分析前检查校准条件 + +选择分析功能前，先检查坐标单位：FID 使用秒，频谱使用 Hz 或 ppm。缺失的校准信息 +不会按零处理。 + +- **Reference** 需要 ppm 坐标，只平移坐标，不改变强度。 +- **CRAFT** 需要复数 FID，以及已知的谱宽、观测频率、化学位移参考和数字滤波延迟。 +- **DOSY 图**需要已校准为 ppm 的频域谱系列。 +- **多重峰分析**需要 ppm 谱和已知的化学位移参考频率，才能报告以 Hz 为单位的耦合常数。 + +观测频率与化学位移参考频率用途不同，解读 ppm 与 Hz 的换算时不能混用。 +对导入的 Bruker 已处理谱，`SF` 提供 ppm 间隔换算为 Hz 的参考频率，但不能补全 +缺失的采集频率或载频信息，也不能证明数据已完成数字滤波校正。 + +## 重建非均匀采样谱 + +非均匀采样（NUS）只采集间接时间轴上的部分点。PlotX 根据采样表重建完整网格， +再通过 F1 FFT 得到二维频谱。网格点数不必为 2 的幂。 + +1. 导入受支持的 Bruker NUS 或 JEOL 采集及其采样表。若缺少采样表，请补齐厂商配套 + 文件，或使用 [Import NMR with Sampling Table](/zh-cn/guides/importing-data/#补录缺失的-nmr-采样表)。 +2. 等待默认处理完成。PlotX 自动估计噪声，无需选择噪声区或输入噪声值。 +3. 解读结果前，检查状态栏是否报错。导入时若重建失败，原始观测仍可查看，但该视图 + **不是重建后的频谱**。后续修改处理设置若失败，画面会保留上一次成功的结果。 + +### 噪声与收敛设置 + +自动噪声估计基于当前 F2 处理后的频谱，修改 F2 步骤后会重新计算。 +至少需要 48 条观测和 32 个 F2 频率点；若用留出的观测检验估计结果，则至少需要 +32 条观测和 128 个频率点。满足数量要求不保证估计可靠，还须通过噪声质量检查。 + +如果已有独立测得的噪声标准差，可在 **Non-uniform sampling** 下启用 +**Override automatic noise estimate**。输入值的单位应与 F2 处理后频谱的幅度一致。 +0 表示无噪声输入，不表示自动估计。若修改 F2 处理改变了噪声幅度，须相应更新此值。 + +最大迭代次数可设为 1–2048；达到上限仍未收敛时会报错。要得到两个频率轴，请保持 +F2 和 F1 的 FFT 步骤启用。噪声设置随项目保存，但不会通过可复用处理配方应用到其他采集。 + +### 输入限制 + +重建要求采样表没有重复坐标，且具有重建间接轴所需的实部和虚部信号信息。 +并非所有采集排列都受支持；不支持的排列会报错。 +重复观测可以导入和保存，但目前不能重建；不要通过删除采样表中的重复项绕过此限制。 + ## 处理界面在哪里 处理以卡片形式出现在画布右上角，可从 Ribbon 的 **Process** 页签打开，或用 @@ -104,6 +151,11 @@ FFT 是一个普通的 *Time to Frequency* 类型步骤，而不是列表中固 **Group-delay correction** 开关，按数据集设置，在管线之前应用。它对 1D 与 2D 数据一视同仁：在 2D 数据集上关掉它，直接维同样保持 未校正。 +Bruker 数据优先使用非负的 `GRPDLY` 作为延迟值，包括零。缺少该参数或其值为 -1 时， +PlotX 根据受支持的 `DSPFVS`/`DECIM` 设置确定延迟；否则延迟保持未知，原始数据首先 +显示为 FID。若要查看未校正的频谱，请在 **Advanced** 中关闭 **Group-delay correction** +并启用 FFT。检查结果是否存在滤波引起的失真；关闭校正不能补全 CRAFT 所需的延迟信息。 + ## 切趾 点击 **Apodize** 步骤即可展开它的设置。所有控件都直接显示在 @@ -161,6 +213,13 @@ FFT 是一个普通的 *Time to Frequency* 类型步骤，而不是列表中固 通过[自动化](/zh-cn/guides/automation/)读写时，这些值保持各自的单位：相位角 为弧度，pivot 为分数。 +二维数据使用所选自动方法，从最强实部或虚部信号所在的谱线估计相位，再对整个系列 +施加相同的校正，保留各行的相对符号。若自动方法失败，请查看错误，选择其他方法或 +手动调整相位。 + +JEOL COSY 的 F2 和 F1 均支持相位校正与 Reference。提取行得到 F2 谱线，提取列得到 +F1 谱线；两者均保留实部和虚部，可继续进行相位校正。 + ## 基线校正 基线校正默认关闭。谱图需要时启用该步骤即可。 diff --git a/docs/src/content/docs/zh-cn/reference/cli.md b/docs/src/content/docs/zh-cn/reference/cli.md index c2553f90..df9f3ca5 100644 --- a/docs/src/content/docs/zh-cn/reference/cli.md +++ b/docs/src/content/docs/zh-cn/reference/cli.md @@ -16,9 +16,9 @@ description: 不打开应用即可运行导入、处理、导出和已保存的 ## 检查与处理数据 ```sh -plotx-cli inspect [--json] +plotx-cli inspect [--json] [--sampling-declaration ] plotx-cli craft --output [--region ]... [--expected-ratio ]... -plotx-cli process --scheme --output [--format svg|pdf|png|tiff|jpeg] +plotx-cli process --scheme --output [--format svg|pdf|png|tiff|jpeg] [--sampling-declaration ] ``` `inspect` 检测、加载并描述一个受支持的数据集；`--json` 输出稳定的机器 @@ -27,6 +27,15 @@ plotx-cli process --scheme --output [--format 对 XPS 还会报告测量位置数、谱区数、总点数、谱区名称，以及具有结合能轴或仅有 动能轴的谱区数量。 +对 NMR，`inspect` 描述源数据，不执行处理。Bruker 实验目录优先选择原始数据； +要检查已处理谱，请明确指定谱文件。目录有歧义时，请指定具体文件或处理目录。 +NUS 数据报告的形状包含未采样网格点，不等于实际观测数。时间、频率或参数轴混合时， +报告中的 `domain` 为 `"mixed"`。 + +处理前请检查报告中的警告。缺失的校准或数字滤波延迟保持未知，详见 +[NMR 格式限制](/zh-cn/reference/file-formats/#nmr-数据与项目)。处理失败会指出 +具体步骤，并返回非零退出码。 + `process` 是"一次导入、一个[处理配方](/zh-cn/guides/templates/)、一次 图形导出"的便捷路径。省略 `--format` 时按输出文件扩展名推断格式。 @@ -38,12 +47,63 @@ plotx-cli process --scheme --output [--format 各区域相干振幅、恰好两个区域时的振幅比、诊断和质量检查。即使选择范围较宽， 结果也仍按你给出的区域汇总。 +CRAFT 要求已知谱宽、观测频率、化学位移参考和数字滤波延迟。信息缺失时， +报告会为该输入记录失败。ppm 参考频率与观测频率分别保留；报告中的 +`chemical_shift_reference.reference_frequency_mhz` 决定 Hz 到 ppm 的换算。 +FFT 交叉检查幅度受建模区间、指数窗和零填充影响，用于评估拟合； +比较区域振幅比时，应使用报告中的相干振幅。 + 恰好指定两个区域时，可按输入顺序重复 `--expected-ratio`，将测得的振幅比与参考值 比较。报告会记录相对误差，误差不超过 5% 时通过检查。`all_succeeded` 表示所有 计算是否完成；`all_quality_checks_passed` 的要求更严格：输入或拟合出现警告、 选定区域没有分量，或达到诊断限制时都会为 `false`。即使命令完成，质量结果为 `false` 也表示数据需要进一步的科学复核。 +## 采样声明 + +`inspect` 和 `process` 可使用 `--sampling-declaration sampling.json`，为支持范围内的 +二维 Bruker NUS 或仅采集部分间接网格点的 JEOL 数据补录采样表。JSON 文件上限为 8 MiB， +每个字段都必须明确提供，例如： + +```json +{ + "assertion_id": "my-sampling-table-1", + "source": "user-provided sampling table from experiment notes", + "grid_shape": [4], + "coordinates": [[4], [2]], + "index_base": "one", + "component_counts": [2] +} +``` + +请根据原始采集记录填写各字段： + +| 字段 | 填写内容 | +| --- | --- | +| `assertion_id` | 此采样声明的标识。 | +| `source` | 采样表来源，例如采集日志。 | +| `grid_shape` | 间接轴完整网格点数（包括未采样点），写成单元素数组。 | +| `coordinates` | 按采集顺序填写每次观测的间接索引，每个索引各占一个数组。 | +| `index_base` | 索引从 0 开始填 `"zero"`，从 1 开始填 `"one"`。 | +| `component_counts` | 每次观测的分量记录数（lane 数），写成单元素数组。 | + +示例表示完整网格有 4 点，每次观测有 2 个分量，按从 1 开始的索引依次采集第 4、2 点。 +每行坐标代表该次观测的全部分量。必须保留采集顺序和重复观测；重复观测可以导入， +但目前不能进行 NUS 重建。 + +PlotX 会将声明与采集数据核对。缺少网格或校准信息、分量或观测数量不符、与已有采样表 +冲突，都会报错。此选项不适用于 Varian 数据、已处理谱或支持范围以外的二维采集形式。 + +例如，将声明保存为 `sampling.json`，再检查采集数据： + +```sh +plotx-cli inspect experiment/ser --sampling-declaration sampling.json --json +``` + +工作流工具 `data.import` 的可选参数 `sampling_declaration` 接受同一个 JSON 对象。 +声明必须适用于该导入节点中的每个路径；不同采样表应使用不同节点。保存项目会保留 +声明，不会修改厂商文件。 + ## 运行工作流 ```sh diff --git a/docs/src/content/docs/zh-cn/reference/file-formats.md b/docs/src/content/docs/zh-cn/reference/file-formats.md index 0d6c39a5..83670ea0 100644 --- a/docs/src/content/docs/zh-cn/reference/file-formats.md +++ b/docs/src/content/docs/zh-cn/reference/file-formats.md @@ -48,16 +48,37 @@ TIFF Pages…** 导入 PlotX 能够读取的所有页面。导入后的各页可 任何机器上应用到一整个系列的同类实验。见 [配方与模板](/zh-cn/guides/templates/)。 +## NMR 数据与项目 + +PlotX 支持一维和二维 NMR，包括带参数轴的系列。请根据要使用的数据选择输入： + +- **Bruker：**选择实验目录可读取原始数据，选择 `pdata` 内的谱文件可读取已处理谱。 + 若目录中有多个无法区分的候选，请指定具体文件或处理目录。支持 States 二维数据 + （`FnMODE=4`）。已处理谱保留文件中提供的实部和虚部分量。 +- **JEOL：**选择含有受支持的原始或已处理采集数据的 `.jdf` 文件。 +- **JCAMP-DX：**支持 5.00 或 5.01 版普通 XYDATA 频谱，每个坐标对应一个强度值， + 坐标单位可为 Hz 或 ppm。不支持 LINK、NTUPLES 和峰表。 + +缺失的核种、频率参考或数字滤波延迟信息保持未知。Hz 谱可以直接显示，但部分分析 +需要额外校准。滤波延迟未知的原始数据首先显示为 FID；查看未校正频谱的方法见 +[群延迟校正](/zh-cn/guides/processing/#群延迟校正)，选择分析功能前请核对 +[校准条件](/zh-cn/guides/processing/#分析前检查校准条件)。坐标未知或数据排列不受支持时， +导入会给出错误或诊断信息。 + +JEOL 与 Bruker NUS 为实验性支持，未覆盖所有仪器和采集设置。使用不熟悉的采集类型时， +请检查导入诊断，并与可信参考结果比较后再使用分析结果。 + +保存 `.plotx` 项目会保留导入的 NMR 分量、采样顺序、来源信息、警告和处理设置。 +重开项目时，PlotX 根据保存的设置重新计算频谱，无需原始厂商文件。 + ## Varian/Agilent VnmrJ 原始 NMR -PlotX 可导入原始时域 1D 和常规 2D 采集。请选择 `.fid` 目录或其中的 -`fid` 文件；该目录中必须同时存在 `fid` 和 `procpar`。仅有 `.fid` -目录名不足以识别数据。 +选择同时含有 `fid` 与 `procpar` 的 `.fid` 目录，或其中的 `fid` 文件。支持原始一维 +数据、仅改变一个非分组参数的系列，以及 `phase=[1,2]` 的二维数据。样本格式可为 +大端字节序的 16 位整数、32 位整数或 32 位浮点数。 -导入器支持常见的 16 位整数、32 位整数和 32 位浮点样本格式，包括常规 -States 2D 数据。暂不支持处理后的谱图、3D 或 4D 实验、成像、伪 2D 实验、 -非均匀采样，以及除 phase 以外的参数数组。如果文件记录的维度与数据不一致， -导入也会停止。 +不支持分组或多个参数数组、其他 phase 顺序、Varian NUS、超过二维的数据和已处理谱。 +记录的维度或分量排列与实际数据不符时，导入会停止。 ## SCIEX legacy WIFF