Phase C: extended geometry, wwPDB evidence, ensembles, cryo-EM maps and batch CLI

.github/workflows/scientific-tests.yml

@@ -32,9 +32,21 @@ jobs:
         run: |
           node --check shinyRam/www/custom.js
           node --check shinyRam/www/prediction.js
+          node --check shinyRam/www/density.js
+          node tests/density-ui.test.cjs
           node tests/ui.test.cjs
           node tests/prediction-ui.test.cjs
         shell: bash
+      - name: Install optional official wwPDB XML parser
+        run: Rscript -e 'install.packages("xml2", repos="https://cloud.r-project.org")'
+        shell: bash
+      - name: Verify Phase C geometry, ensembles and offline CLI
+        run: |
+          Rscript tests/geometry.R
+          Rscript tests/experimental.R
+          Rscript tests/ensemble.R
+          Rscript tests/batch.R
+        shell: bash
       - name: Verify AlphaFold and ESMFold confidence formats
         run: Rscript tests/predictions.R
         shell: bash

.github/workflows/ui-preview.yml

@@ -26,7 +26,7 @@ jobs:
           use-public-rspm: true
       - name: Install the application's R dependencies
         run: |
-          Rscript -e 'install.packages(c("shiny", "shinyWidgets", "colourpicker", "bio3d", "NGLVieweR", "DT", "jsonlite"))'
+          Rscript -e 'install.packages(c("shiny", "shinyWidgets", "colourpicker", "bio3d", "NGLVieweR", "DT", "jsonlite", "xml2"))'
       - name: Validate publication figure and report exports
         run: Rscript tests/reports.R
       - name: Install headless-browser controller
@@ -53,6 +53,8 @@ jobs:
             https://files.rcsb.org/download/1D3Z.pdb \
             -o benchmarks/output/ui-preview/1D3Z.pdb
           Rscript tests/model-integration.R
+      - name: Exercise offline batch analysis and NMR-ensemble exports
+        run: Rscript tests/batch-integration.R
       - name: Start the Shiny application
         shell: bash
         run: |
@@ -71,6 +73,8 @@ jobs:
         run: node tests/ui-browser.cjs
       - name: Verify AlphaFold and ESMFold uploads in the live browser
         run: node tests/prediction-browser.cjs
+      - name: Verify official wwPDB evidence, density UI and NMR ensemble
+        run: node tests/phase-c-browser.cjs
       - name: Archive desktop/mobile screenshot previews and logs
         if: always()
         uses: actions/upload-artifact@v4

README.md

@@ -39,6 +39,20 @@ AlphaFold DB accession lookup and explicit AlphaFold 2/3, ColabFold and ESMFold
 
 For screenshots, limitations and the complete workflow see the [inspection and publication guide](docs/inspection-user-guide.md). The interface layout and browser verification history are documented in [interface-refresh.md](docs/interface-refresh.md).
 
+## Extended verification and research workflows (Phase C)
+
+RamplotR now provides optional **peptide omega and descriptive chi1
+measurements**, imports official **wwPDB rotamer, clash and bond/angle outlier
+annotations** as separate evidence, and can overlay **local CCP4/MRC cryo-EM
+maps** in NGL. Consistent multi-model structures can be analysed as circular
+phi/psi ensembles with residue-level classification agreement. For
+reproducible offline studies, `scripts/ramplotr-batch.R` processes a directory
+of PDB/mmCIF files and writes CSV, JSON, SVG and standalone HTML reports.
+None of the newly computed angle diagnostics is presented as an official
+MolProbity or density-fit score. See the
+[Phase C guide](docs/phase-c-guide.md) for usage, validation provenance,
+resource limits and scientific caveats.
+
 ## Independent Phase A validation
 
 The [independent validation protocol](docs/wwpdb-validation.md) uses official

docs/phase-c-guide.md

@@ -0,0 +1,138 @@
+# Phase C: structural verification, ensembles and batch analysis
+
+Phase C extends the established RamplotR Ramachandran analysis without
+changing its reference densities, four-region labels, or the historical
+`v0.1.0-legacy` release.
+
+## Extended native geometry
+
+Once a structure loads, expand **Extended structure verification** below the
+main plots. RamplotR calculates the peptide dihedral **omega** (CA–C–N–CA)
+only across geometrically connected peptide bonds, and **chi1** (N–CA–CB–X1)
+for residues with an appropriate first side-chain atom. Cβ measurements
+include the observed **CA–CB distance** (Å) and signed N–CA–C–CB tetrahedral
+volume (ų) for residues with all four atoms. These are descriptive
+measurements, not independently validated Cβ-deviation or chirality-outlier
+classifications. All angles are in degrees. Missing atoms, chain breaks and terminal residues have undefined
+measurements; they are not scored as outliers.
+
+For exploration, omega within 30° of 0° is labelled *cis*, omega within 30°
+of ±180° is labelled *trans*, and other measured values are marked *twisted*.
+These are descriptive flags, **not an independent MolProbity assessment**.
+Chi1 is a measurement, not a rotamer outlier prediction.
+
+## Independent experimental validation
+
+For a deposited structure, obtain the official wwPDB validation XML or
+`XML.gz` for the **same deposited entry and structural model**. Attach it in
+the expandable verification panel and confirm that it belongs to the
+loaded experimental structure. The app rejects official reports for declared
+predicted-model inputs, including AlphaFold DB. For local or accession-loaded
+experimental structures, you must still verify accession/model provenance:
+matching sequence numbering alone cannot establish that two deposits are the
+same experiment. The report is parsed locally and matched by **model,
+chain, residue number, insertion code and residue type**, with unlabelled
+alternate conformations preferred over alternate A.
+
+When available, the inspector and detailed CSV show independent wwPDB
+Ramachandran, side-chain rotamer, local clash, symmetry clash, bond-length
+outlier, bond-angle outlier, RSCC and RSRZ annotations. A missing independent
+record remains missing. The app reports exact matching coverage and preserves
+the source file's MD5 checksum in its HTML report.
+
+The original RamplotR contour interpretation and the wwPDB/MolProbity
+reference systems are **not equivalent**. The independent outlier-rich
+[Phase A results](phase-a-results.md) explicitly show genuine discrepancies.
+Importing a report does not rewrite the original region classification.
+Do not attach experimental wwPDB validation reports to an unrelated
+AlphaFold/ESMFold prediction, even if their sequences are similar.
+
+Official validation information:
+https://www.wwpdb.org/validation/validation-reports
+
+## Optional cryo-EM map overlay
+
+Open **Local cryo-EM density map** beneath the NGL viewer, choose your own
+CCP4/MRC file, and select *Show map*. The map is read by NGL directly in the
+browser, with no external map-fitting service. It is rendered as a translucent
+teal isosurface and can be adjusted from 0.5 to 5 sigma. *Remove* clears
+the map; loading a different structure also clears any previous overlay.
+Only files up to 64 MB are supported to protect ordinary laptop sessions.
+NGL can be slow with large maps; the overlay is a qualitative visual aid,
+**not an RSCC/Q-score or local map-model-fit measurement**. Use the
+corresponding official report or a validated external map-fit tool when
+quantitative claims are required.
+
+NGL stage and volume API:
+https://nglviewer.org/ngl/api/class/src/stage/stage.js~Stage.html
+
+## NMR / multi-model ensemble
+
+For an input containing multiple atom-compatible models, expand **Ensemble
+analysis** on the Summary tab, then select *Analyse ensemble*. It analyses
+up to the first 30 models on demand, using the currently chosen reference
+dataset, classification mode and plotting background. The tool matches
+residues by chain, residue number, insertion code and amino-acid identity,
+not by row number. It reports circular means and standard deviations of
+phi/psi angles, observed-model counts and the proportion of models agreeing
+on a class. Single-angle observations have undefined variability.
+
+A changed reference invalidates the previous analysis until recalculated.
+Clicking an ensemble residue selects it in the shared inspector, plot and 3D
+viewer. Use **Export ensemble CSV** to keep the per-residue results. A
+prediction ensemble is an ensemble of output conformations, not proof of
+experimental flexibility or pLDDT uncertainty.
+
+## Offline batch mode
+
+From the repository root, with R and Bio3D installed, run:
+
+```bash
+Rscript scripts/ramplotr-batch.R --input structures/ --output results/ \
+  --reference original --mode residue --model 1 --report
+```
+
+The command accepts one local PDB/mmCIF file or a nonrecursive directory of
+up to 1000 supported structure files. By default it produces residue-level
+CSV and JSON, a batch-summary CSV, and optionally a standalone SVG plus HTML
+report. Each file's output is named from its source filename, and existing
+files are protected unless `--overwrite` is specified. The full CSV
+includes omega/chi1 and available prediction-confidence or independent
+validation annotations.
+
+For a consistent NMR structure, request an ensemble table:
+
+```bash
+Rscript scripts/ramplotr-batch.R --input 1D3Z.pdb --output results/ \
+  --model 1 --ensemble-models 20 --report
+```
+
+For **one** experimental structure, add its official XML:
+
+```bash
+Rscript scripts/ramplotr-batch.R --input 1CRN.pdb --output results/ \
+  --validation-xml 1crn_validation.xml.gz --report
+```
+
+For an ESMFold prediction, explicitly set
+`--prediction-source esmfold`; never request this for an experimental
+structure. For AlphaFold 2/3, the CLI can read declared pLDDT from compatible
+B-factor files; AF3 confidence sidecar parsing remains available in the
+interactive Phase B uploader.
+
+Use `--no-json` to avoid the optional jsonlite dependency; `--report`
+requires htmltools and an SVG-capable R graphics device, while wwPDB XML
+requires xml2. The CLI never downloads structures or sends coordinates to
+external servers. Nonzero exit status indicates at least one failed input,
+and `batch-summary.csv` records any per-file errors.
+
+## Reproducibility and methodological limits
+
+The outputs record the current model, file MD5 checksum, reference file MD5,
+R/Bio3D versions, chosen scientific mode and prediction provenance where
+declared. The HTML report distinguishes computed geometry from imported
+independent evidence and prints model-ensemble statistics when available.
+Use the pinned Phase A corpus and reference-validation reports before
+making formal validation-performance claims. Avoid presenting any visual
+density overlay or geometric heuristic as an official experimental
+fit or MolProbity-equivalent score.

docs/phase-c-worklog.md

@@ -0,0 +1,15 @@
+# Phase C — structural verification and research workflows
+
+Base: `dad437d132c58452caa46950d644172e19b2eaa6` (Phase A and Phase B merged into `main`). Preserve scientific reference grids and `v0.1.0-legacy`.
+
+## Work packages
+
+- [ ] **Extended geometry**: compute interpretable omega cis/trans/twisted peptide flags and independent side-chain chi1 and Cβ geometry. Use explicit methodology and missing-value semantics; for authoritative rotamer/clash results, import wwPDB validation results rather than claim an ad hoc calculation is MolProbity-equivalent.
+- [ ] **Experimental evidence**: allow optional local official wwPDB validation XML and show its residue-specific rotamer/clash/outlier annotations; link to source and show differences against the native RamplotR region. Optional NGL density-map view for local cryo-EM maps with user-controlled contours, not an unvalidated map/model fit score.
+- [ ] **Ensembles**: compare coherent NMR/prediction models residue by residue using circular angle statistics, coverage and class-change summaries. Identify missing/misaligned atom records, not index-only matching.
+- [ ] **Batch interface**: a scriptable offline R command, safe output directories, per-residue CSV/JSON and standalone HTML/SVG exports; consistent options and provenance. Structure and reference files remain local.
+- [ ] Tests on synthetic fixtures and genuine multimeric/NMR samples, CI and docs. Keep advanced controls optional; no extra permanent tabs without need.
+
+## Methodological limits
+
+Independent wwPDB classifications and MolProbity rotamers/clashscores have different algorithms/reference populations and must not be relabelled as new native RamplotR reference classifications. Experimental B factors are not pLDDT. Map visuals are qualitative unless a validated map-fit engine is integrated. Aggregate ensemble measurements require matching residue IDs and explicitly report missing data. AlphaFold/ESMFold predictions remain supported, with their provenance retained.

scripts/ramplotr-batch.R

@@ -0,0 +1,35 @@
+#!/usr/bin/env Rscript
+# From the repository root:
+# Rscript scripts/ramplotr-batch.R --input structures/ --output results/ \
+#   --reference original --model 1 --report --ensemble-models 20
+help <- paste(
+  "RamplotR batch analysis (local structures; no network requests).",
+  "Usage: Rscript scripts/ramplotr-batch.R --input FILE_OR_DIRECTORY --output DIR",
+  "       [--reference original|alphafold|alphafold_filtered|astral2.08|custom_high_resolution]",
+  "       [--background General|GLY|PRO|preProline] [--mode residue|legacy]",
+  "       [--model N] [--ensemble-models N] [--max-files N]",
+  "       [--prediction-source experimental|alphafold2|alphafold3|esmfold|other_prediction]",
+  "       [--validation-xml PATH] [--report] [--no-json] [--overwrite]",
+  "",
+  "CSV output includes native diagnostic omega/chi1 angles. Rotamer/clash",
+  "annotations are exclusively from the optional independent wwPDB XML.",
+  "For ensemble analysis, the first N consistent atom-record models are analysed.",
+  sep="\n"
+)
+required <- c("shinyRam/R/io.R","shinyRam/R/batch.R")
+if(!all(file.exists(required)))
+  stop("Run this script from the RamplotR repository root.",call.=FALSE)
+for(filename in c("io","backbone","ramachandran","geometry","experimental",
+                  "ensemble","predictions","reports","batch"))
+  source(file.path("shinyRam","R",paste0(filename,".R")))
+args <- commandArgs(trailingOnly=TRUE)
+if("--help" %in% args) {cat(help,"\n");quit(status=0L)}
+options <- tryCatch(ram_batch_options(args),error=function(e) {
+  cat("Error:",conditionMessage(e),"\n\n",help,"\n",file=stderr())
+  quit(status=2L)
+})
+if(options$json && !requireNamespace("jsonlite",quietly=TRUE))
+  stop("Install jsonlite for machine-readable JSON, or supply --no-json.")
+results <- ram_batch_run(options,repo_root=".")
+print(results,row.names=FALSE)
+quit(status=if(any(results$status=="error")) 1L else 0L)