2026 10 gg loinc sct upgrade

CHANGELOG.md

@@ -5,6 +5,36 @@ All notable changes to Health Intersections FHIRsmith will be documented in this
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [0.18.1] - 2026-10-25
+
+
+### Security
+
+-
+
+### Added
+
+- ICF (`http://hl7.org/fhir/sid/icf`): qualified codes. A category may be followed by `.` and one qualifier digit per position, the positions depending on the component: extent for body functions (`b`); extent, nature and localization for body structures (`s`); performance, capacity, and optionally capacity with assistance and performance without assistance for activities and participation (`d`); one for environmental factors (`e`), a barrier after `.` or a facilitator after `+` in place of the `.`. Each digit is checked against its scale in the code system (`q1`-`q6`, added to the ICF code systems in fhir.tx.support with a `kind` property). A qualified code gets a generated display (8, not specified, is left out of it; `d` qualifiers are labelled `performance`, `capacity`, ...), its qualifiers as properties in `$lookup` (`extent`, `nature`, `localization`, `performance`, `capacity`, `capacity-with-assistance`, `performance-without-assistance`, `barrier`, `facilitator`), and sits under its category for subsumption and `is-a` filters. Value sets can control qualifiers with an `exists` `true`/`false` filter on any of those properties (8 counts as absent) and `qualified` `=` `false`. ICF is now marked as not closed. The behaviour is in `tx/cs/cs-icf.js`, a subclass of the CodeSystem provider that `FhirCodeSystemProvider.create()` picks by url - the way to give any code system loaded from a package behaviour of its own
+- SNOMED CT alternate identifiers (`sct2_Identifier`, e.g. the LOINC Extension's LOINC codes): the importer reads them into the cache (cache format 18, written only when there are identifiers, so other caches stay 17). `$lookup` returns them as Coding properties named for the scheme (`loinc` = `http://loinc.org#...`), by default and when asked for. `$expand`: the filter `mapped:<scheme>` `exists` `true`/`false` (e.g. `mapped:30051010000102`) for concepts that do or don't have one. `$translate`: each module with identifiers defines an implicit map `<edition>/version/<date>?fhir_cm=<module>` (e.g. `?fhir_cm=11010000107`), `equivalent` in both directions, nominated by url or found from the systems (`targetSystem` `http://loinc.org` from a SNOMED CT coding, or `http://snomed.info/sct|<edition>` from a LOINC one)
+- `$lookup` on SNOMED CT: when the request names properties, only those are returned - effectiveTime, module, parent, child and the defining relationships are no longer added regardless
+- Implicit ConceptMaps nominated by url are looked for in every version of a code system, not just the first loaded (a SNOMED CT edition's maps are named by its edition URI)
+- SNOMED CT test edition: a subset of the LOINC Extension (LOINC ontology, module `11010000107`) - twelve LOINC-coded concepts with their LOINC identifiers (`sct2_Identifier`), plus the module and identifier scheme concepts - generated from the LOINC Extension release by `utilities/make-sct-loinc-subset.js`. The test cache is now `sct_test_20250909d.cache` (2280 concepts)
+- LOINC importer: mappings from LOINC codes, parts and answers to other code systems, in two new tables - `ExternalCodes` (system, version, code, display; one row per target) and `ExternalMappings` (LOINC code to target, with an equivalence). Read from `PartFile/PartRelatedCodeMapping.csv` (the file's own equivalence; the RadLex rows are left out for now), `LoincIeeeMedicalDeviceCodeMappingTable` (MDC, `urn:iso:std:iso:11073:10101`, the CF_CODE10 as the code and the REFID as display; IEEE's `broader`/`narrower` are from the LOINC side, so they become `narrower`/`wider`) and the `ExtCode*` columns of `AnswerList.csv` (`equivalent`; LOINC's `CMS/HCPCS` and `https://www.cdc.gov/phin` systems become `http://www.cms.gov/Medicare/Coding/HCPCSReleaseCodeSets` and `urn:oid:2.16.840.1.113883.6.238`). Both mapping files are optional. LOINC caches need rebuilding to get them; nothing uses them yet
+- LOINC importer (and subsetter): CSV files are read by record, not by line, so a quoted field with a line break in it is one record. LOINC 2.83's `ComponentHierarchyBySystem.csv` has one (the text of `LP409416-7`), and the import failed with `UNIQUE constraint failed: Config.ConfigKey` - the second half of the record looked like a second root
+- LOINC: the mappings to other code systems are used. `$lookup` returns them as Coding properties named for the map (`sct`, `chebi`, `mdc`, `race`...), with the equivalence as a subproperty, by default and when asked for. `$translate`: each target system has an implicit map `http://loinc.org/cm/<name>` - `loinc`, `ncbi` (PubChem), `sct`, `hcpcs`, `gene-names`, `rxnorm`, `chebi`, `ncbi-clinvar`, `ncbi-gene`, `ncbi-taxo`, `mdc`, `race` - that works in both directions (LOINC to the system, and the system to LOINC), found from the target system or nominated by url. Found from the systems, a map is only offered for a code it has a mapping for, so translations that a SNOMED CT edition answers (the LOINC Extension) are unchanged. Targets in other systems (`CMS LCDS`, `Unicode`) are not used yet. Code system providers' `findImplicitConceptMaps` now also gets the code being translated
+
+### Changed
+
+-
+
+### Fixed
+
+-
+
+### Tx Conformance Statement
+
+(paste)
+
 ## [0.18.0] - 2026-10-23
 
 A big release: a new IPS module (prototype), a secured and documented registry with uptime reporting, a syndication feed for the terminology server, a daily email report, ICF qualified codes, SNOMED CT edition to edition translation and much more ECL, and the ICD-11 representation agreed at the October 2026 test-case review meetings. Plus the outcome of security reviews of the registry, IPS and token modules. The README now says what state each module is in.
@@ -43,7 +73,6 @@ Note: We are heading towards a 1.0.0 release.
 - Home page and dashboard: the uptime box now also reports the server's overall uptime - `Uptime: 38m 16s (99.3% overall)` - the proportion of the time since the first recorded run that the server has been running. Each run is recorded in the statistics database (`server_runs` in `stats.db`) with a heartbeat every minute, so a crash loses at most a minute. Not shown when statistics are turned off
 - `library/fhir-search-values.js`: the date range, prefix and comma-splitting parts of FHIR search, moved out of the testing module for sharing
 - `library/package-manager.js`: a cached package folder with no `package/package.json` (a download that failed after the folder was made) is fetched again instead of being used
-- ICF (`http://hl7.org/fhir/sid/icf`): qualified codes. A category may be followed by `.` and one qualifier digit per position, the positions depending on the component: extent for body functions (`b`); extent, nature and localization for body structures (`s`); performance, capacity, and optionally capacity with assistance and performance without assistance for activities and participation (`d`). Qualifiers on environmental factors (`e`) are not supported yet. Each digit is checked against its scale in the code system (`q1`-`q4`, added to the ICF code systems in fhir.tx.support with a `kind` property). A qualified code gets a generated display (8, not specified, is left out of it; `d` qualifiers are labelled `performance`, `capacity`, ...), its qualifiers as properties in `$lookup` (`extent`, `nature`, `localization`, `performance`, `capacity`, `capacity-with-assistance`, `performance-without-assistance`), and sits under its category for subsumption and `is-a` filters. Value sets can control qualifiers with an `exists` `true`/`false` filter on any of those properties (8 counts as absent) and `qualified` `=` `false`. ICF is now marked as not closed. The behaviour is in `tx/cs/cs-icf.js`, a subclass of the CodeSystem provider that `FhirCodeSystemProvider.create()` picks by url - the way to give any code system loaded from a package behaviour of its own
 - `$translate`: SNOMED CT edition to edition translation with no map - from an edition to one it depends on (per the module dependency reference set), using the implicit map `<edition>/version/<date>?fhir_cm=edition/<target module>` (the url agreed with the SNOMED committee), found from `targetSystem` or nominated by url. A concept in the target edition's modules is its own translation; an inactive concept translates through its historical associations; a fully defined extension concept translates to its normal form (or the target concept with that normal form), when that uses only target edition content; anything else translates to its proximal target-edition ancestors as `source-is-narrower-than-target`, unless those are only top-level hierarchies, which is reported as the reason there is no translation. The source concept in a match carries its version, as given
 - `$translate`: when the target code a map directs to is not in its code system's normal form, the match has a `normalized-concept`. A source code that has a normal form is looked up both as given and in its normal form (a map entry written in the normal form matches any code with that normal form; the map's own entries are not normalised). A target in a code system the server doesn't know gets a `note` saying it was not validated, and a target code that is not valid gives an `error`
 - `$lookup`: a warning when the `displayLanguage` asked for has no display for the code, and the display is in the default language (`LOOKUP_DISPLAY_LANGUAGE_FALLBACK`)

library/daily-report.js

@@ -85,6 +85,15 @@ function formatSpan(ms) {
   return parts.join(' ');
 }
 
+/** "24 hours", "3 days" - for the report title; rounded, since it's a description of the period */
+function describeSpan(ms) {
+  const hours = Math.round(ms / 3600000);
+  if (hours < 48) {
+    return plural(Math.max(1, hours), 'hour');
+  }
+  return plural(Math.round(hours / 24), 'day');
+}
+
 function plural(n, word) {
   return `${n} ${word}${n === 1 ? '' : 's'}`;
 }
@@ -243,8 +252,7 @@ class DailyReport {
    */
   async build(since, now = Date.now()) {
     const parts = [];
-    parts.push(`# ${this.serverId} daily report`);
-    parts.push(`${localDateTime(now)}, covering the ${formatSpan(now - since)} since ${localDateTime(since)}.`);
+    parts.push(`# ${this.serverId} daily report for ${localDateTime(now)} covering the last ${describeSpan(now - since)}`);
     parts.push('## Stability');
     parts.push(this.stabilityParagraph(since, now));
     for (const [name, module] of Object.entries(this.modules)) {
@@ -347,25 +355,26 @@ class DailyReport {
    * the disk they're on.
    */
   systemParagraph(now) {
-    const sentences = [];
-    sentences.push(`${this.serverId} is running FHIRsmith v${this.version} on Node.js ${process.version} (${os.type()} ${os.release()}, ${os.arch()}).`);
+    // one line each; the two trailing spaces are markdown's line break
+    const lines = [];
+    lines.push(`Versions: FHIRsmith = v${this.version}. Node = ${process.version} (${os.type()} ${os.release()}, ${os.arch()})`);
     if (this.stats && typeof this.stats.startTime === 'number') {
-      let up = `This run has been up for ${formatSpan(now - this.stats.startTime)}, since ${localDateTime(this.stats.startTime)}`;
+      let up = `Uptime: This run = ${formatSpan(now - this.stats.startTime)}`;
       const overall = typeof this.stats.overallUptime === 'function' ? this.stats.overallUptime(now) : null;
       if (overall) {
         const pct = overall.percent >= 100 ? '100' : (Math.floor(overall.percent * 10) / 10).toFixed(1);
-        up += `; over all its recorded runs (since ${localDate(overall.since)}) the server has been up ${pct}% of the time`;
+        up += `. All runs, ${pct}%`;
       }
-      sentences.push(up + '.');
+      lines.push(up);
     }
-    sentences.push(this.logsSentence());
+    lines.push(`Space: ${this.logsText()}`);
     if (this.npmAudit && typeof this.npmAudit.reportText === 'function') {
       const audit = this.npmAudit.reportText();
       if (audit) {
-        sentences.push(audit);
+        lines.push(`Audit: ${audit}`);
       }
     }
-    return sentences.join(' ');
+    return lines.join('  \n');
   }
 
   /**
@@ -388,7 +397,7 @@ class DailyReport {
     return `${plural(failed.length, 'background task')} ${failed.length === 1 ? 'is' : 'are'} in a failed state:\n\n${lines.join('\n')}`;
   }
 
-  logsSentence() {
+  logsText() {
     const logDir = folders.logsDir();
     try {
       const files = fs.readdirSync(logDir).filter(f => f.endsWith('.log'));
@@ -402,13 +411,13 @@ class DailyReport {
         const free = fstats.bavail * fstats.bsize;
         const size = fstats.blocks * fstats.bsize;
         const freePct = size > 0 ? Math.round((free * 100) / size) : 0;
-        disk = `, on a disk with ${formatBytes(free)} free of ${formatBytes(size)} (${freePct}% free)`;
+        disk = ` on a disk with ${formatBytes(free)} free of ${formatBytes(size)} (${freePct}% free)`;
       } catch {
-        disk = ', on a disk whose free space could not be read';
+        disk = ' on a disk whose free space could not be read';
       }
-      return `The logs folder (${logDir}) holds ${plural(files.length, 'log file')} totalling ${formatBytes(total)}${disk}.`;
+      return `${plural(files.length, 'log file')} totalling ${formatBytes(total)}${disk}.`;
     } catch (e) {
-      return `The logs folder (${logDir}) could not be read: ${e.message}.`;
+      return `the logs folder (${logDir}) could not be read: ${e.message}.`;
     }
   }
 }
@@ -417,6 +426,7 @@ module.exports = DailyReport;
 module.exports.nextMidnight = nextMidnight;
 module.exports.localDate = localDate;
 module.exports.formatSpan = formatSpan;
+module.exports.describeSpan = describeSpan;
 // for modules writing their paragraph: "3 packages", "1 package"
 module.exports.plural = plural;
 module.exports.localDateTime = localDateTime;

package.json

@@ -1,6 +1,6 @@
 {
   "name": "fhirsmith",
-  "version": "0.18.0",
+  "version": "0.18.1-snapshot",
   "txVersion": "1.9.6",
   "description": "A Node.js server that provides a collection of tools to serve the FHIR ecosystem",
   "main": "server.js",