diff --git a/CHANGELOG.md b/CHANGELOG.md index fbf902bc..63a6bccc 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -13,6 +13,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Added +- IPS: `modules.ips.validator.jar` - a validator jar of the module's own (a local build of org.hl7.fhir.core, say), used as it is, never downloaded or updated +- IPS: national profiles - the profile on Bundle that a national IG defines for an IPS document (its jurisdiction - an ISO 3166 country code, or a UN M49 region - the IG's package id and version, and the profile's url). Shared across all projects at `/ips/national-profiles`; only the administrator adds, edits and deletes them. A new processing step, after validation, validates each document against each national profile (loading the IG's package into the validator), and keeps the outcome and whether it passed (no errors); the document's new National Profiles view lists them, and shows each one's validation as the Validation view does - Registry: `GET /tx-reg/choose-server` - given the code systems a request involves and the client's default server, which one server to send it to, and a log of why. The same choice the Java tooling makes, for clients that aren't Java - SNOMED CT ECL: member filters (`^ refset {{ M ... }}`) - `effectiveTime` (with a time value or set), `active`, `moduleId` (any sub expression) and the reference set's own fields (`targetComponentId = << X`, `mapTarget = "..."`, `mapGroup >= #2`, ...), comma-separated and repeatable. A member filter anywhere other than directly after a memberOf constraint is a syntax error (`INVALID_ECL`). - SNOMED CT ECL: concept filters (`{{ C ... }}`) - `active`, `definitionStatus` (primitive/defined), `definitionStatusId`, `moduleId` (a concept, a bracketed concept list or any sub expression) and `effectiveTime` @@ -27,10 +29,16 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Changed +- IPS: a clinical profile's page is the same for everyone. The required sections and entries are a table of Expectation, Section and Entry (`Code is system|code`, or `Code is in` a value set, with `[m..n]` when the cardinality isn't 1..1), and the rules about resources a table of Expectation, Resource Set (`Conditions where Code is ...`) and Rule (in words for `.exists()`, `.x.exists()`, `.x.y.exists()`, `.all(sameSubject())`, `.conformsTo(url)` and `.category.hasCoding(system, code)`). The administrator adds, edits and deletes the rules on that page one at a time, in a dialog, each saved by itself; the full editor is now Edit all - Move the hard-coded link about adding to the terminology server ecosystem to the registry control file +- Code systems, value sets and concept maps that both hl7.terminology and an endpoint's FHIR core package provide: THO's is still always the default for the url, but the core versions are no longer dropped - a reference that names a core version (`url|5.0.0`) gets it, where THO doesn't have that version. Previously they weren't available at all, so a reference to one failed, including THO's own value sets that pin a core version (`ValueSet/concept-property-type` 6.0.1 includes `concept-property-type` 5.0.0); and `valueSetVersion=5.0.0` found nothing ### Fixed +- VSAC: the cache folder `terminology-cache/vsac` is created if it doesn't exist. Only its parent was being created (the folder name was passed as if it were a file name), so on a fresh install the VSAC database could not be created (`SQLITE_CANTOPEN`) and every VSAC value set failed +- CodeSystem resources: the `notSelectable` property is recognised as the tx-ecosystem requirements say - by the `concept-properties#notSelectable` uri whatever local code the code system gives it, and by the code `notSelectable` only where the code system doesn't declare that code with a different uri. A code system that used `notSelectable` for a property of its own had its concepts marked `abstract` in expansions and treated as not selectable (`notSelectable-unprop-all`, `notSelectable-unprop-true`); a filter on that code is still a filter on the code system's own property +- LOINC: `$lookup` returns `CLASS` as a string, the class name (e.g. `CHEM`), with the Part code for the class as `partCode`; it returned the Part code as a code +- IPS: the pages link the module's browser scripts with their version (`?v=`), so a browser picks up a changed script as soon as the server is restarted with it, instead of using its cached copy for up to an hour - SNOMED CT ECL: a hierarchy operator on a bracketed expression was silently dropped - `< (A OR B)` returned A and B themselves rather than their descendants. It now applies to every member of the set, as do operators on `*` and on `^ refset`, which were previously rejected as unsupported - SNOMED CT ECL: `^` on a reference set whose members are descriptions (a language reference set) returned an empty expansion; it is now reported as `UNSUPPORTED_ECL` when the reference set is named, and skipped when it merely falls inside a computed operand such as `^ *` - SNOMED CT ECL: an attribute name with no comparison (`: 363698007` with nothing after it) was accepted by the parser and failed later as unsupported; it is a syntax error diff --git a/ips/ips.js b/ips/ips.js index 73ffaaf7..353219c4 100644 --- a/ips/ips.js +++ b/ips/ips.js @@ -39,10 +39,12 @@ * @module ips/ips */ +const http = require('http'); const express = require('express'); const path = require('path'); const fs = require('fs'); const crypto = require('crypto'); +const scripts = require('./scripts'); const rateLimit = require('express-rate-limit'); const escape = require('escape-html'); const session = require('express-session'); @@ -79,7 +81,69 @@ const TX_STARTUP_GRACE_MS = 5 * 60 * 1000; const FHIR_JSON = 'application/fhir+json; charset=utf-8'; const CODE_RE = /^[a-z0-9][a-z0-9-]{1,31}$/; // paths at the root that can't be project codes -const RESERVED = new Set(['login', 'logout', 'admin', 'metadata', 'openapi', 'static', 'assets', 'lookup']); +const { jurisdiction: jurisdictionOf } = require('./jurisdictions'); +const LOAD_IG_TIMEOUT_MS = 300000; +const RESERVED = new Set(['login', 'logout', 'admin', 'metadata', 'openapi', 'static', 'assets', 'lookup', 'national-profiles']); + +/** + * Ask the validator service to load a package (id#version) - POST /loadIG with {"ig": ...} as + * the validator's LoadIGHTTPHandler takes it. fhir-validator-wrapper's own loadIG() (to 1.6.0 + * at least) sends the package id and version as query parameters with no body, which the + * service refuses ("Error parsing JSON source: ... Eof at Line 1"), so it is only used for a + * validator with no address of its own (as the tests' are). + * @returns {Promise} the OperationOutcome the service answers with + */ +function loadIG(validator, packageId, version) { + if (!validator.baseUrl) { + return validator.loadIG(packageId, version); + } + const body = JSON.stringify({ ig: `${packageId}#${version}` }); + const url = new URL(`${validator.baseUrl}/loadIG`); + return new Promise((resolve, reject) => { + const req = http.request({ + hostname: url.hostname, port: url.port, path: url.pathname, method: 'POST', + headers: { 'Content-Type': 'application/json', 'Accept': 'application/fhir+json', 'Content-Length': Buffer.byteLength(body) } + }, (res) => { + const chunks = []; + res.on('data', (c) => chunks.push(c)); + res.on('end', () => { + const text = Buffer.concat(chunks).toString('utf8'); + try { + resolve(JSON.parse(text)); + } catch (e) { + reject(new Error(`The validator answered HTTP ${res.statusCode} with a response that is not JSON (${text.length} bytes)`)); + } + }); + res.on('error', reject); + }); + req.on('error', reject); + // loading a package can mean downloading it, and its dependencies + req.setTimeout(LOAD_IG_TIMEOUT_MS, () => req.destroy(new Error('The validator took too long to load the package'))); + req.end(body); + }); +} + +/** + * What's wrong with a national profile's details, or null: a jurisdiction (a country's ISO 3166 + * two-letter code, or a region's UN M49 three-digit code - see jurisdictions.js), a FHIR package id (lower case, at + * least two parts separated by dots), a version (as package versions are written), and the + * canonical URL (http or https) of the IG's profile on Bundle. + */ +function nationalProfileProblem(jurisdiction, packageId, version, url) { + if (!jurisdictionOf(jurisdiction)) { + return 'The jurisdiction must be a country, by its ISO 3166 two-letter code (e.g. AU), or a region, by its UN M49 three-digit code (e.g. 150 for Europe)'; + } + if (packageId.length > 128 || !/^[a-z0-9][a-z0-9_-]*(\.[a-z0-9][a-z0-9_-]*)+$/.test(packageId)) { + return 'The package id must be a FHIR package id: lower case letters, digits, - and _, in two or more parts separated by dots (e.g. hl7.fhir.au.ips)'; + } + if (!/^[0-9A-Za-z][0-9A-Za-z.+-]{0,63}$/.test(version)) { + return 'The version must be a package version (e.g. 1.0.0, or 1.1.0-ballot)'; + } + if (url.length > 1024 || !/^https?:\/\/[^\s/?#]+(?:[/?#]\S*)?$/.test(url)) { + return 'The profile must be the canonical URL (http or https) of the IG\'s profile on Bundle'; + } + return null; +} /** * What's wrong with a submitted document, if anything. @@ -165,25 +229,6 @@ function preferValue(req, name) { return null; } -// the browser scripts: the module's own files, read once and served from memory (so no -// request reaches the file system), with an ETag so a browser can keep its copy -const SCRIPTS = new Map(); -function script(name) { - if (!SCRIPTS.has(name)) { - const body = fs.readFileSync(path.join(__dirname, name), 'utf8'); - SCRIPTS.set(name, { body, etag: `"${crypto.createHash('sha256').update(body).digest('base64url').substring(0, 27)}"` }); - } - const s = SCRIPTS.get(name); - return (req, res) => { - res.setHeader('Cache-Control', 'public, max-age=3600'); - res.setHeader('ETag', s.etag); - if (req.get('If-None-Match') === s.etag) { - return res.status(304).end(); - } - res.type('application/javascript').send(s.body); - }; -} - function wantsHtml(req) { const f = req.query._format; if (typeof f === 'string' && f !== '') { @@ -308,6 +353,7 @@ class IpsModule { this.setupSession(); this.setupRootRoutes(); this.setupProfileRoutes(); + this.setupNationalProfileRoutes(); this.setupProjectRoutes(); this.setupValidator(); this.setupValueSets(); @@ -451,7 +497,7 @@ class IpsModule { next(); }; - r.get('/profile-editor.js', script('profile-editor.js')); + r.get('/profile-editor.js', scripts.handler('profile-editor.js')); r.get('/lookup', this.session, adminJson, ...this.lookups, h('lookup', (req, res) => this.systemLookup(req, res))); r.get('/lookup-vs', this.session, adminJson, ...this.lookups, h('lookup', (req, res) => this.valueSetLookup(req, res))); r.get('/StructureDefinition', ...this.web, h('profiles', (req, res) => this.profileList(req, res))); @@ -460,12 +506,37 @@ class IpsModule { r.get('/StructureDefinition/:id', profile, ...this.web, h('profile', (req, res) => this.profileRead(req, res))); r.get('/StructureDefinition/:id/edit', profile, this.session, this.csrf, admin, h('profile', (req, res) => this.profileEditor(req, res, req.profile))); r.post('/StructureDefinition/:id', profile, this.sameOrigin, this.session, json, this.csrf, adminJson, h('profile', (req, res) => this.profileSave(req, res, req.profile))); + // one constraint at a time, from the profile's page: add, edit (keeping its key), delete + r.post('/StructureDefinition/:id/constraint', profile, this.sameOrigin, this.session, json, this.csrf, adminJson, h('profile', (req, res) => this.constraintSave(req, res, null))); + r.post('/StructureDefinition/:id/constraint/:key', profile, this.sameOrigin, this.session, json, this.csrf, adminJson, h('profile', (req, res) => this.constraintSave(req, res, req.params.key))); + r.post('/StructureDefinition/:id/constraint/:key/delete', profile, this.sameOrigin, this.session, this.form, this.csrf, admin, h('profile', (req, res) => this.constraintDelete(req, res))); r.post('/StructureDefinition/:id/delete', profile, this.sameOrigin, this.session, this.form, this.csrf, admin, h('profile', (req, res) => { this.store.deleteProfile(req.profileId); res.redirect(req.baseUrl + '/StructureDefinition'); })); } + /** + * The national profiles, across all projects: everyone can see them, only the administrator + * adds, edits and deletes them. Before the project routes, so /national-profiles isn't taken + * for a project code. + */ + setupNationalProfileRoutes() { + const r = this.router; + const h = (name, fn) => (req, res) => this.handle(req, res, name, () => fn(req, res)); + const admin = this.adminOnly; + r.get('/national-profiles', ...this.web, h('national-profiles', (req, res) => this.sendNationalProfiles(req, res))); + r.post('/national-profiles', this.sameOrigin, this.session, this.form, this.csrf, admin, h('national-profiles', (req, res) => this.saveNationalProfile(req, res, null))); + r.post('/national-profiles/:id', this.sameOrigin, this.session, this.form, this.csrf, admin, h('national-profiles', (req, res) => this.saveNationalProfile(req, res, req.params.id))); + r.post('/national-profiles/:id/delete', this.sameOrigin, this.session, this.form, this.csrf, admin, h('national-profiles', (req, res) => { + const np = /^\d{1,15}$/.test(req.params.id) ? this.store.nationalProfile(parseInt(req.params.id, 10)) : null; + if (np && this.store.deleteNationalProfile(np.id)) { + this.log.info(`IPS module: national profile ${np.packageId}#${np.version} deleted`); + } + res.redirect(req.baseUrl + '/national-profiles'); + })); + } + setupProjectRoutes() { const r = this.router; const h = (name, fn) => (req, res) => this.handle(req, res, name, () => fn(req, res)); @@ -509,7 +580,7 @@ class IpsModule { next(); }; - r.get('/editor.js', script('editor.js')); + r.get('/editor.js', scripts.handler('editor.js')); r.get('/:code', project, ...this.web, h('list', (req, res) => this.htmlList(req, res))); r.get('/:code/metadata', project, h('metadata', (req, res) => this.metadata(req, res))); // The submission form has no CSRF token and starts no session: submitting is open to @@ -595,15 +666,22 @@ class IpsModule { * wait in the queue. */ setupValidator() { + // the national profiles' packages, as they are loaded into the validator: id#version -> Promise + this.nationalPackages = new Map(); const cfg = this.config.validator; if (!cfg || cfg.enabled === false) { this.log.warn('IPS module: no validator configured (modules.ips.validator) - documents will not be validated'); return; } const FhirValidator = require('fhir-validator-wrapper'); - const jar = folders.filePath('bin', 'validator_cli.jar'); + // a jar of its own (a validator built locally, say): used as it is, never downloaded or + // updated; otherwise the released one in the data folder's bin directory, kept up to date + const localJar = typeof cfg.jar === 'string' && cfg.jar.trim() ? (path.isAbsolute(cfg.jar) ? cfg.jar : folders.filePath(cfg.jar)) : null; + const jar = localJar || folders.filePath('bin', 'validator_cli.jar'); this.validator = new FhirValidator(jar, this.log); const settings = { + autoDownload: !localJar, + skipUpdateCheck: !!localJar, version: cfg.version || '4.0.1', txServer: cfg.txServer || 'http://tx.fhir.org/r4', txLog: cfg.txLog || folders.filePath('logs', 'ips-validator-tx.log'), @@ -611,7 +689,7 @@ class IpsModule { igs: Array.isArray(cfg.packages) ? cfg.packages : ['hl7.fhir.uv.ips#1.1.0'], timeout: cfg.timeout || 120000 }; - this.log.info(`IPS module: starting the FHIR validator on port ${settings.port} (tx ${settings.txServer}; ${settings.igs.join(', ')})`); + this.log.info(`IPS module: starting the FHIR validator${localJar ? ` (${localJar})` : ''} on port ${settings.port} (tx ${settings.txServer}; ${settings.igs.join(', ')})`); this.validatorStarting = this.validator.start(settings).then(() => { this.log.info('IPS module: the FHIR validator is running'); }).catch(e => { @@ -622,6 +700,31 @@ class IpsModule { }); } + /** + * Load a national profile's package into the validator - once: a package that loaded, or + * failed to (with why), is remembered until the national profiles are changed, so each + * document doesn't ask again. The validator's packages (modules.ips.validator.packages) are + * loaded when it starts; these are added to them. + * @returns {Promise} rejects with the reason the package could not be loaded + */ + loadNationalPackage(packageId, version) { + const key = `${packageId}#${version}`; + let p = this.nationalPackages.get(key); + if (!p) { + p = loadIG(this.validator, packageId, version).then(outcome => { + const bad = (Array.isArray(outcome?.issue) ? outcome.issue : []).filter(i => i && (i.severity === 'error' || i.severity === 'fatal')); + if (bad.length > 0 || outcome?.resourceType !== 'OperationOutcome') { + const why = bad.map(i => i.details?.text || i.diagnostics || i.code).filter(Boolean).join('; ') || 'the validator did not return an OperationOutcome'; + throw new Error(`The validator could not load ${key}: ${why}`); + } + this.log.info(`IPS module: national profile package ${key} loaded into the validator`); + }); + p.catch(e => this.log.warn(`IPS module: ${e.message}`)); + this.nationalPackages.set(key, p); + } + return p; + } + /** Read the value sets the terminology check holds documents to: the module's own, in ips/valuesets/. */ setupValueSets() { try { @@ -1295,6 +1398,52 @@ class IpsModule { render.renderSubmit(req.project, this.ctx(req, res), { needsToken: tokenConfigured(req.project.token), aiEnabled: this.store.projectAi(req.project).enabled === true, ...options }), Date.now(), req); } + // ---- national profiles ---- + + /** The national profiles: a page (with the administrator's forms), or JSON. */ + sendNationalProfiles(req, res, message, status = 200) { + const list = this.store.nationalProfiles(); + res.status(status); + if (!wantsHtml(req) && !message) { + return res.json(list.map(np => ({ jurisdiction: np.jurisdiction, packageId: np.packageId, version: np.version, url: np.url }))); + } + this.sendHtml(res, 'National Profiles', render.renderNationalProfiles(list, this.ctx(req, res), message), Date.now(), req); + } + + /** Add (id null) or change a national profile, from the form: packageId, version, url. */ + saveNationalProfile(req, res, id) { + const field = (name) => (typeof req.body[name] === 'string' ? req.body[name].trim() : ''); + const packageId = field('packageId'); + const version = field('version'); + const url = field('url'); + // a country code is upper case; a region's is digits + const jurisdiction = field('jurisdiction').toUpperCase(); + const problem = nationalProfileProblem(jurisdiction, packageId, version, url); + if (problem) { + return this.sendNationalProfiles(req, res, problem, 400); + } + const existing = this.store.nationalProfileByPackage(packageId, version); + if (id === null) { + if (existing) { + return this.sendNationalProfiles(req, res, `There is already a national profile for ${packageId}#${version}`, 409); + } + this.store.createNationalProfile(packageId, version, url, jurisdiction); + this.log.info(`IPS module: national profile ${packageId}#${version} (${jurisdiction}) added (${url})`); + } else { + const n = /^\d{1,15}$/.test(id) ? parseInt(id, 10) : -1; + if (existing && existing.id !== n) { + return this.sendNationalProfiles(req, res, `There is already a national profile for ${packageId}#${version}`, 409); + } + if (!this.store.updateNationalProfile(n, packageId, version, url, jurisdiction)) { + return this.sendNationalProfiles(req, res, 'That national profile no longer exists', 404); + } + this.log.info(`IPS module: national profile ${packageId}#${version} (${jurisdiction}) saved (${url})`); + } + // a package that failed to load is tried again + this.nationalPackages = new Map(); + res.redirect(req.baseUrl + '/national-profiles'); + } + // ---- clinical profiles ---- /** The clinical profiles: a page, or a FHIR searchset of the StructureDefinitions. */ @@ -1321,7 +1470,31 @@ class IpsModule { res.type('application/fhir+json'); return res.send(JSON.stringify(sd)); } - this.sendHtml(res, `Clinical Profile: ${sd.title}`, render.renderProfile(sd, { ...this.ctx(req, res), valueSetTitle: (url) => this.valueSetTitle(url) }), Date.now(), req); + const ctx = { ...this.ctx(req, res), valueSetTitle: (url) => this.valueSetTitle(url), profileTitle: (url) => this.profileTitle(url) }; + // the administrator edits the rows in place, a row at a time + const editor = ctx.user?.isAdmin ? { + constraints: this.editorConstraints(sd), + sections: profiles.IPS_SECTIONS, systems: profiles.SYSTEMS, resourceTypes: profiles.RESOURCE_TYPES, sectionSystem: editorSchema.LOINC, canLookup: !!this.tx + } : null; + this.sendHtml(res, `Clinical Profile: ${sd.title}`, render.renderProfile(sd, ctx, editor), Date.now(), req); + } + + /** + * What to call a profile a rule says resources conform to: one of the clinical profiles (by + * its title), or one of the national profiles (by its jurisdiction and package), or null. + */ + profileTitle(url) { + if (typeof url !== 'string') { + return null; + } + const plain = url.split('|')[0]; + const m = /\/StructureDefinition\/profile-([a-f0-9]{12})$/.exec(plain); + const sd = m ? this.store.profile(m[1]) : null; + if (sd && sd.url === plain) { + return sd.title; + } + const np = this.store.nationalProfiles().find(p => p.url === plain); + return np ? `${np.jurisdiction ? `${np.jurisdiction} ` : ''}national profile (${np.packageId}#${np.version})` : null; } /** The title of one of this module's own value sets, or null. */ @@ -1331,8 +1504,12 @@ class IpsModule { } /** The editor, for a new profile or an existing one. */ + editorConstraints(sd) { + return profiles.constraintsOf(sd).map(c => ({ ...c, shape: profiles.parseShape(c.expression), rshape: profiles.parseResourceShape(c.expression) })); + } + profileEditor(req, res, sd) { - const constraints = profiles.constraintsOf(sd).map(c => ({ ...c, shape: profiles.parseShape(c.expression), rshape: profiles.parseResourceShape(c.expression) })); + const constraints = this.editorConstraints(sd); this.sendHtml(res, sd ? `Edit Clinical Profile: ${sd.title}` : 'New Clinical Profile', render.renderProfileEditor(sd, this.ctx(req, res), { profile: { id: sd ? sd.id : null, title: sd ? sd.title : '', constraints }, @@ -1369,6 +1546,53 @@ class IpsModule { res.status(previous ? 200 : 201).json({ id: sd.id, url: `${req.baseUrl}/StructureDefinition/${sd.id}` }); } + /** + * Add a constraint to a profile ({human, expression}), or replace one (by its key, which it + * keeps). 201/200 with {key}, 404 when there is no constraint with that key, 422 with {problems}. + */ + constraintSave(req, res, key) { + const previous = req.profile; + const body = req.body && typeof req.body === 'object' && !Array.isArray(req.body) ? req.body : {}; + const row = { key, human: body.human, expression: body.expression }; + const constraints = profiles.constraintsOf(previous); + const at = key === null ? constraints.length : constraints.findIndex(c => c.key === key); + if (at < 0) { + return res.status(404).json({ problems: [`The profile has no constraint '${key}'`] }); + } + // the row by itself first, so what is wrong is said of it, not of a number in the list + const alone = profiles.checkInput({ title: previous.title, constraints: [{ human: row.human, expression: row.expression }] }, null); + if (alone.problems.length) { + return res.status(422).json({ problems: alone.problems.map(p => p.replace(/^Constraint 1/, 'The rule')) }); + } + constraints.splice(at, key === null ? 0 : 1, row); + const checked = profiles.checkInput({ title: previous.title, constraints }, previous); + if (checked.problems.length) { + return res.status(422).json({ problems: checked.problems }); + } + this.saveConstraints(previous, checked.constraints); + this.log.info(`IPS module: clinical profile ${previous.id}: constraint ${checked.constraints[at].key} ${key === null ? 'added' : 'saved'}`); + res.status(key === null ? 201 : 200).json({ key: checked.constraints[at].key }); + } + + /** Delete one of a profile's constraints, back to the profile's page. */ + constraintDelete(req, res) { + const previous = req.profile; + const constraints = profiles.constraintsOf(previous); + const at = constraints.findIndex(c => c.key === req.params.key); + if (at < 0) { + return htmlServer.sendErrorResponse(res, TEMPLATE, new Error(`The profile has no constraint '${req.params.key}'`), 404); + } + constraints.splice(at, 1); + this.saveConstraints(previous, constraints); + this.log.info(`IPS module: clinical profile ${previous.id}: constraint ${req.params.key} deleted`); + res.redirect(`${req.baseUrl}/StructureDefinition/${previous.id}`); + } + + saveConstraints(previous, constraints) { + const id = previous.id.substring('profile-'.length); + this.store.saveProfile(id, profiles.build({ id, url: previous.url, title: previous.title, constraints })); + } + // ---- the editor ---- sendCompose(req, res) { @@ -1597,6 +1821,7 @@ class IpsModule { ...this.ctx(req, res), steps: this.steps.map(st => st.name), level: typeof req.query.level === 'string' ? req.query.level : undefined, + np: typeof req.query.np === 'string' ? req.query.np : undefined, problemsOnly: req.query.problems === '1' }; return this.sendHtml(res, `${req.project.name}: Document ${row.id}`, render.renderDocument(req.project, row, view, ctx), start, req); diff --git a/ips/jurisdictions.js b/ips/jurisdictions.js new file mode 100644 index 00000000..b5d44817 --- /dev/null +++ b/ips/jurisdictions.js @@ -0,0 +1,65 @@ +// +// Copyright 2026, Health Intersections Pty Ltd (http://www.healthintersections.com.au) +// +// Licensed under BSD-3: https://opensource.org/license/bsd-3-clause +// + +/** + * Jurisdictions, as http://hl7.org/fhir/ValueSet/jurisdiction has them: a country, by its + * ISO 3166-1 two-letter code (urn:iso:std:iso:3166), or a region, by its UN M49 three-digit + * code (http://unstats.un.org/unsd/methods/m49/m49.htm). The codes and names are the ones + * the terminology module's providers hold (tx/cs/cs-country.js, tx/cs/cs-areacode.js). + * + * @module ips/jurisdictions + */ + +const { CountryCodeFactoryProvider } = require('../tx/cs/cs-country'); +const { AreaCodeFactoryProvider } = require('../tx/cs/cs-areacode'); + +const ISO_3166 = 'urn:iso:std:iso:3166'; +const M49 = 'http://unstats.un.org/unsd/methods/m49/m49.htm'; +// in the country list, but not countries +const UNKNOWN = new Set(['XX', 'ZZ']); + +let tables = null; + +/** + * The providers' code lists, read once. Only their data is wanted, not code system + * providers, so their load() runs on an object with no i18n (it only fills codes/codeMap). + */ +function load() { + if (!tables) { + const countries = Object.create(CountryCodeFactoryProvider.prototype); + const areas = Object.create(AreaCodeFactoryProvider.prototype); + // load() is async in form, but does its work synchronously + countries.load(); + areas.load(); + tables = { countries: countries.codeMap, areas: areas.codeMap }; + } + return tables; +} + +/** + * A jurisdiction by its code: two upper case letters (a country), or three digits (a + * region - an M49 region, not a country's numeric code). + * @param {string} code + * @returns {{system: string, code: string, display: string}|null} null if it is neither + */ +function jurisdiction(code) { + if (typeof code !== 'string') { + return null; + } + const t = load(); + if (/^[A-Z]{2}$/.test(code)) { + const c = t.countries.get(code); + // not the user-assigned codes (XK, QM-QZ...), nor the 'unknown' ones (XX, ZZ): they aren't countries ISO 3166 names + return c && !c.userDefined && !UNKNOWN.has(code) ? { system: ISO_3166, code, display: c.display } : null; + } + if (/^\d{3}$/.test(code)) { + const a = t.areas.get(code); + return a && a.class_ === 'region' ? { system: M49, code, display: a.display } : null; + } + return null; +} + +module.exports = { jurisdiction, ISO_3166, M49 }; diff --git a/ips/pipeline.js b/ips/pipeline.js index e7f0c07a..041a7b8b 100644 --- a/ips/pipeline.js +++ b/ips/pipeline.js @@ -12,10 +12,11 @@ * 1. translate translate CodeableConcepts that fail the terminology check to codes that pass it * 2. narrative generate the narrative with the IPS presenter (presenter.js), into the analysed copy * 3. validation validate the (translated) document with the FHIR validator (fhir-validator-wrapper) - * 4. terminology check each CodeableConcept against the value sets for international exchange - * 5. piqi measure the quality of the document with PIQI (not yet) - * 6. profiles check the document against each clinical profile (profiles.js) - * 7. ai assess the document for consistency and its terminology with an AI service (ai-client.js) + * 4. national validate it against each national profile (the national IGs' profiles on Bundle) + * 5. terminology check each CodeableConcept against the value sets for international exchange + * 6. piqi measure the quality of the document with PIQI (not yet) + * 7. profiles check the document against each clinical profile (profiles.js) + * 8. ai assess the document for consistency and its terminology with an AI service (ai-client.js) * * Each step is { name, run(row, project, module) -> result }. A step keeps what it finds in * one of two places: as a result stored against the document (store.saveResult - the @@ -122,6 +123,66 @@ const validation = { } }; +/** + * National profiles: the document is validated against each national profile - the + * national IG's package is loaded into the validator first (once), then the document is + * validated against the IG's profile on Bundle, as the validation step validates it. For + * each profile, the result keeps the OperationOutcome and whether the document passed: no + * errors (or fatal issues). A profile whose package can't be loaded, or that the validator + * can't validate against, is kept with the reason (pass is then null), and doesn't stop the + * pipeline. + */ +const national = { + name: 'national', + async run(row, project, module) { + const list = module.store.nationalProfiles(); + const started = Date.now(); + if (list.length === 0) { + module.store.saveResult(row.id, 'national', { ms: 0, profiles: [] }); + return 'national profiles: none'; + } + const v = module.validator; + if (!v) { + throw new Error('The validator is not configured (modules.ips.validator)'); + } + if (!v.isRunning()) { + throw new Error('The validator is not running'); + } + const cfg = module.config.validator || {}; + const checkIpsCodes = cfg.checkIpsCodes !== false; + const doc = JSON.stringify(row.analysed || row.bundle); + const profiles = []; + for (const np of list) { + const entry = { id: np.id, jurisdiction: np.jurisdiction, packageId: np.packageId, version: np.version, url: np.url }; + const t0 = Date.now(); + try { + await module.loadNationalPackage(np.packageId, np.version); + const outcome = await v.validate(doc, { profiles: [np.url], checkIpsCodes, timeout: VALIDATE_TIMEOUT_MS }); + if (!outcome || outcome.resourceType !== 'OperationOutcome') { + throw new Error('The validator did not return an OperationOutcome'); + } + entry.counts = issueCounts(outcome); + entry.pass = !entry.counts.error && !entry.counts.fatal; + entry.outcome = outcome; + } catch (e) { + entry.pass = null; + entry.error = e.message; + } + entry.ms = Date.now() - t0; + profiles.push(entry); + } + module.store.saveResult(row.id, 'national', { + ms: Date.now() - started, + validatorVersion: typeof v.jarVersion === 'function' ? v.jarVersion() : null, + profiles + }); + const passed = profiles.filter(p => p.pass === true).length; + const failed = profiles.filter(p => p.pass === false).length; + const unchecked = profiles.length - passed - failed; + return `national profiles: ${passed} passed, ${failed} failed` + (unchecked ? `, ${unchecked} not checked` : ''); + } +}; + // ---- the terminology check ------------------------------------------------------------ /** @@ -1003,10 +1064,10 @@ const aiStep = { }; /** The steps, in order. Only the ones that exist yet. */ -const STEPS = [translate, narrative, validation, terminology, profilesStep, aiStep]; +const STEPS = [translate, narrative, validation, national, terminology, profilesStep, aiStep]; module.exports = { - STEPS, translate, narrative, validation, terminology, profiles: profilesStep, ai: aiStep, checkConstraint, valueSetsIn, lookupDisplay, checkConcept, documentForAi, assessmentOf, aiSystemPrompt, AI_SYSTEM_PROMPT, checkOf, addCheck, stripChecks, translateConcept, translationOf, + STEPS, translate, narrative, validation, national, terminology, profiles: profilesStep, ai: aiStep, checkConstraint, valueSetsIn, lookupDisplay, checkConcept, documentForAi, assessmentOf, aiSystemPrompt, AI_SYSTEM_PROMPT, checkOf, addCheck, stripChecks, translateConcept, translationOf, validateCodeInValueSet, validateCodeInSystem, issueCounts, issueSummary, IPS_BUNDLE_PROFILE, CHECK_EXTENSION, TRANSLATION_EXTENSION, MAX_CHECKS_PER_DOCUMENT, MAX_CACHED_ANSWERS }; diff --git a/ips/profile-editor.js b/ips/profile-editor.js index 76ee52fd..547149f3 100644 --- a/ips/profile-editor.js +++ b/ips/profile-editor.js @@ -14,6 +14,8 @@ * the rest: a row of the second - type, system, code or value set, the rest), and * other rules (FHIRPath). A row's expression is made from it. Saving posts the title and the * constraints (key, description, expression); the server checks them, and keeps the keys. + * With cfg.mode 'rows' it is the profile's own page instead: the rows are the page's tables, + * and one at a time is added or edited, in a dialog, and saved by itself. */ (function () { 'use strict'; @@ -22,6 +24,8 @@ var LOINC = cfg.sectionSystem; var dirty = false; var saving = false; + // what to draw again when a row's fields change shape: the whole editor, or the row being edited + var redraw = function () { draw(); }; function el(tag, attrs, children) { var e = document.createElement(tag); @@ -359,11 +363,13 @@ url.hidden = select.value !== '*'; url.addEventListener('input', function () { r.entSystem = url.value.trim(); changed(); onChange(); }); select.addEventListener('change', function () { - var wasVs = r.entSystem === VS; + var was = r.entSystem; + var wasVs = was === VS; r.entSystem = select.value === '*' ? url.value.trim() : select.value; if (!select.value) { r.entCode = ''; r.entCard = ''; r.entVs = ''; } changed(); - if (wasVs !== (r.entSystem === VS) || !select.value || select.value === '*') { draw(); } else { onChange(); } + // the code (or value set) input comes and goes with the choice + if (!was || wasVs !== (r.entSystem === VS) || !select.value || select.value === '*') { redraw(); } else { onChange(); } }); cell.appendChild(el('div', { class: 'ips-pf-pair' }, [select, url])); if (r.entSystem === VS) { @@ -414,7 +420,7 @@ r.resSystem = select.value === '*' ? url.value.trim() : select.value; changed(); // the code column becomes a value set's, or a code's - if (wasVs !== (r.resSystem === VS)) { draw(); } else { onChange(); } + if (wasVs !== (r.resSystem === VS)) { redraw(); } else { onChange(); } }); cell.appendChild(select); cell.appendChild(url); @@ -573,6 +579,279 @@ .catch(function (e) { saving = false; show(['The profile was not saved: ' + e.message]); }); } + // ---- the profile's page: a rule at a time, added or edited in a dialog ---- + + // what must be true of the resources a rule is about: the rest of the rule, by kind + var NAME_RE = /^[A-Za-z][A-Za-z0-9_]*(\.[A-Za-z][A-Za-z0-9_]*)?$/; + var RULE_KINDS = [ + ['present', 'Must be present somewhere'], + ['subject', 'Patient must match the Document'], + ['element', 'Must have an element'], + ['conforms', 'Conform to a profile'], + ['category', 'Has a category'], + ['other', 'Something else (FHIRPath)'] + ]; + function allOf(body) { return new RegExp('^\\.\\s*(?:' + body + '|all\\s*\\(\\s*' + body + '\\s*\\))$'); } + var CONFORMS_RE = allOf('conformsTo\\s*\\(\\s*' + STR + '\\s*\\)'); + var CATEGORY_RE = allOf('category\\s*\\.\\s*hasCoding\\s*\\(\\s*' + STR + '\\s*,\\s*' + STR + '\\s*(?:,\\s*' + STR + '\\s*)?\\)'); + + /** the rule's kind, and its fields, from the rest of a rule about resources */ + function ruleFields(r) { + var rest = (r.resRest || '').trim(); + var m; + r.ruleKind = 'other'; + r.ruleOther = rest; + r.ruleUrl = ''; r.ruleSystem = cfg.categorySystem || ''; r.ruleCode = ''; r.ruleVersion = ''; r.rulePath = ''; r.ruleDisplay = ''; + r.ruleAll = true; + if (/^\.\s*exists\s*\(\s*\)$/.test(rest)) { + r.ruleKind = 'present'; + } else if (/^\.\s*all\s*\(\s*sameSubject\s*\(\s*\)\s*\)$/.test(rest)) { + r.ruleKind = 'subject'; + } else if ((m = CONFORMS_RE.exec(rest))) { + r.ruleKind = 'conforms'; + r.ruleAll = m[1] === undefined; + r.ruleUrl = unquote(m[1] !== undefined ? m[1] : m[2]); + } else if ((m = CATEGORY_RE.exec(rest))) { + r.ruleKind = 'category'; + r.ruleAll = m[1] === undefined; + var g = m[1] !== undefined ? [m[1], m[2], m[3]] : [m[4], m[5], m[6]]; + r.ruleSystem = unquote(g[0]); r.ruleCode = unquote(g[1]); r.ruleVersion = g[2] !== undefined ? unquote(g[2]) : ''; + } else if ((m = /^\.\s*([A-Za-z][A-Za-z0-9_]*(?:\s*\.\s*[A-Za-z][A-Za-z0-9_]*)?)\s*\.\s*exists\s*\(\s*\)$/.exec(rest))) { + r.ruleKind = 'element'; + r.rulePath = m[1].replace(/\s+/g, ''); + } + } + /** the rest of a rule about resources, from its kind and fields, or null with what is wrong */ + function ruleRest(r) { + var each = function (body) { return r.ruleAll ? '.all(' + body + ')' : '.' + body; }; + switch (r.ruleKind) { + case 'present': return { rest: '.exists()', problems: [] }; + case 'subject': return { rest: '.all(sameSubject())', problems: [] }; + case 'element': return NAME_RE.test(r.rulePath.trim()) ? { rest: '.' + r.rulePath.trim() + '.exists()', problems: [] } : { rest: null, problems: ['rulePath'] }; + case 'conforms': return CANONICAL_RE.test(r.ruleUrl.trim()) ? { rest: each('conformsTo(' + quote(r.ruleUrl.trim()) + ')'), problems: [] } : { rest: null, problems: ['ruleUrl'] }; + case 'category': { + var problems = []; + if (!r.ruleSystem.trim()) { problems.push('ruleSystem'); } + if (!r.ruleCode.trim()) { problems.push('ruleCode'); } + if (problems.length) { return { rest: null, problems: problems }; } + return { rest: each('category.hasCoding(' + quote(r.ruleSystem.trim()) + ', ' + quote(r.ruleCode.trim()) + (r.ruleVersion.trim() ? ', ' + quote(r.ruleVersion.trim()) : '') + ')'), problems: [] }; + } + default: { + var rest = (r.ruleOther || '').trim(); + return rest && rest !== '.' ? { rest: rest.charAt(0) === '.' ? rest : '.' + rest, problems: [] } : { rest: null, problems: ['ruleOther'] }; + } + } + } + /** the expression of a row in the dialog, or null with what is wrong */ + function dialogExpression(r) { + if (r.mode !== 'resource') { return rowExpression(r); } + var t = ruleRest(r); + r.resRest = t.rest || ''; + var e = resourceExpression(r); + return { expression: t.rest ? e.expression : null, problems: t.problems.concat(e.problems.filter(function (p) { return p !== 'resRest'; })) }; + } + + function rowsMode() { + var MODE = { section: 'section', resource: 'resource', other: 'text' }; + var TITLES = { section: 'a required section or entry', resource: 'a rule about resources', text: 'a rule' }; + var byKey = {}; + (cfg.profile.constraints || []).forEach(function (c) { byKey[c.key] = c; }); + var dialog = el('dialog', { class: 'ips-pf-dialog', 'aria-labelledby': 'ips-pf-dialog-title' }); + document.body.appendChild(dialog); + var editing = null; + + function showIn(box, problems, kind) { + box.innerHTML = ''; + if (!problems.length) { return; } + var alert = el('div', { class: 'alert alert-' + (kind || 'danger'), style: 'margin: 0' }); + if (problems.length === 1) { alert.textContent = problems[0]; } else { + alert.appendChild(el('ul', { style: 'margin: 0' }, problems.map(function (p) { return el('li', { text: p }); }))); + } + box.appendChild(alert); + } + // a labelled part of the dialog: a label, what goes in it, and a hint + function field(label, controls, hint) { + return el('div', { class: 'ips-pf-field' }, [el('label', { class: 'ips-pf-label', text: label })].concat(controls).concat(hint ? [el('div', { class: 'ips-pf-hint', text: hint })] : [])); + } + function howMany(r, fieldName, update, fields) { + var input = cardInput(r, fieldName, update); + fields[fieldName] = input; + return el('div', { class: 'ips-pf-pair ips-pf-how' }, [el('span', { text: 'How many' }), input, + el('span', { class: 'ips-pf-hint', text: 'blank: at least one; or min..max - 1..1, 0..1, 1..*' })]); + } + // a code system for the category: one of the named ones, or another by URL + function categorySystem(r, update) { + var cell = el('div', { class: 'ips-pf-pair' }); + var select = el('select', { class: 'form-control ips-ed-input' }); + cfg.systems.forEach(function (s) { select.appendChild(el('option', { value: s.url, text: s.name })); }); + (cfg.categorySystems || []).forEach(function (s) { select.appendChild(el('option', { value: s.url, text: s.name })); }); + select.appendChild(el('option', { value: '*', text: 'Another system...' })); + var known = Array.prototype.some.call(select.options, function (o) { return o.value === r.ruleSystem; }); + select.value = known ? r.ruleSystem : '*'; + var url = el('input', { type: 'text', class: 'form-control ips-ed-input', placeholder: 'code system URL' }); + url.value = known ? '' : r.ruleSystem; + url.hidden = known; + url.addEventListener('input', function () { r.ruleSystem = url.value.trim(); changed(); update(); }); + select.addEventListener('change', function () { + url.hidden = select.value !== '*'; + r.ruleSystem = select.value === '*' ? url.value.trim() : select.value; + changed(); + update(); + }); + cell.appendChild(select); + cell.appendChild(url); + cell.url = url; + return cell; + } + + /** the dialog's contents, (re)drawn for the row being edited */ + function drawDialog() { + var e = editing; + var r = e.r; + dialog.innerHTML = ''; + var fields = {}; + var exprLine = el('div', { class: 'ips-pf-expr' }); + var update = function () { + var t = dialogExpression(r); + exprLine.textContent = r.mode === 'text' ? '' : (t.expression || '(incomplete - the fields in red)'); + Object.keys(fields).forEach(function (k) { fields[k].classList.toggle('ips-pf-bad', t.problems.indexOf(k) >= 0); }); + }; + var body = el('div', { class: 'ips-pf-dialog-body' }); + var human = el('textarea', { class: 'form-control', rows: 2, placeholder: 'e.g. A problem list, with the patient\'s hypertension' }); + human.value = r.human; + human.addEventListener('input', function () { r.human = human.value; changed(); }); + body.appendChild(field('Expectation', [human], 'What is expected, as people will read it.')); + + if (r.mode === 'section') { + body.appendChild(field('Section', [sectionCell(r, update), howMany(r, 'secCard', update, fields)])); + var ent = entryCell(r, update); + if (ent.code) { fields.entCode = ent.code.input; } + if (ent.vs) { fields.entVs = ent.vs.input; } + body.appendChild(field('Entries', [ent].concat(r.entSystem ? [howMany(r, 'entCard', update, fields)] : []), + 'Optionally, entries the section must have: with a code in a code system, or a code in a value set.')); + } else if (r.mode === 'resource') { + var type = typeCell(r, update); + var system = systemCell(r, update); + var byVs = r.resSystem === VS; + var code = byVs ? vsInput(r, 'resVs', 'resVsTitle', update) : codeInput(r, 'resCode', 'resDisplay', function () { return r.resSystem; }, update); + fields.resType = type.other; + fields.resSystem = system.url; + fields[byVs ? 'resVs' : 'resCode'] = code.input; + body.appendChild(field('Resources', [type], 'The type of resource, or any.')); + body.appendChild(field('With a code', [system, code], 'Anywhere in the document, whatever section they are in.')); + var kind = el('select', { class: 'form-control ips-ed-input' }); + RULE_KINDS.forEach(function (k) { kind.appendChild(el('option', { value: k[0], text: k[1] })); }); + kind.value = r.ruleKind; + kind.addEventListener('change', function () { r.ruleKind = kind.value; changed(); drawDialog(); }); + var more = []; + if (r.ruleKind === 'element') { + var path = el('input', { type: 'text', class: 'form-control ips-ed-input', placeholder: 'e.g. onset, or code.text' }); + path.value = r.rulePath; + path.addEventListener('input', function () { r.rulePath = path.value; changed(); update(); }); + fields.rulePath = path; + more.push(field('Element', [path], 'An element of the resources, or an element of one of their elements (one dot, no deeper).')); + } else if (r.ruleKind === 'conforms') { + var url = el('input', { type: 'text', class: 'form-control ips-ed-input', placeholder: 'the profile\'s canonical URL' }); + url.value = r.ruleUrl; + url.addEventListener('input', function () { r.ruleUrl = url.value; changed(); update(); }); + fields.ruleUrl = url; + more.push(field('Profile', [url], 'Each of the resources must conform to it. It can be one of the clinical profiles, or a profile the validator has loaded.')); + } else if (r.ruleKind === 'category') { + var csys = categorySystem(r, update); + var ccode = codeInput(r, 'ruleCode', 'ruleDisplay', function () { return r.ruleSystem; }, update); + var version = el('input', { type: 'text', class: 'form-control ips-ed-input', placeholder: 'version (optional)' }); + version.value = r.ruleVersion; + version.addEventListener('input', function () { r.ruleVersion = version.value; changed(); update(); }); + fields.ruleSystem = csys.url; + fields.ruleCode = ccode.input; + more.push(field('Category', [csys, ccode, version], 'Each of the resources must have a category with this code.')); + } else if (r.ruleKind === 'other') { + var other = el('textarea', { class: 'form-control ips-ed-input', rows: 3, placeholder: '.count() = 1', style: 'font-family: monospace' }); + other.value = r.ruleOther; + other.addEventListener('input', function () { r.ruleOther = other.value; changed(); update(); }); + fields.ruleOther = other; + more.push(field('FHIRPath', [other], 'FHIRPath on the resources found, starting with a dot.')); + } + body.appendChild(field('What must be true', [kind].concat(more))); + } else { + var text = el('textarea', { class: 'form-control ips-ed-input', rows: 5, placeholder: 'A FHIRPath expression on the document (the Bundle), true when it meets the expectation', style: 'font-family: monospace' }); + text.value = r.expression; + text.addEventListener('input', function () { r.expression = text.value; changed(); }); + body.appendChild(field('FHIRPath', [text], 'Evaluated on the document (the Bundle): the rule is met when it is true.')); + } + if (r.mode !== 'text') { body.appendChild(field('As FHIRPath', [exprLine])); } + body.appendChild(e.msg); + + var save = el('button', { type: 'button', class: 'btn btn-primary', text: 'Save', onclick: function () { saveRow(); } }); + var cancel = el('button', { type: 'button', class: 'btn btn-default', text: 'Cancel', onclick: function () { stop(false); } }); + e.save = save; + dialog.appendChild(el('div', { class: 'ips-pf-dialog-head' }, [ + el('h4', { id: 'ips-pf-dialog-title', text: (r.key ? 'Edit ' : 'Add ') + TITLES[r.mode] }), + el('button', { type: 'button', class: 'ips-pf-dialog-x', 'aria-label': 'Close', text: '×', onclick: function () { stop(false); } }) + ])); + dialog.appendChild(body); + dialog.appendChild(el('div', { class: 'ips-pf-dialog-foot' }, [cancel, ' ', save])); + update(); + if (!e.focused) { e.focused = true; human.focus(); } + } + + function start(r) { + if (editing) { return; } + if (r.mode === 'resource') { ruleFields(r); } + editing = { r: r, msg: el('div', { class: 'ips-pf-messages' }) }; + dirty = false; + drawDialog(); + dialog.showModal(); + } + // close the dialog: when there are changes, only if the user says so (unless saved) + function stop(saved) { + if (!editing) { return; } + if (!saved && dirty && !window.confirm('Discard your changes to this rule?')) { return; } + editing = null; + dirty = false; + dialog.close(); + } + dialog.addEventListener('cancel', function (ev) { ev.preventDefault(); stop(false); }); + + function saveRow() { + var e = editing; + if (!e || saving) { return; } + var t = dialogExpression(e.r); + if (!t.expression && e.r.mode !== 'text') { showIn(e.msg, ['The rule is incomplete (the fields in red)']); return; } + saving = true; + e.save.disabled = true; + showIn(e.msg, ['Saving...'], 'info'); + var url = cfg.urls.constraint + (e.r.key ? '/' + encodeURIComponent(e.r.key) : ''); + fetch(url, { method: 'POST', credentials: 'same-origin', headers: { 'Content-Type': 'application/json', Accept: 'application/json' }, + body: JSON.stringify({ human: e.r.human, expression: t.expression || '', _csrf: cfg.csrf }) }) + .then(function (res) { + return res.json().then(function (j) { return { ok: res.ok, j: j }; }, function () { return { ok: false, j: { problems: ['The server answered ' + res.status + (res.status === 403 ? ' (has your session expired? reload the page)' : '')] } }; }); + }) + .then(function (a) { + saving = false; + if (a.ok) { stop(true); location.reload(); return; } + e.save.disabled = false; + showIn(e.msg, a.j.problems || ['The rule was not saved']); + }) + .catch(function (x) { saving = false; e.save.disabled = false; showIn(e.msg, ['The rule was not saved: ' + x.message]); }); + } + + redraw = drawDialog; + Array.prototype.forEach.call(document.querySelectorAll('.ips-pf-edit'), function (b) { + b.addEventListener('click', function () { + var c = byKey[b.closest('tr').getAttribute('data-key')]; + if (c) { start(rowFrom(c)); } + }); + }); + Array.prototype.forEach.call(document.querySelectorAll('.ips-pf-add'), function (b) { + b.addEventListener('click', function () { + var r = rowFrom({}); + r.mode = MODE[b.getAttribute('data-part')]; + start(r); + }); + }); + } + window.addEventListener('beforeunload', function (e) { if (dirty) { e.preventDefault(); e.returnValue = ''; } }); - draw(); + if (cfg.mode === 'rows') { rowsMode(); } else { draw(); } })(); diff --git a/ips/readme.md b/ips/readme.md index 5ac3c71b..364c73fe 100644 --- a/ips/readme.md +++ b/ips/readme.md @@ -37,7 +37,9 @@ network. Without it there are no links and no analysis. `/ips` lists the projects. The administrator logs in at `/ips/login` and, at `/ips/admin/projects` and `/ips/admin/users`, creates and deletes projects and users, and -says which users manage which projects. A project has a code - 2-32 lower case letters, +says which users manage which projects. The administrator also looks after the +[clinical profiles](#clinical-profiles) and the [national profiles](#national-profiles), +which apply across all projects. A project has a code - 2-32 lower case letters, digits and hyphens, fixed once it exists - and lives at `/ips/{code}`. Deleting a project deletes its documents. @@ -156,9 +158,9 @@ it (`ips/pipeline.js`), and marks it `processed` with the steps' notes, or `erro message of the step that failed. Each step's own state (`waiting`, `running`, `done` or `failed`, with its note or error) is kept in its result row as it goes, which is what the document page's step line shows; processing again clears every step's result first. The pipeline, in order, will be: **translate** CodeableConcepts -to the international terminologies; generate the **narrative**; **validate** the document; -measure its quality with PIQI; check it against the **clinical profiles**; **assess** it with AI. Translation, the narrative, -validation, the **terminology check**, the clinical profiles check and the AI assessment exist so far (PIQI does not). A step keeps what it finds in one of two places: as a result stored against the +to the international terminologies; generate the **narrative**; **validate** the document; validate it against the +**national profiles**; measure its quality with PIQI; check it against the **clinical profiles**; **assess** it with AI. Translation, the narrative, +validation, the national profiles, the **terminology check**, the clinical profiles check and the AI assessment exist so far (PIQI does not). A step keeps what it finds in one of two places: as a result stored against the document in `document_results` (one row per step - the validation outcome), or in the **analysed copy** - a second copy of the document (`documents.analysed_json`) that the steps adorn with extensions, so that the master document is never changed. The analysed copy is @@ -236,7 +238,12 @@ view shows it. `modules.ips.validator` configures a FHIR validator service, exactly as the SHL module's does (`fhir-validator-wrapper`: the validator CLI jar in the data folder's `bin` directory, run as a service on its own `port`, with `txServer` - tx.fhir.org, by default - and the IPS -package loaded). It starts in the background when the module does, and the queue waits for +package loaded). `jar` names a validator jar of its own instead - a validator built locally from +org.hl7.fhir.core, say: absolute, or relative to the data folder (`bin/local/validator_cli.jar`) - which +is used as it is, never downloaded or updated. Keep such a jar out of `bin/validator_cli.jar`: the +wrapper takes that jar's version from the `.version` file it wrote when it downloaded the release +there, so a different jar in its place is started as if it were that release (and is replaced by +the next one). It starts in the background when the module does, and the queue waits for it. Each document - its analysed copy, so the translations are seen (the validator will remark on the translation extension until it is given its definition) - is validated against the IPS Bundle profile (`http://hl7.org/fhir/uv/ips/StructureDefinition/Bundle-uv-ips`; set `profiles` to validate @@ -246,6 +253,23 @@ its `validation` view: the counts, each issue's severity, code, location and mes the raw OperationOutcome. Without a configured, running validator the step fails, and the document's status is `error`. +### National profile validation + +The national step runs after validation. It validates the document (the analysed copy, as validation does) against each +[national profile](#national-profiles) in turn: the national IG's package is loaded into the validator (once - a package +that has loaded, or has failed to, is remembered until the national profiles are changed), then the document is +validated against the IG's profile on Bundle, with the same `checkIpsCodes` setting. For each profile the result keeps +the OperationOutcome, its counts, and whether the document passed - it passes when the validator reports no errors (or +fatal issues). A profile whose package can't be loaded, or that the validator can't validate against, is kept with the +reason and counts as not checked; it doesn't stop the pipeline. With no national profiles there is nothing to do, and no +validator is needed; with some, the step fails, as validation does, without a configured, running validator. + +The document's **National Profiles** view lists the profiles as they were when the document was processed, with the +outcome (passed, failed, or not checked) and the issue counts for each; a profile's page shows its validation as the +validation view shows the base validation (`?view=national&np={n}`). The packages loaded for national profiles stay +loaded in the validator alongside its own (`packages`), so two national IGs that define the same things differently can +affect each other. + ### Terminology check After validation, every CodeableConcept the terminology view lists is checked for its @@ -403,12 +427,31 @@ The editor (`/ips/StructureDefinition/new`, or Edit on a profile) is in three pa - **Rules about resources:** the third shape, each a row of a table with four columns (the type, which is optional, the system, the code, and the rest of the rule as FHIRPath starting with a dot). - **Other rules:** any other FHIRPath. +The page that shows a profile is the same for everyone. Its first part is a table of three columns: the **Expectation** (the description), the **Section** (an IPS section by its title, or `system|code`) and the **Entry** (`Code is system|code`, or `Code is in` a value set, by its title, or its URL when the title isn't known). A code system is shown by name when it is a known one. A cardinality, `[m..n]`, follows a section or an entry unless it is `1..1` or not given (none means at least one). The second part is also a table of three columns: the **Expectation**, the **Resource Set** (the type, as `MedicationAdministrations where` or `Any resource where`, then the code as in the Entry column) and the **Rule**, which is the rest of the expression in words where it is one of these: `.exists()` is "Must be present somewhere"; `.all(sameSubject())` is "Patient must match the Document"; `.x.exists()` and `.x.y.exists()` are "Must have an x" and "Must have an x.y" (no deeper); `.conformsTo(url)` (or `.all(conformsTo(url))`) is "Conform to" the profile, named by its title when it is one of the clinical profiles or by its jurisdiction and package when it is a national profile; and `.category.hasCoding(system, code [, version])` (or inside `.all()`) is "Has category system|code". Any other rest is shown as FHIRPath, and so are the other rules. On the same page the administrator edits the rules one at a time: each row has Edit and Delete, and each part has an Add button. Add and Edit open a dialog for the one rule - the expectation, then the parts of the rule, each with what it means (for a rule about resources, what must be true of them is a choice: present somewhere, the patient matches the document's, an element, conformance to a profile, a category, or other FHIRPath), and the FHIRPath it makes - and Save stores that one rule by itself (`POST /ips/StructureDefinition/{id}/constraint` to add, `.../constraint/{key}` to change, `.../constraint/{key}/delete` to delete). An edited rule keeps its key. The full editor is still there (Edit all), for the title, or for many changes at once. + A rule moves between the parts when it is switched between table and FHIRPath, and it lands in the part whose shape it fits. A section is one of the IPS sections, or any code in any system. An entry or resource code is a code in a named code system (SNOMED CT, LOINC, ICD-10...) or in any other given by URL, and it is looked up on the terminology server (`GET /ips/lookup?system=&q=`, for the administrator only). Choosing "In a value set..." as the system matches entries or resources by value set instead. The code box becomes a value set search (`GET /ips/lookup-vs?q=`, for the administrator only): this module's own value sets that match the title come first, then the terminology server's (a `ValueSet?title=` search). A suggestion shows its title, with its URL underneath. A canonical can also be typed or pasted in, with `|version` to pin a version. When the editor opens, it looks up the title of each value set by URL (`?url=`). If the terminology server doesn't know a value set, it says so as a warning, not an error: the value set has to be known when documents are checked, not when the profile is saved. A cardinality is blank (no bounds) or `min..max`, where max may be `*`. Before a profile is saved, every expression is parsed and run once against an empty document. That catches unknown functions and wrong numbers of arguments as well as syntax errors. A profile can be downloaded as JSON (`?download=1`), and a downloaded profile can be uploaded again as a new profile, with a new id, url and keys. An upload must be no more than a profile like this: a profile on Bundle with no snapshot, and one differential element (the root) carrying constraints that have only a key, an `error` severity, a description, an expression and a source. Anything more is rejected, with the reasons. A profile downloaded when clinical profiles were on Composition is accepted and moved onto Bundle, as profiles already stored are when the server starts: `Composition.section(...)` becomes `Bundle.section(...)`, and any other `Composition.x` becomes `Bundle.entry.resource.ofType(Composition).x`. As FHIR: `GET /ips/StructureDefinition` is a searchset of them all, and `GET /ips/StructureDefinition/{id}` is one of them. +## National profiles + +A national profile is the profile a national implementation guide defines for an IPS document. Each has four parts: +the jurisdiction it is for, the IG's package id (e.g. `hl7.fhir.au.ips`), the package version, and the canonical URL +of the IG's profile on Bundle. National profiles are shared across all projects (`/ips/national-profiles`). Everyone can see them; only the +administrator adds, edits and deletes them. There is at most one for a package id and version. Every document is +validated against each of them by the [national profiles step](#national-profile-validation). + +The jurisdiction is a code from `http://hl7.org/fhir/ValueSet/jurisdiction`: a country, by its ISO 3166 two-letter +code (`AU`; lower case is accepted, and stored in upper case), or a region, by its UN M49 three-digit code (`150` for +Europe, `053` for Australia and New Zealand). The pages show the country's or region's name with it, from the code +lists the terminology module holds (`tx/cs/cs-country.js`, `tx/cs/cs-areacode.js`); ISO's user-assigned and 'unknown' codes +(`XK`, `XX`, `ZZ`...) and countries' M49 numeric codes aren't accepted. National profiles made before the jurisdiction was asked +for show it as not given, until they are saved with one. The package id must be a FHIR package id (lower case letters, digits, `-` and `_`, in two or more parts separated by +dots), the version a package version (`1.0.0`, `1.1.0-ballot`), and the profile an http or https URL. `GET +/ips/national-profiles` with `Accept: application/json` returns them as `[{jurisdiction, packageId, version, url}]`. + ## FHIRPath functions `ips/fhirpath-ips.js` adds two functions to fhirpath.js, for expressions on IPS documents: diff --git a/ips/render.js b/ips/render.js index 66180275..486aead9 100644 --- a/ips/render.js +++ b/ips/render.js @@ -26,7 +26,10 @@ const { STATUSES } = require('./search'); const profiles = require('./profiles'); const { terminologyOf, systemName, describeVersion, linkForSystem, linkForCode, isExternalSystem } = require('./terminology'); const { issueSummary, checkOf, translationOf } = require('./pipeline'); +const { jurisdiction } = require('./jurisdictions'); const { dateText } = require('./presenter'); +const scripts = require('./scripts'); +const { LOINC } = require('./editor-schema'); const STATUS_COLOURS = { submitted: '#6c757d', @@ -114,6 +117,27 @@ const STYLE = ` .ips-pf-bad { border-color: #c82333 !important; } .ips-pf-rule { width: 100%; table-layout: fixed; } .ips-pf-rule td { vertical-align: top; overflow-wrap: anywhere; } + .ips-pf-rule .ips-pf-actions { width: 7em; white-space: nowrap; text-align: right; } + .ips-pf-rule td.ips-pf-actions { padding-top: 1px; padding-bottom: 1px; } + .ips-pf-part-head { display: flex; align-items: baseline; gap: 12px; margin-top: 18px; } + .ips-pf-part-head h4 { margin: 0 0 6px; } + .ips-pf-part-head .ips-pf-add { font-size: 80%; padding: 0 8px; line-height: 1.6; } + .ips-pf-rule .ips-pf-actions .btn { font-size: 75%; padding: 0 5px; line-height: 1.5; border-radius: 3px; vertical-align: baseline; } + .ips-pf-how { align-items: center; gap: 8px; margin-top: 6px; } .ips-pf-how input { width: 7em; flex: none; } .ips-pf-how .ips-pf-hint { margin: 0; } + .ips-pf-dialog { width: min(760px, 95vw); max-height: 90vh; padding: 0; border: 1px solid #bbb; border-radius: 6px; box-shadow: 0 8px 30px rgba(0,0,0,0.3); } + .ips-pf-dialog::backdrop { background: rgba(0,0,0,0.35); } + .ips-pf-dialog-head { display: flex; align-items: center; justify-content: space-between; padding: 10px 16px; border-bottom: 1px solid #dee2e6; background: #f8f9fa; } + .ips-pf-dialog-head h4 { margin: 0; } + .ips-pf-dialog-x { border: none; background: none; font-size: 22px; line-height: 1; cursor: pointer; color: #666; } + .ips-pf-dialog-body { padding: 12px 16px; overflow: visible; } + .ips-pf-dialog-foot { padding: 10px 16px; border-top: 1px solid #dee2e6; text-align: right; } + .ips-pf-field { margin-bottom: 14px; } + .ips-pf-field > .ips-pf-label { display: block; font-weight: bold; margin-bottom: 4px; } + .ips-pf-field .ips-ed-input, .ips-pf-field textarea { width: 100%; box-sizing: border-box; } + .ips-pf-field > * + * { margin-top: 4px; } + .ips-pf-field .ips-pf-pair .ips-ed-input { width: auto; flex: 1; } + .ips-pf-field .ips-pf-how .ips-ed-input { width: 7em; flex: none; } + .ips-pf-hint { font-size: 85%; color: #6c757d; } .ips-pf-rule .ips-pf-card-text { white-space: nowrap; } .ips-ed-section { border: 1px solid #dee2e6; border-radius: 4px; margin-bottom: 8px; } .ips-ed-section-required { border-color: #b8c4e8; } @@ -241,6 +265,7 @@ function userBar(ctx) { parts.push(`Projects`); parts.push(`Users`); parts.push(`Clinical profiles`); + parts.push(`National profiles`); } parts.push(`
${csrfField(ctx.csrf)}
`); return `
${parts.join('  |  ')}
`; @@ -272,7 +297,8 @@ function renderHome(projects, ctx) { let h = STYLE + userBar(ctx); h += '

Each project is a collection of International Patient Summary documents, with its own FHIR endpoint ' + '(POST / GET {project}/Bundle) and web pages. ' + - `Clinical profiles say what kinds of clinical document are expected to contain.

`; + `Clinical profiles say what kinds of clinical document are expected to contain, and ` + + `national profiles are the IPS document profiles that national implementation guides define.

`; if (projects.length === 0) { return h + '

There are no projects yet' + (ctx.user?.isAdmin ? `: create one` : '') + '.

'; } @@ -1074,9 +1100,21 @@ function renderValidation(row, options = {}) { if (!r || (r.result.status && r.result.status !== 'done')) { return noResultNote(row, 'validation', 'There is no validation result for this document', options); } - const v = r.result; + return validationOutcome(r.made, r.result, { ...options, hidden: { view: 'validation' } }); +} + +/** + * What the validator made of the document, once: when, with what, the issue counts, the + * issues (as filtered by options.level) and the OperationOutcome. The validation view and + * each national profile's page show it. + * @param {string} made - when the validation was done + * @param {{counts?: Object, outcome?: Object, profiles?: string[], validatorVersion?: string, ms?: number}} v + * @param {{level?: string, docUrl?: string, hidden?: Object}} options - hidden: the fields the + * level form keeps (the view, and which national profile) + */ +function validationOutcome(made, v, options = {}) { const counts = v.counts || {}; - let h = `

Validated ${escape(r.made)}` + + let h = `

Validated ${escape(made)}` + (v.validatorVersion ? ` with validator ${escape(String(v.validatorVersion))}` : '') + (Array.isArray(v.profiles) && v.profiles.length ? ` against ${v.profiles.map(p => `${escape(p)}`).join(', ')}` : '') + (typeof v.ms === 'number' ? ` in ${escape(String(v.ms))} ms` : '') + '

'; @@ -1089,7 +1127,8 @@ function renderValidation(row, options = {}) { const min = SEVERITY_RANK[level]; const issues = all.filter(i => (SEVERITY_RANK[text(i.severity)] ?? 3) >= min); // the minimum level: a form that submits itself on change (and has a button without script) - h += `
` + + const hidden = Object.entries(options.hidden || { view: 'validation' }).map(([k, val]) => ``).join(''); + h += `${hidden}` + ' '; @@ -1121,6 +1160,58 @@ document.querySelectorAll('.ips-oo-toggle').forEach(function (b) { return h; } +// a national profile's outcome on the page: [label, colour] +const NATIONAL_OUTCOMES = { + pass: ['Passed', '#1e7e34'], + fail: ['Failed', '#c82333'], + unknown: ['Not checked', '#d39e00'] +}; + +/** + * The national profiles step's result: the list of national profiles, and whether the + * document passed each (no errors); with options.np (an index into the list), that + * profile's validation, shown as the validation view shows the base validation. + * @param {Object} row - from store.read(), with .results.national or not + * @param {{np?: string, level?: string, docUrl?: string, base?: string, canManage?: boolean}} [options] + */ +function renderNational(row, options = {}) { + const r = row.results?.national; + if (!r || (r.result.status && r.result.status !== 'done') || !Array.isArray(r.result.profiles)) { + return noResultNote(row, 'national', 'The document has not been validated against the national profiles', options); + } + const v = r.result; + const base = escape(options.base || ''); + const docUrl = escape(options.docUrl || ''); + if (v.profiles.length === 0) { + return `

There were no national profiles to validate the document against when it was processed.

`; + } + const outcome = (p) => NATIONAL_OUTCOMES[p.pass === true ? 'pass' : p.pass === false ? 'fail' : 'unknown']; + const index = typeof options.np === 'string' && /^\d{1,4}$/.test(options.np) ? parseInt(options.np, 10) : -1; + const p = index >= 0 ? v.profiles[index] : null; + if (p) { + const [label, colour] = outcome(p); + let h = `

National profiles » ${escape(`${p.packageId}#${p.version}`)}

`; + h += `

${jurisdictionHtml(p.jurisdiction)}: ${escape(p.url)} (${escape(`${p.packageId}#${p.version}`)}): ${escape(label)}

`; + if (p.pass === null || p.pass === undefined) { + return h + `

The document could not be validated against this profile${options.canManage && p.error ? `: ${escape(p.error)}` : ' (the reason is shown to the project\'s managers)'}.

`; + } + return h + validationOutcome(r.made, { counts: p.counts, outcome: p.outcome, profiles: [p.url], validatorVersion: v.validatorVersion, ms: p.ms }, + { ...options, hidden: { view: 'national', np: String(index) } }); + } + let h = ''; + v.profiles.forEach((np, i) => { + const [label, colour] = outcome(np); + const issues = np.pass === null || np.pass === undefined ? '(not validated)' : escape(issueSummary(np.counts || {})); + h += `` + + ``; + }); + h += '
JurisdictionPackageOutcomeIssues
${jurisdictionHtml(np.jurisdiction)}${escape(`${np.packageId}#${np.version}`)}${escape(label)}${issues}
'; + h += `

Validated ${escape(r.made)} against the national profiles as they were when the document was processed ` + + `(${v.profiles.length} profile${v.profiles.length === 1 ? '' : 's'}, ${escape(String(v.ms))} ms); a document passes a profile when the validator reports no errors. ` + + 'Reprocess the document to validate it against them as they are now.

'; + return h; +} + /** The editor page: a container the script fills, and what it needs. */ function renderCompose(project, ctx, options) { const purl = projectUrl(ctx, project); @@ -1139,10 +1230,70 @@ function renderCompose(project, ctx, options) { }; // JSON in a script: and the like must not end the block h += ``; - h += ``; + h += ``; return h; } +// ---- national profiles ----------------------------------------------------------------- + +/** A national profile's jurisdiction, for the page: its code, and the country's or region's name. */ +function jurisdictionHtml(code) { + if (!code) { + return '(not given)'; + } + const j = jurisdiction(code); + return `${escape(code)}` + (j ? ` ${escape(j.display)}` : ''); +} + +/** + * The national profiles: for each, the national IG's package (id and version) and the url of + * its profile on Bundle. Everyone sees the list; the administrator adds, edits and deletes. + * @param {Array} list - from store.nationalProfiles() + */ +function renderNationalProfiles(list, ctx, message) { + const base = escape(ctx.base); + const isAdmin = !!ctx.user?.isAdmin; + let h = STYLE + userBar(ctx) + `

IPS projects » National profiles

` + alert(message); + h += '
National profiles
'; + h += '

A national profile is the profile a national implementation guide defines for an IPS document: the country or region it is for, the IG\'s package (its id and version), ' + + 'and the canonical URL of its profile on Bundle. They apply across all projects.' + (isAdmin ? '' : ' Only the administrator can change them.') + '

'; + if (list.length === 0) { + h += '

There are no national profiles yet.

'; + } + if (!isAdmin) { + if (list.length === 0) { + return h; + } + h += ''; + for (const np of list) { + h += ``; + } + return h + '
JurisdictionPackageVersionProfile on Bundle
${jurisdictionHtml(np.jurisdiction)}${escape(np.packageId)}${escape(np.version)}${escape(np.url)}
'; + } + const fields = (form, np) => + `` + + (np && np.jurisdiction && jurisdiction(np.jurisdiction) ? `${escape(jurisdiction(np.jurisdiction).display)}` : '') + '' + + `` + + `` + + ``; + // the package id and version are short; the url takes the rest of the row + h += ''; + for (const np of list) { + const form = `np-${np.id}`; + const action = `${base}/national-profiles/${np.id}`; + h += '' + fields(form, np) + + `'; + } + h += '' + fields('np-new', null) + + `'; + return h + '
JurisdictionPackageVersionProfile on Bundle
${csrfField(ctx.csrf)}` + + ' ' + + `
${csrfField(ctx.csrf)}` + + '
${csrfField(ctx.csrf)}` + + '
'; +} + // ---- clinical profiles ----------------------------------------------------------------- function profileCrumbs(ctx, extra) { @@ -1201,63 +1352,180 @@ function renderProfiles(list, ctx) { // the parts of a profile, as the view and the editor show them const PROFILE_PARTS = [['section', 'Required sections and entries'], ['resource', 'Rules about resources'], ['other', 'Other rules']]; -/** One rule of a profile, for reading: the shape as words, or the expression. */ -function ruleHtml(expression, titleOf) { - const rshape = profiles.parseResourceShape(expression); - if (rshape) { - const d = profiles.describeResource(rshape, titleOf); - const which = d.valueSet !== undefined - ? `with a code in the value set ${escape(d.title || d.valueSet)}${d.title ? ` (${escape(d.valueSet)})` : ''}` - : `with the code ${escape(d.code)}`; - return `${d.type ? `${escape(d.type)} resources` : 'Resources'} ${which}: ${escape(d.rest)}` + - `
${escape(expression)}
`; - } - const shape = profiles.parseShape(expression); - if (!shape) { - return `${escape(expression)}`; - } - const part = (p, isSection) => { - const d = profiles.describePart(p, isSection, titleOf); - return `${escape(d.what)}${d.code ? ` (${escape(d.code)})` : ''}` + - (d.card ? ` [${escape(d.card)}]` : ''); - }; - const entry = !shape.entry ? '' : (shape.entry.valueSet !== undefined ? ', with entries in the value set ' : ', with an entry ') + part(shape.entry, false); - return 'Section ' + part(shape.section, true) + entry + `
${escape(expression)}
`; +// the code systems for the categories of the resources in an IPS, for a rule's category +const CATEGORY_SYSTEMS = [ + { url: 'http://terminology.hl7.org/CodeSystem/condition-category', name: 'Condition category' }, + { url: 'http://terminology.hl7.org/CodeSystem/observation-category', name: 'Observation category' }, + { url: 'http://terminology.hl7.org/CodeSystem/medicationrequest-category', name: 'MedicationRequest category' } +]; + +/** A cardinality, for the profile's page: " [m..n]", unless it is unstated or 1..1. */ +function cardSuffix(p) { + return p.min === null || p.min === undefined || (p.min === 1 && p.max === 1) ? '' : ` [${p.min}..${p.max}]`; +} + +/** The name of a code system, when it has one we know, or its URL. */ +function profileSystemName(url) { + const sys = profiles.SYSTEMS.find(s => s.url === url); + return sys ? sys.name : (systemName(url) || url); +} + +/** The Section and Entry columns of a required section/entry row. */ +function sectionColumns(shape, titleOf) { + const s = shape.section; + const named = s.system === LOINC ? profiles.IPS_SECTIONS.find(x => x.code === s.code) : null; + const section = (named + ? `${escape(named.title)} (${escape(`${profileSystemName(s.system)}|${s.code}`)})` + : escape(`${profileSystemName(s.system)}|${s.code}`)) + escape(cardSuffix(s)); + const e = shape.entry; + let entry = ''; + if (e && e.valueSet !== undefined) { + const title = typeof titleOf === 'function' ? titleOf(e.valueSet) : null; + entry = `Code is in ${escape(title || e.valueSet)}` + (title ? ` (${escape(e.valueSet)})` : '') + escape(cardSuffix(e)); + } else if (e) { + entry = `Code is ${escape(`${profileSystemName(e.system)}|${e.code}`)}` + escape(cardSuffix(e)); + } + return [section, entry]; } -/** A clinical profile, read-only (the administrator gets edit and delete). */ -function renderProfile(sd, ctx) { +/** The resources a rule is about, as a plural: MedicationAdministrations, or Any resource. */ +function resourceTypePlural(type) { + if (!type) { + return 'Any resource'; + } + return /[^aeiou]y$/.test(type) ? `${type.slice(0, -1)}ies` : `${type}s`; +} + +// the tails of a rule about resources that are said in words: each of the patterns, as it +// is, or (where a resource at a time is what is meant) inside .all(...) +const STR_ARG = "\\s*'((?:[^'\\\\]|\\\\.)*)'\\s*"; +const NAME = '[A-Za-z][A-Za-z0-9_]*'; +const allOf = (body) => new RegExp(`^\\.\\s*(?:${body}|all\\s*\\(\\s*${body}\\s*\\))$`); +const TAILS = [ + [/^\.\s*exists\s*\(\s*\)$/, () => 'Must be present somewhere'], + [/^\.\s*all\s*\(\s*sameSubject\s*\(\s*\)\s*\)$/, () => 'Patient must match the Document'], + [allOf(`conformsTo\\s*\\(${STR_ARG}\\)`), (m, ctx) => { + const url = unescapeArg(m[1] !== undefined ? m[1] : m[2]); + const title = typeof ctx.profileTitle === 'function' ? ctx.profileTitle(url) : null; + return `Conform to ${escape(title || url)}` + (title ? ` (${escape(url)})` : ''); + }], + [allOf(`category\\s*\\.\\s*hasCoding\\s*\\(${STR_ARG},${STR_ARG}(?:,${STR_ARG})?\\)`), (m) => { + const [system, code, version] = (m[1] !== undefined ? [m[1], m[2], m[3]] : [m[4], m[5], m[6]]).map(x => x === undefined ? undefined : unescapeArg(x)); + return `Has category ${escape(`${profileSystemName(system)}|${code}`)}` + (version !== undefined ? ` (version ${escape(version)})` : ''); + }], + // an element, or an element's element (no deeper): Must have an x, Must have an x.y + [new RegExp(`^\\.\\s*(${NAME})\\s*(?:\\.\\s*(${NAME})\\s*)?\\.\\s*exists\\s*\\(\\s*\\)$`), (m) => { + const path = m[2] ? `${m[1]}.${m[2]}` : m[1]; + return `Must have ${/^[aeiou]/i.test(path) ? 'an' : 'a'} ${escape(path)}`; + }] +]; + +/** A FHIRPath string's content, unescaped. */ +function unescapeArg(s) { + return s.replace(/\\(u[0-9a-fA-F]{4}|.)/g, (x, c) => c.length === 5 ? String.fromCharCode(parseInt(c.substring(1), 16)) : ({ n: '\n', r: '\r', t: '\t', f: '\f' }[c] || c)); +} + +/** The Resource Set and Rule columns of a rule about resources. */ +function resourceColumns(rshape, ctx) { + const titleOf = ctx.valueSetTitle; + let which; + if (rshape.valueSet !== undefined) { + const title = typeof titleOf === 'function' ? titleOf(rshape.valueSet) : null; + which = `Code is in ${escape(title || rshape.valueSet)}` + (title ? ` (${escape(rshape.valueSet)})` : ''); + } else { + which = `Code is ${escape(`${profileSystemName(rshape.system)}|${rshape.code}`)}`; + } + const set = `${escape(resourceTypePlural(rshape.type))} where ${which}`; + const rest = rshape.rest.trim(); + for (const [re, words] of TAILS) { + const m = re.exec(rest); + if (m) { + return [set, words(m, ctx)]; + } + } + return [set, `${escape(rest)}`]; +} + +/** + * A clinical profile: the same page for everyone. The administrator also gets add, edit and + * delete on each row (a row at a time, the script edits it in place), and the full editor. + */ +function renderProfile(sd, ctx, editor = null) { const base = escape(ctx.base); const url = `${base}/StructureDefinition/${escape(sd.id)}`; + const admin = !!(editor && ctx.user?.isAdmin); let h = STYLE + userBar(ctx) + profileCrumbs(ctx, sd.title); h += `
${escape(sd.title)}` + `
Clinical profile on Bundle  ·  ${escape(sd.url)}  ·  updated ${escape(dateOnly(sd.date))}
`; h += '
' + `Download JSON` + - (ctx.user?.isAdmin ? ` Edit ` + + (admin ? ` Edit all ` + `
${csrfField(ctx.csrf)}` + - '
' : '') + '
'; - const constraints = sd.differential?.element?.[0]?.constraint || []; - if (constraints.length === 0) { - h += '

This profile has no constraints yet.

'; - } else { - for (const [kind, heading] of PROFILE_PARTS) { - const these = constraints.filter(c => profiles.kindOf(c.expression || '') === kind); - if (!these.length) { - continue; + '' : '') + ''; + const constraints = (sd.differential?.element?.[0]?.constraint || []).filter(c => c && typeof c === 'object'); + // the actions on a row, for the administrator + const actions = (c) => !admin ? '' : '' + + ' ' + + `
${csrfField(ctx.csrf)}` + + '
'; + const rowAttrs = (c) => (admin ? ` data-key="${escape(c.key || '')}"` : '') + ` title="${escape(c.expression || '')}"`; + const actionsHead = admin ? '' : ''; + const ADD = { section: 'Add a required section/entry', resource: 'Add a rule about resources', other: 'Add another rule' }; + for (const [kind, heading] of PROFILE_PARTS) { + const these = constraints.filter(c => profiles.kindOf(c.expression || '') === kind); + if (!these.length && !admin) { + continue; + } + // the heading, and for the administrator the part's Add button, to its right + h += `

${escape(heading)}

` + + (admin ? `` : '') + '
'; + if (kind === 'section') { + h += `${actionsHead}`; + for (const c of these) { + const [section, entry] = sectionColumns(profiles.parseShape(c.expression || ''), ctx.valueSetTitle); + h += `${actions(c)}`; + } + } else if (kind === 'resource') { + h += `
ExpectationSectionEntry
${escape(c.human || '')}${section}${entry}
${actionsHead}`; + for (const c of these) { + const [set, rule] = resourceColumns(profiles.parseResourceShape(c.expression || ''), ctx); + h += `${actions(c)}`; } - h += `

${escape(heading)}

ExpectationResource SetRule
${escape(c.human || '')}${set}${rule}
`; + } else { + h += `
ExpectationRule
${actionsHead}`; for (const c of these) { - h += ``; + h += `${actions(c)}`; } - h += '
ExpectationRule
${escape(c.human || '')}${ruleHtml(c.expression || '', ctx.valueSetTitle)}
${escape(c.human || '')}${escape(c.expression || '')}
'; } + h += ''; + } + if (constraints.length === 0 && !admin) { + h += '

This profile has no constraints yet.

'; } h += '

Each rule is checked against the document (the Bundle). section(system, code, min, max) finds the sections of the document\'s Composition ' + '(nested ones too) with that code, when there are min..max of them; entry(system, code, min, max) finds a section\'s entries with that code ' + '(their code, medicationCodeableConcept or vaccineCode), when there are min..max of them; ' + 'resources(system, code, type) finds the resources in the document with that code (of that type, when one is given); ' + - 'entryIn(valueSet, min, max) and resourcesIn(valueSet, type) are the same with a code in a value set, asked of the terminology server.

'; + 'entryIn(valueSet, min, max) and resourcesIn(valueSet, type) are the same with a code in a value set, asked of the terminology server. ' + + 'A cardinality [m..n] after a section or an entry is how many there may be; with none, there must be at least one.

'; + if (admin) { + const cfg = { + mode: 'rows', profile: { id: sd.id, title: sd.title, constraints: editor.constraints }, + sections: editor.sections, systems: editor.systems, resourceTypes: editor.resourceTypes, sectionSystem: editor.sectionSystem, csrf: ctx.csrf || '', + parts: PROFILE_PARTS, + urls: { + constraint: `${ctx.base}/StructureDefinition/${sd.id}/constraint`, + lookup: editor.canLookup ? `${ctx.base}/lookup` : null, + lookupVs: `${ctx.base}/lookup-vs` + }, + // the code systems offered for a resource's category, first the default + categorySystem: CATEGORY_SYSTEMS[0].url, + categorySystems: CATEGORY_SYSTEMS + }; + h += '
'; + h += ``; + h += ``; + } return h; } @@ -1286,7 +1554,7 @@ function renderProfileEditor(sd, ctx, options) { } }; h += ``; - h += ``; + h += ``; return h; } @@ -1362,6 +1630,7 @@ const VIEWS = [ ['translation', 'Translation'], ['generated', 'Generated Narrative'], ['validation', 'Validation'], + ['national', 'National Profiles'], ['terminology', 'Terminology'], ['profiles', 'Clinical Profiles'], ['ai', 'AI Assessment'], @@ -1469,6 +1738,8 @@ function renderDocument(project, row, view, ctx) { h += renderAi(row, { canManage: ctx.canManage }); } else if (view === 'validation') { h += renderValidation(row, { level: ctx.level, docUrl: durl, canManage: ctx.canManage }); + } else if (view === 'national') { + h += renderNational(row, { np: ctx.np, level: ctx.level, docUrl: durl, base: ctx.base, canManage: ctx.canManage }); } else { h += assembledNarrative(row.bundle); } @@ -1477,6 +1748,6 @@ function renderDocument(project, row, view, ctx) { module.exports = { renderHome, renderLogin, renderProjects, renderUsers, renderProject, renderSubmit, renderSettings, renderDocument, - assembledNarrative, renderTerminology, renderTranslation, renderGenerated, renderAi, renderValidation, renderShare, renderCompose, renderProfileChecks, - renderProfiles, renderProfile, renderProfileEditor, VIEWS + assembledNarrative, renderTerminology, renderTranslation, renderGenerated, renderAi, renderValidation, renderNational, renderShare, renderCompose, renderProfileChecks, + renderProfiles, renderProfile, renderProfileEditor, renderNationalProfiles, VIEWS }; diff --git a/ips/scripts.js b/ips/scripts.js new file mode 100644 index 00000000..c393b5f2 --- /dev/null +++ b/ips/scripts.js @@ -0,0 +1,46 @@ +// +// Copyright 2026, Health Intersections Pty Ltd (http://www.healthintersections.com.au) +// +// Licensed under BSD-3: https://opensource.org/license/bsd-3-clause +// + +/* + * The module's browser scripts: its own files, read once and served from memory (so no + * request reaches the file system), with an ETag. A page links a script with its version + * (?v=, from the script's content), so a browser that has kept a copy picks up a changed + * script as soon as the server serves it, not when its cached copy expires. + */ +const fs = require('fs'); +const path = require('path'); +const crypto = require('crypto'); + +const SCRIPTS = new Map(); + +function load(name) { + if (!SCRIPTS.has(name)) { + const body = fs.readFileSync(path.join(__dirname, name), 'utf8'); + const hash = crypto.createHash('sha256').update(body).digest('base64url'); + SCRIPTS.set(name, { body, etag: `"${hash.substring(0, 27)}"`, version: hash.substring(0, 12) }); + } + return SCRIPTS.get(name); +} + +/** The route handler that serves the script. */ +function handler(name) { + const s = load(name); + return (req, res) => { + res.setHeader('Cache-Control', 'public, max-age=3600'); + res.setHeader('ETag', s.etag); + if (req.get('If-None-Match') === s.etag) { + return res.status(304).end(); + } + res.type('application/javascript').send(s.body); + }; +} + +/** The script's URL, for a page: base/name?v=version (not escaped). */ +function src(base, name) { + return `${base}/${name}?v=${load(name).version}`; +} + +module.exports = { handler, src }; diff --git a/ips/store.js b/ips/store.js index 63096660..7c59ca0d 100644 --- a/ips/store.js +++ b/ips/store.js @@ -102,6 +102,17 @@ const SCHEMA = ` json TEXT NOT NULL ); + -- national profiles: the profile a national IG defines for an IPS document (the Bundle), across all projects + CREATE TABLE IF NOT EXISTS national_profiles ( + id INTEGER PRIMARY KEY AUTOINCREMENT, + package_id TEXT NOT NULL, -- the IG's package id + version TEXT NOT NULL, -- the package version + url TEXT NOT NULL, -- the canonical URL of the IG's profile on Bundle + jurisdiction TEXT, -- the country (ISO 3166 two-letter code) or region (UN M49 three digits) + updated_ms INTEGER NOT NULL, + UNIQUE (package_id, version) + ); + CREATE TABLE IF NOT EXISTS ips_index ( doc_id INTEGER NOT NULL REFERENCES documents(id) ON DELETE CASCADE, res TEXT NOT NULL, -- Bundle | Composition | Patient @@ -158,6 +169,11 @@ function summaryOf(bundle) { }; } +/** A national_profiles row, as the store gives it. */ +function nationalProfileOf(r) { + return { id: r.id, jurisdiction: r.jurisdiction || null, packageId: r.package_id, version: r.version, url: r.url, updated_ms: r.updated_ms }; +} + class IpsStore { /** * @param {string} dbPath - file name, or ':memory:' @@ -203,6 +219,9 @@ class IpsStore { if (!has('documents', 'analysed_json')) { this.db.exec('ALTER TABLE documents ADD COLUMN analysed_json TEXT'); } + if (!has('national_profiles', 'jurisdiction')) { + this.db.exec('ALTER TABLE national_profiles ADD COLUMN jurisdiction TEXT'); + } if (!has('projects', 'ai_json')) { // the project's AI assessment settings (JSON; the API key encrypted) this.db.exec('ALTER TABLE projects ADD COLUMN ai_json TEXT'); @@ -261,6 +280,42 @@ class IpsStore { return this.db.prepare('DELETE FROM clinical_profiles WHERE id = ?').run(id).changes > 0; } + // ---- national profiles ---- + + /** All the national profiles: {id, jurisdiction, packageId, version, url, updated_ms}, by package id and version. */ + nationalProfiles() { + return this.db.prepare('SELECT id, package_id, version, url, jurisdiction, updated_ms FROM national_profiles ORDER BY package_id, version, id').all() + .map(nationalProfileOf); + } + + /** @returns {Object|null} {id, jurisdiction, packageId, version, url, updated_ms} */ + nationalProfile(id) { + const r = this.db.prepare('SELECT id, package_id, version, url, jurisdiction, updated_ms FROM national_profiles WHERE id = ?').get(id); + return r ? nationalProfileOf(r) : null; + } + + /** The national profile for a package id and version, or null. */ + nationalProfileByPackage(packageId, version) { + const r = this.db.prepare('SELECT id FROM national_profiles WHERE package_id = ? AND version = ?').get(packageId, version); + return r ? this.nationalProfile(r.id) : null; + } + + /** @returns {number} the new national profile's id */ + createNationalProfile(packageId, version, url, jurisdiction, nowMs = Date.now()) { + return Number(this.db.prepare('INSERT INTO national_profiles (package_id, version, url, jurisdiction, updated_ms) VALUES (?, ?, ?, ?, ?)') + .run(packageId, version, url, jurisdiction || null, nowMs).lastInsertRowid); + } + + /** @returns {boolean} whether there was such a national profile */ + updateNationalProfile(id, packageId, version, url, jurisdiction, nowMs = Date.now()) { + return this.db.prepare('UPDATE national_profiles SET package_id = ?, version = ?, url = ?, jurisdiction = ?, updated_ms = ? WHERE id = ?') + .run(packageId, version, url, jurisdiction || null, nowMs, id).changes > 0; + } + + deleteNationalProfile(id) { + return this.db.prepare('DELETE FROM national_profiles WHERE id = ?').run(id).changes > 0; + } + // ---- projects ---- /** All projects, with their document counts and managers; by name. */ diff --git a/package-lock.json b/package-lock.json index cbf73b64..35ddb996 100644 --- a/package-lock.json +++ b/package-lock.json @@ -24,7 +24,7 @@ "express-rate-limit": "^7.4.1", "express-session": "^1.19.0", "fast-xml-parser": "^5.3.4", - "fhir-validator-wrapper": "1.5.0", + "fhir-validator-wrapper": "1.6.1", "fhirpath": "^5.2.1", "fs-extra": "^11.3.3", "ini": "^6.0.0", @@ -4758,9 +4758,9 @@ "license": "MIT" }, "node_modules/fhir-validator-wrapper": { - "version": "1.5.0", - "resolved": "https://registry.npmjs.org/fhir-validator-wrapper/-/fhir-validator-wrapper-1.5.0.tgz", - "integrity": "sha512-Ug7PsSnHIpLlgLCn164YvjgC9K1j+JJz51rxp8hCAQD+zifkbTTS/wtMeq7i+DbLL6GC8HwyAfq+MtxtrtB7tA==", + "version": "1.6.1", + "resolved": "https://registry.npmjs.org/fhir-validator-wrapper/-/fhir-validator-wrapper-1.6.1.tgz", + "integrity": "sha512-cxx0Dh3nyMJ5TcvyhULvqkAku0xCDJ7K8NArDBjQk7fVO9emj69WsxvxqkoHThaQ7HGEvJEzbaFQ30y9DLE3kg==", "license": "Apache-2.0", "engines": { "node": ">=12.0.0" diff --git a/package.json b/package.json index b4590633..ea7aa018 100644 --- a/package.json +++ b/package.json @@ -48,7 +48,7 @@ "express-rate-limit": "^7.4.1", "express-session": "^1.19.0", "fast-xml-parser": "^5.3.4", - "fhir-validator-wrapper": "1.5.0", + "fhir-validator-wrapper": "1.6.1", "fhirpath": "^5.2.1", "fs-extra": "^11.3.3", "ini": "^6.0.0", diff --git a/tests/cm/cm-package.test.js b/tests/cm/cm-package.test.js index 97e81bf6..f2c7b748 100644 --- a/tests/cm/cm-package.test.js +++ b/tests/cm/cm-package.test.js @@ -294,7 +294,7 @@ describe('PackageConceptMapProvider', () => { }); }); - describe('excludeUrls', () => { + describe('keepPinnedOnly', () => { let reduced; let gone; @@ -303,15 +303,30 @@ describe('PackageConceptMapProvider', () => { await reduced.initialize(); reduced.assignIds('core'); gone = [...reduced.conceptMapMap.values()][0]; - reduced.excludeUrls(new Set([gone.url])); + reduced.keepPinnedOnly(new Map([[gone.url, new Set(['999.0.0'])]])); }); - test('an excluded concept map is not fetched, by url or by id', async () => { + test('a restricted concept map is still fetched by its own version, where THO does not have that version', async () => { + expect(gone.version).toBeTruthy(); + expect(await reduced.fetchConceptMap(gone.url, gone.version)).toBe(gone); + }); + + test('a restricted concept map is not fetched by a version that THO has', async () => { + const thoToo = new PackageConceptMapProvider(reduced.packageLoader); + await thoToo.initialize(); + thoToo.assignIds('core'); + const cm = [...thoToo.conceptMapMap.values()].find(r => r.url === gone.url); + thoToo.keepPinnedOnly(new Map([[cm.url, new Set([cm.version])]])); + expect(await thoToo.fetchConceptMap(cm.url, cm.version)).toBeNull(); + expect(await thoToo.fetchConceptMap(cm.url, null)).toBeNull(); + }); + + test('a restricted concept map is not fetched without a version, or by id', async () => { expect(await reduced.fetchConceptMap(gone.url, null)).toBeNull(); expect(await reduced.fetchConceptMapById(gone.id)).toBeNull(); }); - test('an excluded concept map is not found by search', async () => { + test('a restricted concept map is not found by search', async () => { for (const elements of [null, ['id', 'url']]) { const results = await reduced.searchConceptMaps([], elements); expect(results.some(r => r.id === gone.id)).toBe(false); diff --git a/tests/cs/cs-cs-notselectable.test.js b/tests/cs/cs-cs-notselectable.test.js new file mode 100644 index 00000000..7af15827 --- /dev/null +++ b/tests/cs/cs-cs-notselectable.test.js @@ -0,0 +1,92 @@ +// +// How a CodeSystem's notSelectable property is recognised (tx-ecosystem requirements, +// Concept Properties): by the canonical uri whatever local code it has, by the standard +// code where the code system declares nothing, and never where the code system declares +// the code with a different uri - that is a different property +// + +const { CodeSystem } = require('../../tx/library/codesystem'); +const { FhirCodeSystemProvider } = require('../../tx/cs/cs-cs'); +const { FilterExecutionContext } = require('../../tx/cs/cs-api'); +const { OperationContext } = require('../../tx/operation-context'); +const { TestUtilities } = require('../test-utilities'); + +const NS_URI = 'http://hl7.org/fhir/concept-properties#notSelectable'; + +function codeSystem(code, uri) { + const prop = (value) => ({ code, valueBoolean: value }); + const cs = { + resourceType: 'CodeSystem', + url: 'http://example.org/fhir/CodeSystem/ns-' + code + '-' + (uri ? uri.split('#')[1] : 'none'), + version: '0.1.0', + status: 'active', + content: 'complete', + concept: [ + { code: 'codeU', display: 'Unknown Selectable State' }, + { code: 'codeS', display: 'Selectable Code', property: [prop(false)] }, + { code: 'codeNS', display: 'Not Selectable Code', property: [prop(true)] } + ] + }; + if (uri !== undefined) { + cs.property = [{ code, type: 'boolean' }]; + if (uri) { + cs.property[0].uri = uri; + } + } + return cs; +} + +describe('recognising the notSelectable property', () => { + let opContext; + + beforeEach(async () => { + opContext = new OperationContext('en-US', + await TestUtilities.loadTranslations(await TestUtilities.loadLanguageDefinitions())); + }); + + const providerFor = (json) => new FhirCodeSystemProvider(opContext, new CodeSystem(json), []); + + async function abstractCodes(provider) { + const result = []; + for (const c of ['codeU', 'codeS', 'codeNS']) { + if (await provider.isAbstract(c)) { + result.push(c); + } + } + return result; + } + + async function filterCodes(provider, value) { + const set = await provider.filter(new FilterExecutionContext(), true, 'notSelectable', '=', value); + return set.concepts.map(c => c.concept.code).sort(); + } + + test('declared with the standard code and uri', async () => { + const provider = providerFor(codeSystem('notSelectable', NS_URI)); + expect(await abstractCodes(provider)).toEqual(['codeNS']); + expect(await filterCodes(provider, 'true')).toEqual(['codeNS']); + expect(await filterCodes(provider, 'false')).toEqual(['codeS']); + }); + + test('not declared at all: recognised by the standard code', async () => { + const provider = providerFor(codeSystem('notSelectable', undefined)); + expect(await abstractCodes(provider)).toEqual(['codeNS']); + expect(await filterCodes(provider, 'true')).toEqual(['codeNS']); + }); + + test('declared with the standard uri and a local code: recognised by the uri', async () => { + const provider = providerFor(codeSystem('selectable-not', NS_URI)); + expect(await abstractCodes(provider)).toEqual(['codeNS']); + expect(await filterCodes(provider, 'true')).toEqual(['codeNS']); + expect(await filterCodes(provider, 'false')).toEqual(['codeS']); + }); + + test('declared with the standard code and a different uri: a different property', async () => { + const provider = providerFor(codeSystem('notSelectable', NS_URI + 'X')); + // nothing is abstract + expect(await abstractCodes(provider)).toEqual([]); + // but a filter on the code is still a filter on the code system's own property + expect(await filterCodes(provider, 'true')).toEqual(['codeNS']); + expect(await filterCodes(provider, 'false')).toEqual(['codeS']); + }); +}); diff --git a/tests/ips/ips.test.js b/tests/ips/ips.test.js index b1599d2a..eb79dd46 100644 --- a/tests/ips/ips.test.js +++ b/tests/ips/ips.test.js @@ -955,6 +955,138 @@ describe('the validation step', () => { }); }); +describe('the national profiles step', () => { + let app; + let mod; + const AU = 'http://hl7.org.au/fhir/ips/StructureDefinition/au-ips-bundle'; + const NZ = 'http://hl7.org.nz/fhir/ips/StructureDefinition/nz-ips-bundle'; + const XX = 'http://example.org/fhir/StructureDefinition/xx-bundle'; + const failing = { resourceType: 'OperationOutcome', issue: [ + { severity: 'error', code: 'structure', details: { text: 'Bundle.identifier: minimum required = 1' }, expression: ['Bundle'] }, + { severity: 'information', code: 'informational', details: { text: 'Just a note' } }] }; + const passing = { resourceType: 'OperationOutcome', issue: [{ severity: 'warning', code: 'business-rule', details: { text: 'A warning only' } }] }; + beforeEach(async () => { + ({ app, mod } = await makeApp()); + mod.store.createProject('demo', 'Demo', null); + mod.steps = STEPS.filter(s => s.name === 'national'); + }); + afterEach(() => mod.shutdown()); + + const fakeValidator = () => ({ + calls: [], + loads: [], + isRunning: () => true, + jarVersion: () => '6.5.0', + async loadIG(packageId, version) { + this.loads.push(`${packageId}#${version}`); + return packageId === 'hl7.fhir.xx.ips' + ? { resourceType: 'OperationOutcome', issue: [{ severity: 'error', code: 'not-found', details: { text: 'Package not found' } }] } + : { resourceType: 'OperationOutcome', issue: [{ severity: 'information', code: 'informational', details: { text: 'Loaded' } }] }; + }, + async validate(resource, options) { + this.calls.push(options); + return options.profiles[0] === AU ? failing : passing; + }, + async stop() {} + }); + + test('with no national profiles, there is nothing to do', async () => { + await post(app, 'demo', ips()); + await mod.processQueue(); + expect(mod.store.read(1)).toMatchObject({ status: 'processed', status_message: 'national profiles: none' }); + expect(mod.store.readResult(1, 'national').result).toMatchObject({ profiles: [] }); + const page = await request(app).get('/ips/demo/Bundle/1?view=national').set('Accept', 'text/html'); + expect(page.text).toContain('
  • National Profiles
    • '); + expect(page.text).toContain('There were no national profiles to validate the document against'); + }); + + test('the document is validated against each, and passes when there are no errors', async () => { + mod.validator = fakeValidator(); + mod.store.createNationalProfile('hl7.fhir.au.ips', '1.0.0', AU, 'AU'); + mod.store.createNationalProfile('hl7.fhir.nz.ips', '2.0.0', NZ, 'NZ'); + mod.store.createNationalProfile('hl7.fhir.xx.ips', '0.1.0', XX, '150'); + await post(app, 'demo', ips()); + await post(app, 'demo', ips()); + expect(await mod.processQueue()).toBe(2); + // each package is loaded once; the one that won't load isn't validated against + expect(mod.validator.loads).toEqual(['hl7.fhir.au.ips#1.0.0', 'hl7.fhir.nz.ips#2.0.0', 'hl7.fhir.xx.ips#0.1.0']); + expect(mod.validator.calls).toEqual([ + { profiles: [AU], checkIpsCodes: true, timeout: 120000 }, { profiles: [NZ], checkIpsCodes: true, timeout: 120000 }, + { profiles: [AU], checkIpsCodes: true, timeout: 120000 }, { profiles: [NZ], checkIpsCodes: true, timeout: 120000 }]); + expect(mod.store.read(1)).toMatchObject({ status: 'processed', status_message: 'national profiles: 1 passed, 1 failed, 1 not checked' }); + const r = mod.store.readResult(1, 'national').result; + expect(r.validatorVersion).toBe('6.5.0'); + expect(r.profiles.map(p => [p.packageId, p.pass])).toEqual([['hl7.fhir.au.ips', false], ['hl7.fhir.nz.ips', true], ['hl7.fhir.xx.ips', null]]); + expect(r.profiles[0]).toMatchObject({ version: '1.0.0', url: AU, counts: { error: 1, warning: 0, information: 1, fatal: 0 }, outcome: failing }); + expect(r.profiles[2].error).toBe('The validator could not load hl7.fhir.xx.ips#0.1.0: Package not found'); + + // the list + const page = await request(app).get('/ips/demo/Bundle/1?view=national').set('Accept', 'text/html'); + expect(page.text).toContain('AU Australiahl7.fhir.au.ips#1.0.0'); + expect(page.text).toContain('150 Europe'); + expect(page.text).toContain('Failed1 error, 1 note'); + expect(page.text).toContain('Passed1 warning'); + expect(page.text).toContain('Not checked(not validated)'); + + // one profile's validation, as the validation view shows it + const au = await request(app).get('/ips/demo/Bundle/1?view=national&np=0').set('Accept', 'text/html'); + expect(au.text).toContain('National profiles » hl7.fhir.au.ips#1.0.0'); + expect(au.text).toContain(`against ${AU}`); + expect(au.text).toContain('1 error, 1 note'); + expect(au.text).toContain('Bundle.identifier: minimum required = 1'); + expect(au.text).toContain(''); + expect(au.text).toContain('Show the OperationOutcome'); + const filtered = await request(app).get('/ips/demo/Bundle/1?view=national&np=0&level=error').set('Accept', 'text/html'); + expect(filtered.text).toContain('Showing 1 of 2 issues (errors and above)'); + // the one that wasn't validated: why, for managers + const xx = await request(app).get('/ips/demo/Bundle/1?view=national&np=2').set('Accept', 'text/html'); + expect(xx.text).toContain("could not be validated against this profile (the reason is shown to the project's managers)"); + const { agent } = await loginAs(app, 'admin', 'admin-secret'); + expect((await agent.get('/ips/demo/Bundle/1?view=national&np=2').set('Accept', 'text/html')).text).toContain('Package not found'); + // a profile that isn't there: the list + expect((await request(app).get('/ips/demo/Bundle/1?view=national&np=7').set('Accept', 'text/html')).text).toContain('np=1">hl7.fhir.nz.ips#2.0.0'); + + // changing the national profiles: a package that failed to load is tried again + expect((await form(agent, '/ips/national-profiles', { jurisdiction: 'GB', packageId: 'hl7.fhir.yy.ips', version: '1.0.0', url: XX })).status).toBe(302); + mod.store.requeue(1, 1); + await mod.processQueue(); + expect(mod.validator.loads.filter(l => l === 'hl7.fhir.xx.ips#0.1.0')).toHaveLength(2); + }); + + test('a package is loaded by posting {ig: id#version} to the validator service\'s /loadIG', async () => { + // the validator service: what it was asked, and what it answers + const http = require('http'); + const seen = []; + const server = http.createServer((req, res) => { + let body = ''; + req.on('data', c => { body += c; }); + req.on('end', () => { + seen.push({ method: req.method, url: req.url, type: req.headers['content-type'], body }); + res.setHeader('Content-Type', 'application/fhir+json'); + res.end(JSON.stringify({ resourceType: 'OperationOutcome', issue: [{ severity: 'information', code: 'informational', details: { text: 'IG loaded successfully' } }] })); + }); + }); + await new Promise(r => server.listen(0, '127.0.0.1', r)); + try { + mod.validator = { baseUrl: `http://127.0.0.1:${server.address().port}`, loadIG: () => { throw new Error('not this one'); } }; + await mod.loadNationalPackage('hl7.fhir.au.ps', 'current'); + expect(seen).toEqual([{ method: 'POST', url: '/loadIG', type: 'application/json', body: '{"ig":"hl7.fhir.au.ps#current"}' }]); + // once + await mod.loadNationalPackage('hl7.fhir.au.ps', 'current'); + expect(seen).toHaveLength(1); + } finally { + await new Promise(r => server.close(r)); + } + }); + + test('without a validator, documents fail processing (when there are national profiles)', async () => { + mod.store.createNationalProfile('hl7.fhir.au.ips', '1.0.0', AU, 'AU'); + await post(app, 'demo', ips()); + await mod.processQueue(); + expect(mod.store.read(1)).toMatchObject({ status: 'error', status_message: 'national: The validator is not configured (modules.ips.validator)' }); + }); +}); + describe('the terminology step', () => { let app; let mod; @@ -1258,7 +1390,7 @@ describe('the terminology step', () => { test('the translate step adds the translations of concepts that fail the check, before validation, and the check then passes', async () => { mod.steps = STEPS.filter(s => s.name === 'translate' || s.name === 'terminology'); - expect(STEPS.map(s => s.name)).toEqual(['translate', 'narrative', 'validation', 'terminology', 'profiles', 'ai']); + expect(STEPS.map(s => s.name)).toEqual(['translate', 'narrative', 'validation', 'national', 'terminology', 'profiles', 'ai']); await post(app, 'demo', withCondition({ coding: [{ system: 'http://loinc.org', code: 'LA-mod', display: 'Moderate' }] })); await mod.processQueue(); const row = mod.store.read(1); @@ -1873,7 +2005,8 @@ describe('the editor', () => { expect(page.status).toBe(200); expect(page.text).toContain('

    Compose a document

    '); expect(page.text).toContain('
    '); - expect(page.text).toContain(''); + // with its version, so a changed script isn't taken from a browser's cache + expect(page.text).toMatch(/