fhirsmith 0.14.2 → 0.15.1

package/CHANGELOG.md

@@ -5,6 +5,61 @@ All notable changes to Health Intersections FHIRsmith will be documented in this
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [0.15.1] - 2026-10-15
+
+### Security
+
+- Packages: the SSRF guard on the crawler's outbound requests could be bypassed with an IP address in the URL (`http://127.0.0.1/`, `http://169.254.169.254/`, `http://[::1]/`), because Node makes no DNS lookup for an IP address and the guard only ran there. IP addresses are now checked too
+
+### Added
+
+- Packages: `/packages/npm-check` checks any registry against the npm registry protocol, the way npm clients use it (search by name, package document, version document, tarball), and reports each check with the reason for any failure. In a browser it is a form; otherwise it returns the report as JSON. The checks also run once a day against registry.npmjs.org, and the page says if any fail there. Requests are limited in time and size, may not go to non-public addresses other than loopback (so a server can check itself), and are rate limited per client. `npmCheck` in the packages config sets the limits; `"enabled": false` turns it off
+- Packages: npm-protocol test suites that run the same checks against registry.npmjs.org (the control: if npm changes its behaviour, this is where it shows), and against the packages module with known content
+- Packages: the package document has npm's `time` (each version's publication date, plus `created` and `modified`)
+- Packages: `latest` can be used in place of a version (`/packages/{id}/latest`)
+- Packages: `packages/dedupe-versions.js`, a one-off repair for an existing database: removes duplicate id#version rows, rewrites mirror files that don't match the database, and adds the unique index. It needs nothing but Node 24; see Repairing an older database in packages/readme.md
+
+### Changed
+
+- Packages: `GET /packages/{id}/{version}` returns npm's version document (JSON) when the request's `Accept` asks for `application/json` (or `application/vnd.npm.install-v1+json`), and the tarball otherwise. On the npm registry this URL is always the version document, and npm clients and tools that treat this server as an npm registry (including the HL7 Java tools, when they are configured with a server of npm type) ask for it with `Accept: application/json` and then download from `dist.tarball`. But FHIR clients have always downloaded the tarball from this URL - the Java tools without sending an `Accept` header at all, browsers and curl with ones that don't ask for JSON - so the tarball stays the default, and only a request that explicitly asks for JSON gets the document. The risk is a client that downloads the tarball from here while asking for JSON (typically a generic HTTP wrapper that sets `Accept: application/json` on every request). For those, and as a guarantee for the Java tools, a request with a `FHIR-Package-Download` header (any value; the Java PackageClient sends `tarball`) always gets the tarball, whatever its `Accept`. The response carries `Vary: Accept, FHIR-Package-Download` so caches keep the two apart
+- Packages: `/packages/-/v1/search` puts the package whose id is the whole search text first, as npm does. `/packages/catalog` is unchanged
+- Packages: each id#version is stored once. Simplifier republishes packages that other feeds publish directly, so its copy is a secondary echo: a version already held is not fetched from Simplifier, and a version held only from Simplifier is replaced when it arrives directly from its publisher's feed. A second direct copy from a different GUID is logged as an error and ignored. The database enforces this with a unique index on (Id, Version); on an existing database the server logs a warning until `dedupe-versions.js` has been run
+- Packages: package downloads keep the bytes the server sent: a `.tgz` labelled `Content-Encoding: gzip` is no longer unzipped by the HTTP client
+
+### Fixed
+
+- Packages: the tarball served for some versions didn't match the shasum in the package document (46 of 6961 versions on packages2.fhir.org), so npm clients would reject them. Two causes: the same id#version stored twice from different feeds (the document showed one copy's hash, the mirror held the other), and a tarball that failed validation (e.g. a draft flagged `notForPublication`) overwriting the mirror file before it was rejected. The mirror file is now written only after the package has been validated and stored
+- Packages: packages from feeds that label `.tgz` files `Content-Encoding: x-gzip` (e.g. doc.mednet.swiss) failed with "incorrect header check"
+- Packages: the search, catalog and updates log lines showed `[object Object]` instead of the query
+- Packages: the crawler log showed `??` for an item skipped because another feed had it in the same crawl
+
+### Tx Conformance Statement
+
+FHIRsmith v0.15.1 passed all 3630 HL7 terminology service tests (modes tx.fhir.org+omop+general+snomed+mimetypes+icd-11+closure, tests v1.9.6, runner v7.0.1)
+
+## [0.15.0] - 2026-10-08
+
+### Added
+
+- Library YAML: a source can give the id prefix for the resources it loads (`npm:hl7.terminology=tho`), so their ids stay the same when sources are added, removed or reordered. Sources without one are numbered, as before. See Resource Ids in tx/README.md
+- `CapabilityStatement.software.releaseDate` (and the TerminologyCapabilities equivalent) is the date the running version was released, taken from its heading in CHANGELOG.md, which is now included in the Docker image. A snapshot reports the time the server started
+- Registry: the crawler records each server's `software.releaseDate`. The Software page uses it to date software other than FHIRsmith, and FHIRsmith releases (including development builds) that the release list doesn't know about yet
+
+### Changed
+
+- **Breaking - resource ids:** every CodeSystem, ValueSet and ConceptMap is served with an id in the id space of its source - `prefix-id`, e.g. `CodeSystem/tho-v3-ActCode` - and the FHIR core package each endpoint loads uses `core` (`CodeSystem/core-administrative-gender`). An unprefixed id is not found by read or by any `[type]/[id]/$operation`. Library CodeSystems used to keep the ids they had in their packages (ValueSets and ConceptMaps were already numbered), and so could clash with the endpoint's core CodeSystems
+- THO overrides the FHIR core packages the same way the Java validator does: where hl7.terminology has a url, the core package's CodeSystems, ValueSets and ConceptMaps with that url are not loaded at all. Older core versions used to stay available by `url|version`, so the server reported versions the validator doesn't know (e.g. v3-MaritalStatus `2018-08-12`). The exception is the R4 v2 tables 0006, 0360 and 0391, which R4 core has in two versions each with the version in the url: those versions stay available, and THO's is the default
+- `software.releaseDate` is no longer the time of the request
+
+### Fixed
+
+- Search: two CodeSystems with the same id from different packages (e.g. v3-MaritalStatus from hl7.fhir.r4.core and hl7.terminology) linked to the same resource, and a read of that id always returned the core one
+- On the R4 endpoint, a ValueSet asked for without a version (e.g. `http://terminology.hl7.org/ValueSet/v3-MaritalStatus`) was the core package's copy rather than hl7.terminology's
+
+### Tx Conformance Statement
+
+FHIRsmith v0.15.0 passed all 3585 HL7 terminology service tests (modes tx.fhir.org+omop+general+snomed+mimetypes+icd-11+closure, tests v1.9.6, runner v7.0.0)
+
 ## [0.14.2] - 2026-10-06
 
 ### Security
@@ -596,7 +651,7 @@ FHIRsmith passed all 2729 HL7 terminology service tests (modes tx.fhir.org+omop+
 
 FHIRsmith passed all 2729 HL7 terminology service tests (modes tx.fhir.org+omop+general+snomed, tests v1.9.2, runner v6.9.12)
 
-## [v0.11.0] - 2026-mm-dd
+## [v0.11.0] - 2026-07-30
 
 ### Added
 

package/config-template.json

@@ -74,6 +74,16 @@
       "crawler": {
         "enabled": true,
         "schedule": "0 * * * *"
+      },
+      // /packages/npm-check: checks any registry against the npm registry protocol. On by
+      // default; the other settings are the defaults. "control": false stops the daily
+      // check against the npm registry itself.
+      "npmCheck": {
+        "enabled": true,
+        "maxConcurrentRuns": 2,
+        "runsPerWindow": 10,
+        "requestTimeout": 30000,
+        "maxDownloadMB": 50
       }
     },
     "registry": {

package/library/changelog.js

@@ -0,0 +1,57 @@
+// library/changelog.js
+// Release dates from CHANGELOG.md. A release is dated in its heading when it is made -
+// "## [0.14.2] - 2026-10-06" (or "## [v0.13.4] - ...") - so the changelog that ships with
+// the code is where a server finds out when its own version was released. A snapshot's
+// heading has no date yet, so it has no release date.
+
+const fs = require('fs');
+const path = require('path');
+
+const DEFAULT_PATH = path.join(__dirname, '..', 'CHANGELOG.md');
+const HEADING = /^##\s*\[v?(\d+\.\d+\.\d+)\]\s*-\s*(\d{4}-\d{2}-\d{2})\s*$/gm;
+
+/**
+ * @param {string} [changelogPath]
+ * @returns {Map<string, string>} version (no leading v) -> release date (YYYY-MM-DD).
+ *   Throws if the file can't be read.
+ */
+function readReleaseDates(changelogPath = DEFAULT_PATH) {
+  const text = fs.readFileSync(changelogPath, 'utf8');
+  const dates = new Map();
+  let m;
+  HEADING.lastIndex = 0;
+  while ((m = HEADING.exec(text)) !== null) {
+    if (!dates.has(m[1])) {
+      dates.set(m[1], m[2]);
+    }
+  }
+  return dates;
+}
+
+/**
+ * The release date of a version, or null if it isn't a dated release (a snapshot, or no
+ * changelog to read)
+ */
+function releaseDateOf(version, changelogPath = DEFAULT_PATH) {
+  try {
+    return readReleaseDates(changelogPath).get(String(version || '').replace(/^v/, '')) || null;
+  } catch (e) {
+    return null;
+  }
+}
+
+/**
+ * The release date a server reports for its version: the date of the release, or, for a
+ * version that hasn't been released (a snapshot), the time the server started - which is
+ * when that build went into service.
+ *
+ * @param {string} version
+ * @param {Date} started - when the server started
+ * @param {string} [changelogPath]
+ * @returns {string} YYYY-MM-DD for a release, an ISO dateTime for a snapshot
+ */
+function reportedReleaseDate(version, started, changelogPath = DEFAULT_PATH) {
+  return releaseDateOf(version, changelogPath) || started.toISOString();
+}
+
+module.exports = { readReleaseDates, releaseDateOf, reportedReleaseDate };

package/library/npm-audit.js

@@ -6,6 +6,9 @@
 // The package list comes from node_modules/.package-lock.json (what is actually
 // installed), falling back to package-lock.json. Dev dependencies are left out, and
 // fhirsmith itself is included, so an advisory published against fhirsmith shows up too.
+// A server installed with `npm install -g fhirsmith` has neither - npm doesn't publish the
+// lock file, and keeps no hidden one for a global package - so then the package list is read
+// from the package.json files in node_modules (a published install has no dev dependencies).
 //
 // Config (all optional), top level of config.json:
 //   "npmAudit": { "enabled": true, "intervalHours": 24 }
@@ -67,9 +70,6 @@ class NpmAudit {
         // try the next one
       }
     }
-    if (!lock || !lock.packages) {
-      throw new Error('No package-lock.json found to audit');
-    }
     const found = new Map();
     const add = (name, version) => {
       if (!found.has(name)) {
@@ -77,13 +77,17 @@ class NpmAudit {
       }
       found.get(name).add(version);
     };
-    for (const [key, info] of Object.entries(lock.packages)) {
-      if (!key || !info || info.dev || info.link || !info.version) {
-        continue;
+    if (lock && lock.packages) {
+      for (const [key, info] of Object.entries(lock.packages)) {
+        if (!key || !info || info.dev || info.link || !info.version) {
+          continue;
+        }
+        const marker = 'node_modules/';
+        const name = info.name || key.substring(key.lastIndexOf(marker) + marker.length);
+        add(name, info.version);
       }
-      const marker = 'node_modules/';
-      const name = info.name || key.substring(key.lastIndexOf(marker) + marker.length);
-      add(name, info.version);
+    } else if (!this.scanInstalled(path.join(this.appDir, 'node_modules'), add)) {
+      throw new Error('No package-lock.json or node_modules found to audit');
     }
     try {
       const pkg = JSON.parse(fs.readFileSync(path.join(this.appDir, 'package.json'), 'utf8'));
@@ -100,6 +104,55 @@ class NpmAudit {
     return result;
   }
 
+  /**
+   * Read the installed packages from the package.json files under a node_modules folder,
+   * including scoped packages and the node_modules nested inside packages. Linked packages
+   * (symlinks) are left out, as they are from a lock file.
+   *
+   * @returns {boolean} false if there is no node_modules folder
+   */
+  scanInstalled(dir, add) {
+    let entries;
+    try {
+      entries = fs.readdirSync(dir, { withFileTypes: true });
+    } catch (e) {
+      return false;
+    }
+    for (const entry of entries) {
+      if (!entry.isDirectory() || entry.name.startsWith('.')) {
+        continue;
+      }
+      const dirs = [];
+      if (entry.name.startsWith('@')) {
+        let scoped = [];
+        try {
+          scoped = fs.readdirSync(path.join(dir, entry.name), { withFileTypes: true });
+        } catch (e) {
+          // unreadable scope folder - nothing in it
+        }
+        for (const s of scoped) {
+          if (s.isDirectory()) {
+            dirs.push(path.join(dir, entry.name, s.name));
+          }
+        }
+      } else {
+        dirs.push(path.join(dir, entry.name));
+      }
+      for (const pkgDir of dirs) {
+        try {
+          const pkg = JSON.parse(fs.readFileSync(path.join(pkgDir, 'package.json'), 'utf8'));
+          if (pkg.name && pkg.version) {
+            add(pkg.name, pkg.version);
+          }
+        } catch (e) {
+          // not a package
+        }
+        this.scanInstalled(path.join(pkgDir, 'node_modules'), add);
+      }
+    }
+    return true;
+  }
+
   async run() {
     if (this.running) {
       return;

package/package.json

@@ -1,7 +1,7 @@
 {
   "name": "fhirsmith",
-  "version": "0.14.2",
-  "txVersion": "1.9.5-SNAPSHOT",
+  "version": "0.15.1",
+  "txVersion": "1.9.6",
   "description": "A Node.js server that provides a collection of tools to serve the FHIR ecosystem",
   "main": "server.js",
   "engines": {

package/packages/npm-check-page.js

@@ -0,0 +1,108 @@
+//
+// Copyright 2026, Health Intersections Pty Ltd (http://www.healthintersections.com.au)
+//
+// Licensed under BSD-3: https://opensource.org/license/bsd-3-clause
+//
+
+// HTML for /packages/npm-check: the form, and the report of a run (see npm-check.js)
+
+const escape = require('escape-html');
+
+const PASS = '<span style="color: #198754; font-weight: bold">&#10003;</span>';
+const FAIL = '<span style="color: #dc3545; font-weight: bold">&#10007;</span>';
+
+function when(iso) {
+  return escape(new Date(iso).toUTCString());
+}
+
+function form(values) {
+  let html = '<form method="get" action="/packages/npm-check" class="mb-4">';
+  html += '<table class="grid">';
+  html += `<tr><td><label for="registry">Registry URL</label></td><td><input type="url" id="registry" name="registry" size="60" required maxlength="500" placeholder="https://registry.example.org/packages" value="${escape(values.registry || '')}"></td></tr>`;
+  html += `<tr><td><label for="package">Package</label></td><td><input type="text" id="package" name="package" size="40" required maxlength="100" value="${escape(values.package || 'hl7.fhir.r4.core')}"></td></tr>`;
+  html += `<tr><td><label for="version">Version</label></td><td><input type="text" id="version" name="version" size="20" required maxlength="50" value="${escape(values.version || '4.0.1')}"></td></tr>`;
+  html += '</table>';
+  html += '<p class="mt-2"><button type="submit" class="btn btn-primary">Run the checks</button> ';
+  html += '<small>A run makes about ten requests to the registry, including downloading the package\'s tarball, and can take up to a minute.</small></p>';
+  html += '</form>';
+  return html;
+}
+
+function controlSummary(control, controlReport) {
+  if (!control) {
+    return '';
+  }
+  const what = `${escape(control.registry)} (${escape(control.pkg)} ${escape(control.version)})`;
+  if (!controlReport) {
+    return `<p><small>The checks are verified daily against the npm registry itself, ${what}. That hasn't run yet; reload this page in a minute to see it.</small></p>`;
+  }
+  const total = controlReport.passed + controlReport.failed;
+  if (controlReport.failed === 0) {
+    return `<p><small>${PASS} The checks are verified daily against the npm registry itself: all ${total} passed against ${what} on ${when(controlReport.finished)}.</small></p>`;
+  }
+  let html = `<div class="alert alert-warning"><p>${FAIL} The checks are verified daily against the npm registry itself, and on ${when(controlReport.finished)} ${controlReport.failed} of ${total} failed against ${what}. The npm registry is the reference, so those checks may be out of date - treat their results below with caution:</p><ul>`;
+  for (const step of controlReport.steps) {
+    for (const c of step.checks.filter(x => x.status === 'fail')) {
+      html += `<li>${escape(step.title)}: ${escape(c.title)} - ${escape(c.message)}</li>`;
+    }
+  }
+  return html + '</ul></div>';
+}
+
+function intro() {
+  let html = '<p>Checks a registry against the npm registry protocol, the way an npm client uses it: search for a package by name, ';
+  html += 'read its package document, read one version\'s document, and download that version\'s tarball. ';
+  html += 'At each step it checks the parts of the response that npm clients depend on. ';
+  html += 'Give the registry\'s base URL (the one you would pass to <code>npm --registry</code>) and a package version it holds.</p>';
+  return html;
+}
+
+function renderForm({ values = {}, control, controlReport }) {
+  return intro() + form(values) + controlSummary(control, controlReport);
+}
+
+function renderReport({ report, control, controlReport }) {
+  const total = report.passed + report.failed;
+  let html = '';
+  html += `<p><strong>Registry:</strong> ${escape(report.registry)}<br>`;
+  html += `<strong>Package:</strong> ${escape(report.package)} ${escape(report.version)}<br>`;
+  html += `<strong>Run:</strong> ${when(report.started)}</p>`;
+  if (report.failed === 0) {
+    html += `<div class="alert alert-success">${PASS} All ${total} checks passed.</div>`;
+  } else {
+    html += `<div class="alert alert-danger">${FAIL} ${report.failed} of ${total} checks failed.</div>`;
+  }
+  html += controlSummary(control, controlReport);
+
+  for (const step of report.steps) {
+    html += `<h4 class="mt-4">${escape(step.title)}: <code>${escape(step.request)}</code></h4>`;
+    html += '<table class="grid">';
+    for (const c of step.checks) {
+      html += `<tr><td>${c.status === 'pass' ? PASS : FAIL}</td><td>${escape(c.title)}`;
+      if (c.status !== 'pass') {
+        html += `<br><span style="color: #dc3545">${escape(c.message)}</span>`;
+        const urls = [...new Set(c.uses.map(u => report.requests[u]?.url).filter(Boolean))];
+        if (urls.length > 0) {
+          html += `<br><small>${urls.map(escape).join('<br>')}</small>`;
+        }
+      }
+      html += '</td></tr>';
+    }
+    html += '</table>';
+  }
+
+  html += '<h4 class="mt-4">Requests</h4>';
+  html += '<table class="grid"><tr><td><b>URL</b></td><td><b>Result</b></td></tr>';
+  for (const r of Object.values(report.requests)) {
+    const result = r.error ? `<span style="color: #dc3545">${escape(r.error)}</span>`
+      : `${escape(String(r.status))} ${escape(r.contentType || '')} (${escape(String(r.bytes))} bytes)`;
+    html += `<tr><td><small>${escape(r.url)}</small></td><td><small>${result}</small></td></tr>`;
+  }
+  html += '</table>';
+
+  html += '<h4 class="mt-4">Check again</h4>';
+  html += form({ registry: report.registry, package: report.package, version: report.version });
+  return html;
+}
+
+module.exports = { renderForm, renderReport };