fhirsmith 0.15.1 → 0.16.1

package/CHANGELOG.md

@@ -5,6 +5,42 @@ All notable changes to Health Intersections FHIRsmith will be documented in this
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [0.16.1] - 2026-10-16
+
+### Added
+
+- The dashboard always says which FHIRsmith release the server runs, and whether a newer one is available (looked up once a day with the npm audit check; before, the latest release was only looked up, and only mentioned, when the audit found something)
+- README: how to follow releases (GitHub release emails, the releases Atom feed, and the dashboard)
+
+### Fixed
+
+- Packages: SUSHI (and anything else using fhir-package-loader) couldn't download packages from this server: `GET /packages/{id}/{version}` returned the version's JSON document instead of the tarball (since 0.16.0). The rule was "JSON when Accept includes `application/json`", but fhir-package-loader downloads with axios's default `Accept: application/json, text/plain, */*`, and plenty of other HTTP clients send JSON alongside `*/*` by default without wanting it. The version document is now returned only when the request prefers JSON: a JSON type ranked above any `*/*`, `application/*` or tarball type by q-value (`application/json` alone, or `application/json, */*;q=0.1`). Everything else gets the tarball, and `FHIR-Package-Download` still forces it
+- A POST with no Content-Type now says so (it gave "Unsupported Media Type: " with nothing after it)
+- A form post whose Content-Type has a charset (`application/x-www-form-urlencoded; charset=UTF-8`) was refused as an unsupported media type
+- Links in published IGs to a resource by the id it had in its source (`ValueSet/2.16.840.1.113762.1.4.1`, from before ids were prefixed, as in `ValueSet/vsac-2.16.840.1.113762.1.4.1`) work again in a browser: a read of an id nothing has redirects to the one resource that had that id in its source, or lists them when there are several. Other requests still get a 404
+
+### Tx Conformance Statement
+
+FHIRsmith v0.16.1 passed all 3630 HL7 terminology service tests (modes tx.fhir.org+omop+general+snomed+mimetypes+icd-11+closure, tests v1.9.6, runner v7.0.2)
+
+## [0.16.0] - 2026-10-15
+
+### Security
+- 
+- Token: the login session store is now FHIRsmith's own (`library/sqlite-session-store.js`), in place of connect-sqlite3. connect-sqlite3 depends on sqlite3 5, which brought node-gyp 8, tar 6 and @tootallnate/once into every npm install of FHIRsmith (12 advisories, one critical). The `overrides` in package.json kept them out of a clone + `npm ci` (and so out of the Docker image and the repo's own `npm audit`), but npm ignores overrides when installing the published package, so `npm install -g fhirsmith` still got them. Sessions are kept in the same `sessions` table of the token database, in the same layout, so existing logins carry over
+- Dependencies: fhirpath 5 (its ucum-lhc no longer brings csv-parse 4) and node-cron 4 (no longer depends on uuid), for the same reason: the csv-parse and uuid overrides didn't reach npm installs. The csv-parse override is gone
+- Dependencies: natural 8 (uuid 13). Its stemmers are unchanged - every stemmer we use gives the same stem as natural 6 on ~1.6M words (RxNorm, LOINC's translations, and English, French, Spanish, Italian and Portuguese word lists), so existing SNOMED caches and RxNorm databases don't need rebuilding. FHIRsmith now loads only `natural/lib/natural/stemmers`: the package index also loads natural's storage backends, which run dotenv (reading any `.env` in the working directory into the environment, and in natural 8 printing a banner to stdout)
+- Development: jest 30 and jest-junit 17, and `npm run dev` uses Node's own `node --watch` in place of nodemon. With these, `npm audit` is clean for the dev tooling too (jest 29 and nodemon brought braces, and jest's coverage tooling sprintf-js, neither of which has a fixed version). The package.json overrides are down to one, js-yaml 4 for @istanbuljs/load-nyc-config, which only affects the dev install; the rest were no longer doing anything
+- The PR pipeline fails on high/critical advisories anywhere in the dependency tree, on moderate ones in what ships, and audits a fresh install of the packed package (installed the way users install it, so the repo's overrides don't apply)
+
+### Changed
+
+- The npm audit self-check looks up the latest FHIRsmith release, and only tells the administrator to upgrade when there is a newer release to upgrade to; a server already on the latest release says so instead
+
+### Tx Conformance Statement
+
+FHIRsmith v0.16.0 passed all 3630 HL7 terminology service tests (modes tx.fhir.org+omop+general+snomed+mimetypes+icd-11+closure, tests v1.9.6, runner v7.0.1)
+
 ## [0.15.1] - 2026-10-15
 
 ### Security

package/README.md

@@ -259,7 +259,7 @@ Each GitHub Release includes:
     - `ghcr.io/healthintersections/fhirsmith:latest`
     - `ghcr.io/healthintersections/fhirsmith:vX.Y.Z`
     - `ghcr.io/healthintersections/fhirsmith:X.Y.Z`
-- **npm package** published to npmjs.org as `fhirsmith` *(if you add this)*
+- **npm package** published to npmjs.org as [`fhirsmith`](https://www.npmjs.com/package/fhirsmith)
 
 Release images carry OCI labels, so an image says which release it is without
 having to trust the tag it was pulled under:
@@ -271,6 +271,19 @@ docker inspect --format '{{json .Config.Labels}}' ghcr.io/healthintersections/fh
 `org.opencontainers.image.version` is the release, `.revision` the commit it was
 built from, and `.source` is what links the package to this repository on GHCR.
 
+### Tracking Releases
+
+Server administrators should follow releases: a server is only as safe as the
+release it runs.
+
+- **Email**: on the [GitHub repository](https://github.com/HealthIntersections/FHIRsmith),
+  choose **Watch → Custom → Releases**. Each release is emailed with its notes (needs a GitHub account)
+- **Atom feed**: `https://github.com/HealthIntersections/FHIRsmith/releases.atom` - each
+  entry carries the release notes, for a feed reader or a Slack/Teams feed integration
+- **The server itself**: once a day the server looks up the latest release on npm, and the
+  dashboard says whether a newer one is available (this is part of the npm audit check, so
+  `npmAudit.enabled = false` turns it off too)
+
 ### CI Images
 
 Every push to main also publishes `:cibuild` and `:cibuild-<sha>` to the same

package/gr.sh

@@ -0,0 +1,3 @@
+#!/usr/bin/env bash 
+git tag v$1
+git push origin v$1

package/library/npm-audit.js

@@ -12,6 +12,12 @@
 //
 // Config (all optional), top level of config.json:
 //   "npmAudit": { "enabled": true, "intervalHours": 24 }
+//
+// Each check also looks up the latest fhirsmith release on npm. The dashboard always says
+// whether a newer release is available, and the advice about any findings says whether
+// upgrading will help. When this server is already on the latest release it won't:
+// npm ignores our package.json "overrides" for anything but a clone + npm ci, so an
+// npm-installed server can carry transitive versions the repo itself has overridden away.
 
 const fs = require('fs');
 const path = require('path');
@@ -19,6 +25,7 @@ const axios = require('axios');
 const escape = require('escape-html');
 
 const DEFAULT_URL = 'https://registry.npmjs.org/-/npm/v1/security/advisories/bulk';
+const DEFAULT_LATEST_URL = 'https://registry.npmjs.org/fhirsmith/latest';
 const SEVERITIES = ['critical', 'high', 'moderate', 'low', 'info'];
 const HOUR_MS = 60 * 60 * 1000;
 const FIRST_RUN_DELAY_MS = 60 * 1000; // let startup finish first
@@ -38,11 +45,34 @@ function describeAgo(time, now = Date.now()) {
   return `${Math.floor(hours / 24)} days ago`;
 }
 
+/**
+ * Compare two x.y.z versions numerically (pre-release and build suffixes ignored).
+ * Returns <0, 0 or >0, or null if either can't be read
+ */
+function compareVersions(a, b) {
+  const parse = v => {
+    const m = /^v?(\d+)\.(\d+)\.(\d+)/.exec(String(v || '').trim());
+    return m ? m.slice(1).map(Number) : null;
+  };
+  const x = parse(a);
+  const y = parse(b);
+  if (!x || !y) {
+    return null;
+  }
+  for (let i = 0; i < 3; i++) {
+    if (x[i] !== y[i]) {
+      return x[i] - y[i];
+    }
+  }
+  return 0;
+}
+
 class NpmAudit {
   constructor(config = {}, logger = null, appDir = path.join(__dirname, '..'), stats = null) {
     this.enabled = config.enabled !== false;
     this.intervalHours = config.intervalHours > 0 ? config.intervalHours : 24;
     this.url = config.url || DEFAULT_URL;
+    this.latestUrl = config.latestUrl || DEFAULT_LATEST_URL;
     this.timeout = config.timeout || 60000;
     this.logger = logger;
     this.appDir = appDir;
@@ -55,6 +85,8 @@ class NpmAudit {
     this.findings = []; // [{name, installed: [versions], id, severity, title, url, range}]
     this.lastError = null; // message from the last attempt, if it failed
     this.lastAttempt = null;
+    this.installedVersion = null; // this server's fhirsmith version, from package.json
+    this.latestVersion = null; // the latest fhirsmith release on npm, if it could be found
   }
 
   /**
@@ -93,6 +125,7 @@ class NpmAudit {
       const pkg = JSON.parse(fs.readFileSync(path.join(this.appDir, 'package.json'), 'utf8'));
       if (pkg.name && pkg.version) {
         add(pkg.name, pkg.version);
+        this.installedVersion = pkg.version;
       }
     } catch (e) {
       // no package.json - nothing to add
@@ -153,6 +186,74 @@ class NpmAudit {
     return true;
   }
 
+  /**
+   * This server's own version, from package.json (collectPackages reads it too, but that
+   * can fail for reasons that have nothing to do with the version)
+   */
+  readInstalledVersion() {
+    try {
+      const pkg = JSON.parse(fs.readFileSync(path.join(this.appDir, 'package.json'), 'utf8'));
+      if (pkg.version) {
+        this.installedVersion = pkg.version;
+      }
+    } catch (e) {
+      // leave it as it was
+    }
+  }
+
+  /**
+   * Dashboard line: this server's version, and whether there is a newer release
+   */
+  renderRelease() {
+    if (!this.installedVersion) {
+      return '';
+    }
+    let html = `<p><strong>FHIRsmith release:</strong> this server runs ${escape(this.installedVersion)}`;
+    const cmp = compareVersions(this.installedVersion, this.latestVersion);
+    if (cmp === null) {
+      html += ' (the latest release could not be checked)';
+    } else if (cmp < 0) {
+      html += ` - <span style="color:#b60">FHIRsmith ${escape(this.latestVersion)} is available</span>` +
+        ' (<a href="https://github.com/HealthIntersections/FHIRsmith/releases">release notes</a>)';
+    } else {
+      html += ', the latest release';
+    }
+    return html + '</p>';
+  }
+
+  /**
+   * Look up the latest fhirsmith release. A failure only means the advice is less specific,
+   * so it is logged and otherwise ignored (the previous answer is kept)
+   */
+  async checkLatest() {
+    try {
+      const response = await axios.get(this.latestUrl, { timeout: this.timeout, headers: { 'Accept': 'application/json' } });
+      const version = response.data && response.data.version;
+      if (typeof version === 'string' && compareVersions(version, version) === 0) {
+        this.latestVersion = version;
+      }
+    } catch (error) {
+      if (this.logger) {
+        this.logger.info('npm audit: could not look up the latest FHIRsmith release: ' + error.message);
+      }
+    }
+  }
+
+  /**
+   * What the administrator can do about the findings
+   */
+  advice() {
+    const cmp = compareVersions(this.installedVersion, this.latestVersion);
+    if (cmp === null) {
+      return 'The server administrator should upgrade to the latest FHIRsmith release.';
+    }
+    if (cmp < 0) {
+      return `The server administrator should upgrade to FHIRsmith ${this.latestVersion} (this server is running ${this.installedVersion}).`;
+    }
+    return `This server is already running the latest FHIRsmith release (${this.installedVersion}), ` +
+      'so these will be fixed by a future FHIRsmith release.';
+  }
+
   async run() {
     if (this.running) {
       return;
@@ -162,6 +263,9 @@ class NpmAudit {
     if (this.stats) {
       this.stats.task('npm audit', 'Checking');
     }
+    // independent of the audit: failures here are logged and otherwise ignored
+    this.readInstalledVersion();
+    await this.checkLatest();
     try {
       const packages = this.collectPackages();
       const response = await axios.post(this.url, packages, {
@@ -278,7 +382,7 @@ class NpmAudit {
     return `<div class="alert ${cls}" role="alert"><strong>Security:</strong> npm audit found ` +
       `${escape(this.summary())} in the packages this server uses ` +
       `(checked ${escape(describeAgo(this.checkedAt.getTime(), now))}). ` +
-      'The server administrator should upgrade to the latest FHIRsmith release. ' +
+      `${escape(this.advice())} ` +
       '<a href="/dashboard#npm-audit">Details</a></div>';
   }
 
@@ -290,6 +394,7 @@ class NpmAudit {
     if (!this.enabled) {
       return html + '<p><strong>npm audit:</strong> disabled (npmAudit.enabled = false)</p></div>';
     }
+    html += this.renderRelease();
     if (!this.checkedAt) {
       html += '<p><strong>npm audit:</strong> ';
       html += this.lastError ? `<span style="color:#b00">could not be run: ${escape(this.lastError)}</span>` : 'not run yet';
@@ -303,6 +408,7 @@ class NpmAudit {
     }
     html += '</p>';
     if (this.findings.length > 0) {
+      html += `<p>${escape(this.advice())}</p>`;
       html += '<table class="grid"><tr><th>Severity</th><th>Package</th><th>Installed</th><th>Vulnerable</th><th>Advisory</th></tr>';
       for (const f of this.findings) {
         const link = /^https:\/\//.test(f.url) ? `<a href="${escape(f.url)}">${escape(f.title || f.url)}</a>` : escape(f.title);
@@ -315,4 +421,4 @@ class NpmAudit {
   }
 }
 
-module.exports = { NpmAudit, describeAgo };
+module.exports = { NpmAudit, describeAgo, compareVersions };

package/library/sqlite-session-store.js

@@ -0,0 +1,133 @@
+// library/sqlite-session-store.js
+// An express-session store kept in a table of an already-open sqlite3 database.
+//
+// This replaces connect-sqlite3, which depends on sqlite3 ^5 and so brought a second, old
+// copy of sqlite3 (and with it node-gyp 8 and tar 6) into every npm install of FHIRsmith.
+// Our package.json "overrides" hid that from npm audit in the repo, but npm ignores
+// overrides for anyone installing the published package.
+//
+// The table layout is the one connect-sqlite3 used - (sid PRIMARY KEY, expired, sess),
+// with expired in epoch ms and sess as JSON - so sessions in an existing database carry over.
+
+const session = require('express-session');
+
+const ONE_DAY_MS = 24 * 60 * 60 * 1000;
+const TABLE_NAME = /^[A-Za-z_][A-Za-z0-9_]*$/;
+
+class SqliteSessionStore extends session.Store {
+  /**
+   * @param {object} options
+   * @param {object} options.db - an open sqlite3 Database; the caller owns it and closes it
+   * @param {string} [options.table='sessions']
+   * @param {number} [options.cleanupIntervalMs=one day] - how often expired sessions are deleted
+   */
+  constructor(options = {}) {
+    super();
+    if (!options.db) {
+      throw new Error('SqliteSessionStore needs an open sqlite3 database');
+    }
+    const table = options.table || 'sessions';
+    if (!TABLE_NAME.test(table)) {
+      throw new Error('Invalid session table name: ' + table);
+    }
+    this.db = options.db;
+    this.table = table;
+    this.timer = null;
+    this.ready = this.run(`CREATE TABLE IF NOT EXISTS ${table} (sid PRIMARY KEY, expired, sess)`)
+      .then(() => this.cleanup());
+    this.ready.catch(() => { /* reported by the first operation that waits on it */ });
+    const interval = options.cleanupIntervalMs > 0 ? options.cleanupIntervalMs : ONE_DAY_MS;
+    this.timer = setInterval(() => void this.cleanup().catch(() => {}), interval);
+    if (this.timer.unref) {
+      this.timer.unref();
+    }
+  }
+
+  run(sql, params = []) {
+    return new Promise((resolve, reject) => {
+      this.db.run(sql, params, function (err) {
+        if (err) {
+          reject(err);
+        } else {
+          resolve(this);
+        }
+      });
+    });
+  }
+
+  query(method, sql, params = []) {
+    return new Promise((resolve, reject) => {
+      this.db[method](sql, params, (err, result) => err ? reject(err) : resolve(result));
+    });
+  }
+
+  // adapt a promise to express-session's callback style
+  done(promise, cb) {
+    promise.then(result => cb && cb(null, result), err => cb && cb(err));
+  }
+
+  cleanup() {
+    return this.run(`DELETE FROM ${this.table} WHERE expired < ?`, [Date.now()]);
+  }
+
+  static expiry(sess) {
+    if (sess && sess.cookie) {
+      if (sess.cookie.expires) {
+        const t = new Date(sess.cookie.expires).getTime();
+        if (!isNaN(t)) {
+          return t;
+        }
+      }
+      if (sess.cookie.maxAge) {
+        return Date.now() + sess.cookie.maxAge;
+      }
+    }
+    return Date.now() + ONE_DAY_MS;
+  }
+
+  get(sid, cb) {
+    this.done(this.ready
+      .then(() => this.query('get', `SELECT sess FROM ${this.table} WHERE sid = ? AND expired >= ?`, [sid, Date.now()]))
+      .then(row => row ? JSON.parse(row.sess) : null), cb);
+  }
+
+  set(sid, sess, cb) {
+    this.done(this.ready
+      .then(() => this.run(`INSERT OR REPLACE INTO ${this.table} (sid, expired, sess) VALUES (?, ?, ?)`,
+        [sid, SqliteSessionStore.expiry(sess), JSON.stringify(sess)]))
+      .then(() => undefined), cb);
+  }
+
+  touch(sid, sess, cb) {
+    this.done(this.ready
+      .then(() => this.run(`UPDATE ${this.table} SET expired = ? WHERE sid = ? AND expired >= ?`,
+        [SqliteSessionStore.expiry(sess), sid, Date.now()]))
+      .then(() => undefined), cb);
+  }
+
+  destroy(sid, cb) {
+    this.done(this.ready
+      .then(() => this.run(`DELETE FROM ${this.table} WHERE sid = ?`, [sid]))
+      .then(() => undefined), cb);
+  }
+
+  length(cb) {
+    this.done(this.ready
+      .then(() => this.query('get', `SELECT COUNT(*) AS count FROM ${this.table} WHERE expired >= ?`, [Date.now()]))
+      .then(row => row.count), cb);
+  }
+
+  clear(cb) {
+    this.done(this.ready.then(() => this.run(`DELETE FROM ${this.table}`)).then(() => undefined), cb);
+  }
+
+  /** Stop the cleanup timer. The database itself belongs to the caller. */
+  close() {
+    if (this.timer) {
+      clearInterval(this.timer);
+      this.timer = null;
+    }
+  }
+}
+
+module.exports = { SqliteSessionStore };

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "fhirsmith",
-  "version": "0.15.1",
+  "version": "0.16.1",
   "txVersion": "1.9.6",
   "description": "A Node.js server that provides a collection of tools to serve the FHIR ecosystem",
   "main": "server.js",
@@ -9,18 +9,18 @@
   },
   "scripts": {
     "start": "node server.js",
-    "dev": "nodemon server.js",
+    "dev": "node --watch server.js",
     "test": "jest",
     "test:watch": "jest --watch",
     "test:coverage": "jest --coverage",
     "test:ci": "jest --ci --coverage --watchAll=false --reporters=default --reporters=jest-junit --verbose",
-    "test:shl": "jest --testPathPattern=shl",
-    "test:vcl": "jest --testPathPattern=vcl",
-    "test:xig": "jest --testPathPattern=xig",
-    "test:packages": "jest --testPathPattern=packages",
-    "test:token": "jest --testPathPattern=token",
-    "test:integration": "jest --testPathPattern=integration",
-    "test:unit": "jest --testPathPattern=unit",
+    "test:shl": "jest --testPathPatterns=shl",
+    "test:vcl": "jest --testPathPatterns=vcl",
+    "test:xig": "jest --testPathPatterns=xig",
+    "test:packages": "jest --testPathPatterns=packages",
+    "test:token": "jest --testPathPatterns=token",
+    "test:integration": "jest --testPathPatterns=integration",
+    "test:unit": "jest --testPathPatterns=unit",
     "lint": "eslint . --ext .js --ignore-path .gitignore",
     "lint:fix": "eslint . --fix --ignore-pattern 'static/assets/js/html5shiv.js'",
     "lint:relaxed": "eslint . --config .eslintrc.relaxed.js",
@@ -41,21 +41,20 @@
     "cli-progress": "^3.12.0",
     "commander": "^14.0.3",
     "commonmark": "^0.31.2",
-    "connect-sqlite3": "^0.9.18",
     "cors": "^2.8.6",
     "express": "^5.2.1",
     "express-rate-limit": "^7.4.1",
     "express-session": "^1.19.0",
     "fast-xml-parser": "^5.3.4",
     "fhir-validator-wrapper": "1.4.0",
-    "fhirpath": "^4.8.3",
+    "fhirpath": "^5.2.1",
     "fs-extra": "^11.3.3",
     "ini": "^6.0.0",
     "inquirer": "^8.2.5",
     "liquidjs": "^10.25.5",
     "lusca": "^1.7.0",
-    "natural": "^6.12.0",
-    "node-cron": "^3.0.3",
+    "natural": "^8.1.1",
+    "node-cron": "^4.6.0",
     "pako": "^2.1.0",
     "passport": "^0.7.0",
     "passport-facebook": "^3.0.0",
@@ -71,18 +70,17 @@
     "yaml": "^2.8.2"
   },
   "devDependencies": {
-    "@types/jest": "^29.5.8",
+    "@types/jest": "^30.0.0",
     "@typescript-eslint/eslint-plugin": "^8.54.0",
     "@typescript-eslint/parser": "^8.54.0",
     "ajv": "^8.20.0",
     "eslint": "^8.57.1",
     "eslint-plugin-promise": "^7.2.1",
-    "jest": "^29.7.0",
-    "jest-environment-node": "^29.7.0",
-    "jest-junit": "^16.0.0",
+    "jest": "^30.5.2",
+    "jest-environment-node": "^30.5.2",
+    "jest-junit": "^17.0.0",
     "mock-fs": "^5.5.0",
     "nock": "^13.4.0",
-    "nodemon": "^3.1.11",
     "supertest": "^7.2.2",
     "tmp": "^0.2.5"
   },
@@ -107,16 +105,8 @@
   "author": "Health Intersections Pty Ltd",
   "license": "BSD-3",
   "overrides": {
-    "tar": "^7.5.7",
-    "fast-xml-parser": "^5.3.4",
-    "inquirer": {
-      "lodash": "4.18.1"
-    },
-    "brace-expansion": "^5.0.8",
-    "csv-parse": "^7.0.3",
-    "uuid": "^11.1.1",
-    "connect-sqlite3": {
-      "sqlite3": "$sqlite3"
+    "@istanbuljs/load-nyc-config": {
+      "js-yaml": "^4.1.0"
     }
   },
   "bin": {

package/packages/openapi.yaml

@@ -350,9 +350,12 @@ paths:
       summary: Download a package tarball, or get a version's document
       description: |
         Returns the package tarball (`.tgz`) - or, when the request's `Accept` header
-        asks for `application/json` (or npm's `application/vnd.npm.install-v1+json`),
-        the version's npm document: the same object as that version's entry in the
-        package document. The response varies on `Accept` and `FHIR-Package-Download`.
+        prefers JSON, the version's npm document: the same object as that version's entry
+        in the package document. Preferring JSON means ranking `application/json` (or npm's
+        `application/vnd.npm.install-v1+json`) above any `*/*`, `application/*` or tarball
+        type by q-value: `application/json` alone does, but `application/json, text/plain, */*`
+        (axios's default, which SUSHI sends) does not. The response varies on `Accept` and
+        `FHIR-Package-Download`.
 
         A request with a `FHIR-Package-Download` header always gets the tarball. The
         HL7 Java tools send it; a client that downloads tarballs from this URL while

package/packages/packages.js

@@ -944,11 +944,13 @@ class PackagesModule {
     }
   }
 
-  stopCrawlerJob() {
+  // node-cron 4: stop() may return a promise
+  async stopCrawlerJob() {
     if (this.crawlerJob) {
       this.crawler.shutdown();
-      this.crawlerJob.stop();
+      const job = this.crawlerJob;
       this.crawlerJob = null;
+      await job.stop();
       pckLog.info('Package crawler job stopped');
     }
   }
@@ -1831,14 +1833,34 @@ class PackagesModule {
     res.send(htmlServer.renderPage('packages', 'npm Registry Check', content, stats));
   }
 
-  // GET /{id}/{version} is npm's version document when the request asks for JSON, and the
-  // tarball otherwise - which is what FHIR clients have always got there, without asking
-  // for JSON. A request carrying the TARBALL_HEADER always gets the tarball.
+  // GET /{id}/{version} is npm's version document when the request prefers JSON, and the
+  // tarball otherwise - which is what FHIR clients have always got there. Preferring JSON
+  // means ranking it above anything else the request accepts: a JSON type with a higher q
+  // than any */*, application/* or tarball type. Many clients send JSON together with */*
+  // at the same q - axios's default, application/json, text/plain, */*, which is what
+  // SUSHI (fhir-package-loader) sends when it downloads a tarball from here - and they get
+  // the tarball. A request carrying the TARBALL_HEADER always gets the tarball.
   wantsVersionDocument(req) {
     if (req.get(TARBALL_HEADER) !== undefined) {
       return false;
     }
-    return /application\/(json|vnd\.npm\.install-v1\+json)/i.test(req.get('accept') || '');
+    let json = 0;
+    let other = 0;
+    for (const part of String(req.get('accept') || '').split(',')) {
+      const [type, ...params] = part.split(';').map(s => s.trim().toLowerCase());
+      const qParam = params.find(p => p.startsWith('q='));
+      const q = qParam ? parseFloat(qParam.substring(2)) : 1;
+      if (!type || Number.isNaN(q)) {
+        continue;
+      }
+      if (type === 'application/json' || type === 'application/vnd.npm.install-v1+json') {
+        json = Math.max(json, q);
+      } else if (['*/*', 'application/*', 'application/tar+gzip', 'application/gzip', 'application/x-gzip',
+        'application/x-tar', 'application/octet-stream'].includes(type)) {
+        other = Math.max(other, q);
+      }
+    }
+    return json > 0 && json > other;
   }
 
   async serveVersionDocument(secure, id, version, req, res) {
@@ -2844,7 +2866,7 @@ class PackagesModule {
   async shutdown() {
     pckLog.info('Shutting down Packages module...');
 
-    this.stopCrawlerJob();
+    await this.stopCrawlerJob();
 
     // Close database connection
     if (this.db) {

package/packages/readme.md

@@ -31,7 +31,7 @@ In summary:
 | `GET /packages/catalog` | Search packages: `name`, `canonical`, `pkgcanonical`, `fhirversion`, `dependency`, `dependson`, `sort` |
 | `GET /packages/-/v1/search` | npm search API: `text`, `size`, `from`, plus the `/catalog` parameters |
 | `GET /packages/{id}` | npm package document: all versions, dependencies, tarball URLs |
-| `GET /packages/{id}/{version}` | Download a package tarball; or, when the request's `Accept` asks for JSON, npm's document for that version. A `FHIR-Package-Download` header always gets the tarball (the HL7 Java tools send it) |
+| `GET /packages/{id}/{version}` | Download a package tarball; or, when the request's `Accept` prefers JSON (ranks it above `*/*`), npm's document for that version. A `FHIR-Package-Download` header always gets the tarball (the HL7 Java tools send it) |
 | `GET /packages/updates` | Versions published since a date |
 | `GET /packages/broken` | Package versions whose dependencies this registry doesn't hold |
 | `GET /packages/npm-check` | Check any registry against the npm registry protocol (a form, in a browser) |

package/shl/shl.js

@@ -418,10 +418,12 @@ class SHLModule {
     }
   }
 
-  stopCleanupJob() {
+  // node-cron 4: stop() may return a promise
+  async stopCleanupJob() {
     if (this.cleanupJob) {
-      this.cleanupJob.stop();
+      const job = this.cleanupJob;
       this.cleanupJob = null;
+      await job.stop();
     }
   }
 
@@ -1060,7 +1062,7 @@ class SHLModule {
   async shutdown() {
     shlLog.info('Shutting down SHL module...');
 
-    this.stopCleanupJob();
+    await this.stopCleanupJob();
 
     // Stop FHIR validator
     if (this.fhirValidator) {

package/token/token.js

@@ -11,7 +11,6 @@ const crypto = require('crypto');
 const path = require('path');
 const fs = require('fs');
 const session = require('express-session');
-const SQLiteStore = require('connect-sqlite3')(session);
 const passport = require('passport');
 const GoogleStrategy = require('passport-google-oauth20').Strategy;
 const FacebookStrategy = require('passport-facebook').Strategy;
@@ -19,6 +18,7 @@ const GitHubStrategy = require('passport-github2').Strategy;
 const rateLimit = require('express-rate-limit');
 const lusca = require('lusca');
 const folders = require('../library/folder-setup');
+const { SqliteSessionStore } = require('../library/sqlite-session-store');
 const escape = require('escape-html');
 
 
@@ -155,17 +155,11 @@ class TokenModule {
   }
 
   initializeSession() {
-    const configDb = this.config.database || 'token.db';
-    const sessionDbPath = path.isAbsolute(configDb)
-      ? configDb
-      : folders.ensureFilePath('token', configDb);
+    // sessions live in the module's own database (initializeDatabase has opened it)
+    this.sessionStore = new SqliteSessionStore({ db: this.db, table: 'sessions' });
 
     const sessionConfig = {
-      store: new SQLiteStore({
-        dir: path.dirname(sessionDbPath),
-        db: path.basename(sessionDbPath),
-        table: 'sessions'
-      }),
+      store: this.sessionStore,
       secret: this.config.sessionSecret || crypto.randomBytes(64).toString('hex'),
       name: 'fhir.token.sid', // Don't use default session name
       resave: false,
@@ -1287,6 +1281,9 @@ class TokenModule {
   }
 
   async shutdown() {
+    if (this.sessionStore) {
+      this.sessionStore.close();
+    }
     if (this.db) {
       await new Promise((resolve) => {
         this.db.close((err) => {

package/tx/importers/import-rxnorm.module.js

@@ -780,7 +780,7 @@ class RxNormImporter {
     if (this.options.verbose) console.log('Generating word stems...');
 
     // Simple English stemmer implementation
-    var natural = require('natural');
+    var natural = require('natural/lib/natural/stemmers'); // just the stemmers: the package index also loads its storage backends, and dotenv with them
     const stems = new Map();
 
     // Get all RXNORM concepts

package/tx/importers/import-sct.module.js

@@ -4,7 +4,7 @@ const inquirer = require('inquirer');
 const path = require('path');
 const readline = require('readline');
 const chalk = require('chalk');
-const natural = require('natural');
+const natural = require('natural/lib/natural/stemmers'); // just the stemmers: the package index also loads its storage backends, and dotenv with them
 
 const {
   SnomedStrings, SnomedWords, SnomedStems, SnomedReferences,

package/tx/library/designations.js

@@ -1,6 +1,6 @@
 const { LanguagePartType, Languages, Language, LanguageDefinitions} = require('../../library/languages');
 const {validateParameter, validateOptionalParameter, validateArrayParameter} = require("../../library/utilities");
-const natural = require('natural');
+const natural = require('natural/lib/natural/stemmers'); // just the stemmers: the package index also loads its storage backends, and dotenv with them
 /**
  * Display checking modes for concept designations
  */

package/tx/library/resource-ids.js

@@ -58,6 +58,21 @@ function stripIdPrefix(id, spaceId) {
   return id.startsWith(prefix) && id.length > prefix.length ? id.substring(prefix.length) : null;
 }
 
+/**
+ * The id a resource had in its source, for an id in any id space: everything after the
+ * prefix, which never contains a '-' (see checkIdPrefix). Null if the id has no prefix.
+ *
+ * @param {string} id
+ * @returns {string|null}
+ */
+function originalId(id) {
+  if (typeof id !== 'string') {
+    return null;
+  }
+  const i = id.indexOf('-');
+  return i > 0 && i < id.length - 1 ? id.substring(i + 1) : null;
+}
+
 /**
  * Whether a prefix given in the library YAML is acceptable. It's the start of an id, so it
  * may only use id characters, and no '-', so that the prefix of any id is unambiguous.
@@ -104,4 +119,4 @@ function assignIdSpace(provider, spaceId) {
 
 const CORE_PREFIX = 'core';
 
-module.exports = { applyIdPrefix, stripIdPrefix, checkIdPrefix, assignIdSpace, CORE_PREFIX };
+module.exports = { applyIdPrefix, stripIdPrefix, originalId, checkIdPrefix, assignIdSpace, CORE_PREFIX };

package/tx/sct/text-index.js

@@ -27,7 +27,7 @@
  * event loop as it goes, and a search takes milliseconds once it exists.
  */
 
-const natural = require('natural');
+const natural = require('natural/lib/natural/stemmers'); // just the stemmers: the package index also loads its storage backends, and dotenv with them
 
 const stemmer = natural.PorterStemmer;
 

package/tx/tx-html.js

@@ -1329,6 +1329,30 @@ class TxHtmlRenderer {
     return html;
   }
 
+  /**
+   * The page for a read of an id that nothing has, but that several resources had in their
+   * source (see ReadWorker.notFound): a list of them, to pick from.
+   *
+   * @param {string} resourceType
+   * @param {string} id - the id that was asked for
+   * @param {Array<{id, url, version, title, source}>} matches
+   * @returns {string} html
+   */
+  renderIdChoices(resourceType, id, matches) {
+    let html = `<p>There is no ${escape(resourceType)} with the id <code>${escape(id)}</code> on this server. ` +
+      `Every resource here has an id that starts with a prefix for the source it came from, and ` +
+      `${matches.length} ${escape(resourceType)} resources had the id <code>${escape(id)}</code> in their source:</p>`;
+    html += '<table class="grid"><tr><th>Id</th><th>URL</th><th>Version</th><th>Title</th><th>Source</th></tr>';
+    for (const m of matches) {
+      // relative to /{type}/{id}, like the redirect when there is only one
+      html += `<tr><td><a href="${escape(encodeURIComponent(m.id))}">${escape(m.id)}</a></td>` +
+        `<td>${escape(m.url || '')}</td><td>${escape(m.version || '')}</td>` +
+        `<td>${escape(m.title || '')}</td><td>${escape(m.source || '')}</td></tr>`;
+    }
+    html += '</table>';
+    return html;
+  }
+
   /**
    * Render Bundle resource
    */

package/tx/tx.js

@@ -516,8 +516,12 @@ class TXModule {
             return res.status(400).json(buildOperationOutcome('error', 'invalid', `Invalid XML: ${e.message}`));
           }
         }
-      } else if (contentType != 'application/x-www-form-urlencoded') {
-        return res.status(415).json(buildOperationOutcome('error', 'invalid', `Unsupported Media Type: ${contentType}`));
+      } else if (contentType.split(';')[0].trim().toLowerCase() != 'application/x-www-form-urlencoded') {
+        // compare the media type alone: a form post may carry a charset parameter
+        const message = contentType
+          ? `Unsupported Media Type: ${contentType}`
+          : `No Content-Type was given for this ${req.method}; use application/fhir+json or application/fhir+xml`;
+        return res.status(415).json(buildOperationOutcome('error', 'invalid', message));
       }
 
       if (req.body) {
@@ -994,7 +998,7 @@ class TXModule {
             ));
           }
           let worker = new ReadWorker(req.txOpContext, this.log, req.txProvider, this.languages, this.i18n);
-          await worker.handle(req, res, resourceType);
+          await worker.handle(req, res, resourceType, (req, res, resourceType, id, matches) => this.renderIdChoices(req, res, resourceType, id, matches));
         } finally {
           this.countRequest(endpointPath, 'read', Date.now() - start);
         }
@@ -1432,6 +1436,17 @@ class TXModule {
     return count;
   }
 
+  // A read of an unprefixed id that several resources had in their source: the page
+  // that lists them (status 300, Multiple Choices). See ReadWorker.notFound.
+  async renderIdChoices(req, res, resourceType, id, matches) {
+    const txhtml = new TxHtmlRenderer(new Renderer(req.txOpContext, req.txEndpoint.provider), this.liquid, this.languages, this.i18n, req.txEndpoint.path);
+    const content = txhtml.renderIdChoices(resourceType, id, matches);
+    const html = await txhtml.renderPage(`${resourceType}/${id}`, content, req.txEndpoint, req.txStartTime);
+    res.status(300);
+    res.setHeader('Content-Type', 'text/html');
+    return res.send(html);
+  }
+
   // $closure usage for the home page: null when closure isn't turned on
   closureStats() {
     if (!this.closureStore) {

package/tx/workers/read.js

@@ -8,6 +8,9 @@ const { TerminologyWorker } = require('./worker');
 const {debugLog} = require("../operation-context");
 
 const { buildOperationOutcome } = require('../library/operation-outcome');
+const { originalId } = require('../library/resource-ids');
+const { acceptsHtml } = require('../tx-html');
+
 class ReadWorker extends TerminologyWorker {
   /**
    * @param {OperationContext} opContext - Operation context
@@ -32,10 +35,12 @@ class ReadWorker extends TerminologyWorker {
    * @param {express.Request} req - Express request (with txProvider attached)
    * @param {express.Response} res - Express response
    * @param {string} resourceType - The resource type (CodeSystem, ValueSet, ConceptMap)
-   * @param {Object} log - Logger instance
+   * @param {Function} [renderChoices] - (req, res, resourceType, id, matches) renders the
+   *   page that lists the resources an unprefixed id could mean (see notFound)
    */
-  async handle(req, res, resourceType) {
+  async handle(req, res, resourceType, renderChoices = null) {
     const { id } = req.params;
+    this.renderChoices = renderChoices;
 
     this.log.debug(`Read ${resourceType}/${id}`);
 
@@ -114,7 +119,7 @@ class ReadWorker extends TerminologyWorker {
       }
     }
 
-    return res.status(404).json(buildOperationOutcome('error', 'not-found', `CodeSystem/${id} not found`));
+    return await this.notFound(req, res, 'CodeSystem', id);
   }
 
   /**
@@ -131,7 +136,7 @@ class ReadWorker extends TerminologyWorker {
       }
     }
 
-    return res.status(404).json(buildOperationOutcome('error', 'not-found', `ValueSet/${id} not found`));
+    return await this.notFound(req, res, 'ValueSet', id);
   }
   /**
    * Handle ConceptMap read
@@ -147,7 +152,84 @@ class ReadWorker extends TerminologyWorker {
       }
     }
 
-    return res.status(404).json(buildOperationOutcome('error', 'not-found', `ConceptMap/${id} not found`));
+    return await this.notFound(req, res, 'ConceptMap', id);
+  }
+
+  /**
+   * Nothing has the id asked for. Published IGs link to resources by the id they had in
+   * their source (ValueSet/2.16.840.1.113762.1.4.1), from before every id was put in its
+   * provider's id space (ValueSet/vsac-2.16.840.1.113762.1.4.1). So for a browser, look
+   * for resources that had that id in their source: if there's one, redirect to it, and
+   * if there are several, list them. Anything other than a browser still gets a 404 -
+   * an unprefixed id can't say which resource it means (see library/resource-ids.js).
+   */
+  async notFound(req, res, resourceType, id) {
+    if (this.renderChoices && acceptsHtml(req)) {
+      const matches = await this.findByOriginalId(resourceType, id);
+      if (matches.length === 1) {
+        req.logInfo = `redirected to ${matches[0].id}`;
+        // relative, so that it works whatever path the server is behind; keeps the query
+        const q = req.originalUrl.indexOf('?');
+        return res.redirect(302, encodeURIComponent(matches[0].id) + (q >= 0 ? req.originalUrl.substring(q) : ''));
+      }
+      if (matches.length > 1) {
+        req.logInfo = `${matches.length} matches`;
+        return await this.renderChoices(req, res, resourceType, id, matches);
+      }
+    }
+    return res.status(404).json(buildOperationOutcome('error', 'not-found', `${resourceType}/${id} not found`));
+  }
+
+  /**
+   * The resources of a type that had this id in their source, across all the providers.
+   *
+   * @returns {Promise<Array<{id, url, version, title, source}>>} in provider order
+   */
+  async findByOriginalId(resourceType, id) {
+    const found = [];
+    const add = (resource) => {
+      const json = resource.jsonObj || resource;
+      if (json.id && !found.some(m => m.id === json.id)) {
+        found.push({
+          id: json.id,
+          url: json.url || null,
+          version: json.version || null,
+          title: json.title || json.name || null,
+          source: typeof resource.sourcePackage === 'string' ? resource.sourcePackage : null
+        });
+      }
+    };
+
+    switch (resourceType) {
+      case 'CodeSystem':
+        // the map holds a code system under more than one key (url, url|version)
+        for (const cs of new Set(this.provider.codeSystems.values())) {
+          this.deadCheck('findByOriginalId-cs');
+          if (originalId(cs.jsonObj.id) === id) {
+            add(cs);
+          }
+        }
+        break;
+      case 'ValueSet':
+        for (const vsp of this.provider.valueSetProviders) {
+          this.deadCheck('findByOriginalId-vs');
+          const vs = vsp.spaceId ? await vsp.fetchValueSetById(`${vsp.spaceId}-${id}`) : null;
+          if (vs) {
+            add(vs);
+          }
+        }
+        break;
+      case 'ConceptMap':
+        for (const cmp of this.provider.conceptMapProviders) {
+          this.deadCheck('findByOriginalId-cm');
+          const cm = cmp.spaceId ? await cmp.fetchConceptMapById(`${cmp.spaceId}-${id}`) : null;
+          if (cm) {
+            add(cm);
+          }
+        }
+        break;
+    }
+    return found;
   }
 }