fhirsmith 0.14.1 → 0.15.0

package/packages/readme.md

@@ -0,0 +1,120 @@
+# Package Server
+
+A registry of FHIR NPM packages. It crawls the package feeds listed in the FHIR IG
+registry, stores every package version it finds, and serves them using the npm
+registry protocol, with FHIR-specific search on top (by canonical URL, FHIR version
+and dependency). It is the server behind packages2.fhir.org.
+
+Its main clients are the HL7 Java tools (`org.hl7.fhir.utilities.npm.PackageClient`,
+used by the IG Publisher and validator). Package documents, tarball URLs and search
+results follow npm's formats, so npm clients can point `--registry` at
+`<server>/packages`.
+
+## API
+
+The public API is described by an OpenAPI 3.1 spec:
+
+| URL | |
+|---|---|
+| `/packages/openapi` | Browsable reference (HTML), with a "try it" form for each GET operation |
+| `/packages/openapi.json` | The spec as JSON |
+| `/packages/openapi.yaml` | The spec as YAML (the source file, [openapi.yaml](openapi.yaml)) |
+
+Every response from the package server carries a
+`Link: </packages/openapi.json>; rel="service-desc"` header (RFC 8631), and the
+HTML pages carry the matching `<link>` element.
+
+In summary:
+
+| Endpoint | Purpose |
+|---|---|
+| `GET /packages/catalog` | Search packages: `name`, `canonical`, `pkgcanonical`, `fhirversion`, `dependency`, `dependson`, `sort` |
+| `GET /packages/-/v1/search` | npm search API: `text`, `size`, `from`, plus the `/catalog` parameters |
+| `GET /packages/{id}` | npm package document: all versions, dependencies, tarball URLs |
+| `GET /packages/{id}/{version}` | Download a package tarball |
+| `GET /packages/updates` | Versions published since a date |
+| `GET /packages/broken` | Package versions whose dependencies this registry doesn't hold |
+
+Most endpoints return an HTML page instead of JSON when the request's `Accept`
+header contains `text/html`. The JSON is the contract; the HTML is for people.
+
+Query parameters are validated strictly. An unknown parameter, a repeated parameter,
+or a value that's too long or badly formed is a 400, not silently ignored.
+
+Not in the spec, deliberately:
+
+* Operational pages: `/packages/stats`, `/packages/log` (the last crawler run),
+  `/packages/status`.
+* `POST /packages/crawl`, which runs a full crawl on demand. It requires the
+  `x-crawl-token` header to match the `crawlToken` setting, and is disabled when no
+  token is configured.
+
+### Keeping the spec honest
+
+[openapi.yaml](openapi.yaml) is maintained by hand. `tests/packages/openapi.test.js`
+fails if:
+
+* a route is added to the router without being described in the spec, or listed in
+  the test's `EXCLUDED` table with a reason;
+* the spec describes a route the router doesn't have;
+* a query or path parameter's name, length limit, pattern or default differs from the
+  validation rules in `packages.js` (`SEARCH_PARAMS`, `V1_SEARCH_PARAMS`,
+  `UPDATES_PARAMS`, `BROKEN_PARAMS`, `PATH_PARAMS`).
+
+So when you change a route or a parameter rule, change `openapi.yaml` with it.
+`tests/packages/search.test.js` covers the search behaviour itself against an
+in-memory database.
+
+## Configuration
+
+In the server config, under `modules.packages`:
+
+```json
+"packages": {
+  "enabled": true,
+  "database": "packages.db",
+  "mirrorPath": "/absolute/path/to/mirror",
+  "crawlToken": "a-long-random-secret",
+  "crawler": {
+    "enabled": true,
+    "schedule": "0 * * * *"
+  }
+}
+```
+
+| Setting | |
+|---|---|
+| `database` | The SQLite database. A relative path is resolved against the data directory's `packages` folder. Created on first run. |
+| `mirrorPath` | Directory where the crawler saves a copy of each tarball, as `{id}-{version}.tgz`. A scoped id `@scope/name` is saved as `$scope$name`. |
+| `bucketPath` | Optional. If set, downloads redirect to `{bucketPath}/{file}` and tarball URLs point there, instead of being served from the database. The file names are the mirror's, so the bucket is expected to be a copy of the mirror. |
+| `baseUrl` | Optional. The public base URL used when building package URLs. Defaults to the request's host. |
+| `crawlToken` | Shared secret for `POST /packages/crawl`. Omit it to disable that endpoint; the scheduled crawler is unaffected. |
+| `crawler.enabled` | Run the crawler at startup and on the schedule. |
+| `crawler.schedule` | Cron expression for crawls. |
+| `masterUrl` | The list of feeds to crawl. Defaults to `https://fhir.github.io/ig-registry/package-feeds.json`. |
+| `localFeedDirs` | Optional. Directories feeds may be read from as local files (for testing). |
+| `allowPrivateAddresses` | Optional, for testing only. Lets the crawler fetch from private and loopback addresses, which it otherwise refuses (SSRF protection). |
+
+## Crawling
+
+Each crawl reads the master feed list, then each RSS feed in it (Simplifier last).
+For each item, it downloads the package, checks the id, version and canonical, and
+stores it.
+
+* The master list's `package-restrictions` say which feeds may publish which package
+  ids. A package from a feed that isn't allowed to publish it is skipped.
+* Items marked `notForPublication` are skipped.
+* Feeds can be paginated (RFC 5005 `rel="next"`). The first page is read on every
+  crawl; older pages are read once and remembered in the `FeedPages` table.
+* A feed that rate-limits (HTTP 429) is abandoned for that crawl and retried next
+  time.
+
+`/packages/log` shows what the last crawl did, feed by feed and item by item.
+
+## Storage
+
+Everything is in the SQLite database. `PackageVersions` holds each version's
+metadata and the tarball itself. `Packages` holds one row per package id, with its
+current version and download count. `PackageDependencies` (stored as `id@version`),
+`PackageFHIRVersions` and `PackageURLs` (the canonical URLs of the resources in each
+version) support search.

package/registry/api.js

@@ -2,6 +2,30 @@
 
 const { ServerRegistryUtilities } = require('./model');
 const escape = require('escape-html');
+const { Utilities } = require('../library/utilities');
+
+const RELEASE_VERSIONS = {
+  R2: '1.0',
+  R2B: '1.4',
+  R3: '3.0',
+  R4: '4.0',
+  R4B: '4.3',
+  R5: '5.0',
+  R6: '6.0'
+};
+
+// The ecosystem IG reports a server's security as boolean flags (open, password, token,
+// oauth, smart, cert). The crawler records a single string; this maps it. 'api-key' is a
+// token. The string itself is still reported as 'security' for existing clients.
+function securityFlags(security) {
+  switch (security) {
+    case 'open': return { open: true };
+    case 'api-key': return { token: true };
+    case 'password': case 'token': case 'oauth': case 'smart': case 'cert':
+      return { [security]: true };
+    default: return {};
+  }
+}
 
 class RegistryAPI {
   constructor(crawler) {
@@ -263,6 +287,10 @@ class RegistryAPI {
 
     return {
       lastRun: data.lastRun,
+      // the page footer: nothing to report until the first crawl has finished
+      crawlerStatus: data.lastRun ?
+        `last updated ${Utilities.describeAgo(data.lastRun)}, ${totalServers} server${totalServers === 1 ? '' : 's'}` :
+        'not yet updated',
       outcome: data.outcome,
       registryCount: data.registries.length,
       serverCount: totalServers,
@@ -317,16 +345,11 @@ class RegistryAPI {
     });
   }
 
+  // Release codes (R4, r4b ...) to the version prefix they select; anything else (4.0.1,
+  // 4.0) is used as given. R4B and R2B can't be derived from the number alone.
   _normalizeFhirVersion(version) {
     if (!version) return version;
-
-    // Convert R4 or r4 to 4.0, R5 or r5 to 5.0, etc.
-    const rMatch = /^[rR](\d+)$/.exec(version);
-    if (rMatch) {
-      return `${rMatch[1]}.0`;
-    }
-
-    return version;
+    return RELEASE_VERSIONS[version.toUpperCase()] || version;
   }
 
   /**
@@ -471,38 +494,9 @@ class RegistryAPI {
       });
     });
 
-    // NEW: Fallback - if no matches found, check for authoritative pattern matches
-    if (authMatches.length === 0 && result.candidates.length === 0) {
-      data.registries.forEach(registry => {
-        registry.servers.forEach(server => {
-          // Excluded content stays hidden in the fallback path too
-          if (server.isExcludedTarget(codeSystem)) return;
-
-          // Check if server supports the requested usage tag
-          if (server.usageList.length === 0 ||
-            (usage && server.usageList.includes(usage))) {
-
-            // Check if server is authoritative for this code system
-            // (taking language specific claims into account)
-            const authMatch = server.matchAuthCS(codeSystem, requestLangs);
-
-            if (authMatch.isAuth) {
-              server.versions.forEach(version => {
-                if (ServerRegistryUtilities.versionMatches(normalizedVersion, version.version)) {
-                  authMatches.push({
-                    entry: this.createServerEntry(server, version, null, authMatch),
-                    score: authMatch.score
-                  });
-                  if (!matchedServers.includes(server.code)) {
-                    matchedServers.push(server.code);
-                  }
-                }
-              });
-            }
-          }
-        });
-      });
-    }
+    // No fallback to servers that claim authority but don't host the code system: per the
+    // ecosystem IG, 'Servers are not listed as authoritative unless they actually host the
+    // CodeSystem(+version) in the request'.
 
     // Language specific matches rank before authoritative-list matches, most specific
     // match first. Array.prototype.sort is stable, so entries with equal scores (e.g.
@@ -598,6 +592,86 @@ class RegistryAPI {
     };
   }
 
+  /**
+   * The ecosystem Discovery API (GET {root} as JSON): one row per server endpoint.
+   *
+   * Filters, all optional: registry and server (codes), fhirVersion (RX or M.n.p), url (a
+   * code system, url or url|version), authoritativeOnly, language (only with url).
+   *
+   * With url, only endpoints that host that code system are listed (an excluded code
+   * system hides the server), and the row's 'candidate' list says whether it's hosted but
+   * not claimed; authoritativeOnly then keeps only the endpoints authoritative for it.
+   * Without url, authoritative/authoritative-valuesets are the server's claim masks and the
+   * candidate lists are omitted - listing everything a server hosts would be huge.
+   */
+  discover(params = {}) {
+    const { registry = '', server = '', fhirVersion = '', url = '', language = '' } = params;
+    const authoritativeOnly = params.authoritativeOnly === true || params.authoritativeOnly === 'true';
+    const normalizedVersion = this._normalizeFhirVersion(fhirVersion);
+    const requestLangs = url ? ServerRegistryUtilities.parseAcceptLanguage(language) : null;
+    const data = this.crawler.getData();
+
+    const authRows = [];
+    const otherRows = [];
+    data.registries.forEach(reg => {
+      if (registry && reg.code !== registry) return;
+      reg.servers.forEach(srv => {
+        if (server && srv.code !== server) return;
+        if (url && srv.isExcludedTarget(url)) return;
+
+        const authMatch = url ? srv.matchAuthCS(url, requestLangs) : { isAuth: false, scoped: false };
+
+        srv.versions.forEach(version => {
+          if (normalizedVersion && !ServerRegistryUtilities.versionMatches(normalizedVersion, version.version)) {
+            return;
+          }
+          let hosted = false;
+          if (url) {
+            hosted = ServerRegistryUtilities.hasMatchingCodeSystem(url, version.codeSystems, false, {});
+            if (!hosted || (authoritativeOnly && !authMatch.isAuth)) {
+              return;
+            }
+          }
+
+          const row = {
+            'server-name': srv.name,
+            'server-code': srv.code,
+            'registry-name': reg.name,
+            'registry-code': reg.code,
+            'registry-url': reg.address,
+            url: version.address,
+            fhirVersion: version.version,
+            error: version.error || null,
+            'last-success': version.lastSuccess ? Math.floor(Date.now() - new Date(version.lastSuccess).getTime()) : null,
+            systems: version.codeSystems.length,
+            authoritative: [...srv.authCSList],
+            'authoritative-valuesets': [...srv.authVSList]
+          };
+          if (url && !authMatch.isAuth) {
+            row.candidate = [url];
+          }
+          if (authMatch.isAuth && authMatch.scoped) {
+            row.languages = [authMatch.tag];
+          }
+          if (version.security) {
+            row.security = version.security;
+            Object.assign(row, securityFlags(version.security));
+          }
+          (authMatch.isAuth ? authRows : otherRows).push({ row, score: authMatch.score || 0 });
+        });
+      });
+    });
+
+    // Endpoints authoritative for url first (language specific matches before the rest;
+    // sort is stable, so registration order is kept otherwise)
+    authRows.sort((a, b) => a.score - b.score);
+    return {
+      'last-update': data.lastRun ? new Date(data.lastRun).toISOString() : null,
+      'master-url': data.address,
+      results: [...authRows, ...otherRows].map(r => r.row)
+    };
+  }
+
   _cleanEmptyArrays(result) {
     const cleanedResult = { ...result };
 
@@ -620,11 +694,13 @@ class RegistryAPI {
   createServerEntry(server, version, content = null, authMatch = null) {
     const entry = {
       'server-name': server.name,
-      url: version.address
+      url: version.address,
+      fhirVersion: version.version
     };
 
     if (version.security) {
       entry.security = version.security;
+      Object.assign(entry, securityFlags(version.security));
     }
     if (server.accessInfo) {
       entry.access_info = server.accessInfo;

package/registry/crawler.js

@@ -333,11 +333,48 @@ class RegistryCrawler {
       this.addLogEntry('error', `Server ${version.address}: Error after ${elapsed}ms: ${error.message}`);
       version.error = error.message;
       version.lastTat = `${elapsed}ms`;
+      this.carryForwardSoftware(version);
     }
     
     return version;
   }
 
+  /**
+   * CapabilityStatement.software.releaseDate, if it means anything. A date within a few
+   * minutes of now is the server's clock, not a release date: FHIRsmith up to 0.14.2 filled
+   * it in with the time of the request, and other software may do the same.
+   */
+  reportedReleaseDate(software, now = Date.now()) {
+    const value = software && software.releaseDate;
+    const t = value ? new Date(value).getTime() : NaN;
+    if (isNaN(t) || Math.abs(now - t) < 10 * 60 * 1000) {
+      return '';
+    }
+    return String(value);
+  }
+
+  /**
+   * A server that can't be reached this time is still running whatever it was running
+   * last time we saw it - keep that, so the software page doesn't lose track of it
+   */
+  carryForwardSoftware(version) {
+    if (version.software && version.software !== 'unknown') {
+      return;
+    }
+    for (const registry of (this.currentData && this.currentData.registries) || []) {
+      for (const server of registry.servers || []) {
+        for (const prev of server.versions || []) {
+          if (prev.address === version.address && prev.software) {
+            version.software = prev.software;
+            version.softwareVersion = prev.softwareVersion || '';
+            version.softwareReleaseDate = prev.softwareReleaseDate || '';
+            return;
+          }
+        }
+      }
+    }
+  }
+
   /**
    * Process an R3 server
    */
@@ -348,6 +385,8 @@ class RegistryCrawler {
     
     version.version = capability.fhirVersion || '3.0.2';
     version.software = capability.software ? capability.software.name : "unknown";
+    version.softwareVersion = capability.software && capability.software.version ? String(capability.software.version) : '';
+    version.softwareReleaseDate = this.reportedReleaseDate(capability.software);
 
     // Get terminology capabilities (R3 uses Parameters resource)
     try {
@@ -392,6 +431,8 @@ class RegistryCrawler {
     
     version.version = capability.fhirVersion || defVersion;
     version.software = capability.software ? capability.software.name : "unknown";
+    version.softwareVersion = capability.software && capability.software.version ? String(capability.software.version) : '';
+    version.softwareReleaseDate = this.reportedReleaseDate(capability.software);
 
     let set = new Set();
 

package/registry/fhirsmith-releases.js

@@ -0,0 +1,167 @@
+// registry/fhirsmith-releases.js
+// The list of FHIRsmith releases and their dates, so the registry can say how old the
+// FHIRsmith version a registered server reports is.
+//
+// The list comes from the GitHub releases API, refreshed with each crawl. Until that
+// succeeds (or if it can't be reached), the dated headings in this server's own
+// CHANGELOG.md are used instead - those only go up to this server's own version.
+
+const { readReleaseDates } = require('../library/changelog');
+const path = require('path');
+const axios = require('axios');
+
+const DEFAULT_RELEASES_URL = 'https://api.github.com/repos/HealthIntersections/FHIRsmith/releases';
+const DAY_MS = 24 * 60 * 60 * 1000;
+
+// "v0.14.1" -> [0, 14, 1]; anything that isn't n.n.n (optionally with a -suffix) -> null
+function parseVersion(v) {
+  const m = /^v?(\d+)\.(\d+)\.(\d+)(?:-([0-9A-Za-z.-]+))?$/.exec(String(v || '').trim());
+  if (!m) {
+    return null;
+  }
+  return { parts: [Number(m[1]), Number(m[2]), Number(m[3])], suffix: m[4] || '' };
+}
+
+function compareParts(a, b) {
+  for (let i = 0; i < 3; i++) {
+    if (a[i] !== b[i]) {
+      return a[i] - b[i];
+    }
+  }
+  return 0;
+}
+
+function isFhirsmith(softwareName) {
+  return /fhirsmith/i.test(softwareName || '');
+}
+
+class FhirsmithReleases {
+  constructor(config = {}, logger = null) {
+    this.url = config.releasesUrl || DEFAULT_RELEASES_URL;
+    this.timeout = config.timeout || 30000;
+    this.userAgent = config.userAgent || 'FHIRRegistryServer/1.0';
+    this.logger = logger;
+    this.releases = []; // [{version: '0.14.1', parts: [0,14,1], date: Date}], newest first
+    this.source = 'none';
+    this.lastRefresh = null;
+  }
+
+  setReleases(list, source) {
+    const seen = new Set();
+    const releases = [];
+    for (const r of list) {
+      const pv = parseVersion(r.version);
+      const date = r.date instanceof Date ? r.date : new Date(r.date);
+      // a release is n.n.n - pre-release suffixes are not tracked
+      if (!pv || pv.suffix || isNaN(date.getTime())) {
+        continue;
+      }
+      const key = pv.parts.join('.');
+      if (!seen.has(key)) {
+        seen.add(key);
+        releases.push({ version: key, parts: pv.parts, date });
+      }
+    }
+    releases.sort((a, b) => compareParts(b.parts, a.parts));
+    this.releases = releases;
+    this.source = source;
+  }
+
+  /**
+   * Read the dated release headings from a CHANGELOG.md - "## [v0.13.4] - 2026-09-17"
+   */
+  loadFromChangelog(changelogPath = path.join(__dirname, '..', 'CHANGELOG.md')) {
+    try {
+      const list = [];
+      for (const [version, date] of readReleaseDates(changelogPath)) {
+        list.push({ version, date: new Date(date + 'T00:00:00Z') });
+      }
+      this.setReleases(list, 'CHANGELOG.md');
+    } catch (error) {
+      if (this.logger) {
+        this.logger.warn('Could not read FHIRsmith release dates from CHANGELOG.md: ' + error.message);
+      }
+    }
+  }
+
+  /**
+   * Fetch the release list from GitHub. On failure the current list is kept.
+   */
+  async refresh() {
+    try {
+      const list = [];
+      for (let page = 1; page <= 10; page++) {
+        const response = await axios.get(`${this.url}?per_page=100&page=${page}`, {
+          timeout: this.timeout,
+          headers: { 'Accept': 'application/vnd.github+json', 'User-Agent': this.userAgent }
+        });
+        const batch = Array.isArray(response.data) ? response.data : [];
+        for (const r of batch) {
+          if (!r.draft && !r.prerelease && r.tag_name && r.published_at) {
+            list.push({ version: r.tag_name, date: r.published_at });
+          }
+        }
+        if (batch.length < 100) {
+          break;
+        }
+      }
+      if (list.length > 0) {
+        this.setReleases(list, 'GitHub');
+        this.lastRefresh = new Date();
+      }
+    } catch (error) {
+      if (this.logger) {
+        this.logger.warn('Could not fetch FHIRsmith releases from GitHub: ' + error.message);
+      }
+    }
+  }
+
+  latest() {
+    return this.releases.length > 0 ? this.releases[0] : null;
+  }
+
+  /**
+   * How a reported FHIRsmith version stands against the releases:
+   *   status: 'current' | 'outdated' | 'dev' (a build between/after releases) | 'unknown'
+   *   release: the release it is (or, for 'dev', the most recent release before it)
+   *   ageDays: days since that release came out ('current'/'outdated' only)
+   *   behind: how many releases have come out since
+   */
+  describe(reportedVersion, now = Date.now()) {
+    const pv = parseVersion(reportedVersion);
+    if (!pv || this.releases.length === 0) {
+      return { status: 'unknown' };
+    }
+    const behind = this.releases.filter(r => compareParts(r.parts, pv.parts) > 0).length;
+    const exact = !pv.suffix ? this.releases.find(r => compareParts(r.parts, pv.parts) === 0) : null;
+    if (exact) {
+      return {
+        status: behind === 0 ? 'current' : 'outdated',
+        release: exact,
+        ageDays: Math.max(0, Math.floor((now - exact.date.getTime()) / DAY_MS)),
+        behind
+      };
+    }
+    // a snapshot (0.14.2-snapshot), or a version we have no release for
+    const base = this.releases.find(r => compareParts(r.parts, pv.parts) < 0) || null;
+    return { status: 'dev', release: base, behind };
+  }
+}
+
+function describeAge(days) {
+  if (days < 1) {
+    return 'today';
+  }
+  if (days < 14) {
+    return `${days} day${days === 1 ? '' : 's'}`;
+  }
+  if (days < 60) {
+    return `${Math.floor(days / 7)} weeks`;
+  }
+  if (days < 730) {
+    return `${Math.floor(days / 30.44)} months`;
+  }
+  return `${(days / 365.25).toFixed(1)} years`;
+}
+
+module.exports = { FhirsmithReleases, isFhirsmith, parseVersion, describeAge };

package/registry/model.js

@@ -11,6 +11,8 @@ class ServerVersionInformation {
     this.lastSuccess = null; // Date object
     this.lastTat = '';
     this.software = ''; // what software is running
+    this.softwareVersion = ''; // CapabilityStatement.software.version, if the server reports one
+    this.softwareReleaseDate = ''; // CapabilityStatement.software.releaseDate, if it is a real date
     this.codeSystems = []; // Array of strings (sorted, unique)
     this.valueSets = []; // Array of strings (sorted, unique)
   }
@@ -58,6 +60,8 @@ class ServerVersionInformation {
       lastTat: this.lastTat,
       terminologies: this.codeSystems,
       software: this.software,
+      'software-version': this.softwareVersion,
+      'software-release-date': this.softwareReleaseDate,
       valuesets: this.valueSets
     };
   }
@@ -71,6 +75,8 @@ class ServerVersionInformation {
     instance.lastSuccess = json['last-success'] ? new Date(json['last-success']) : null;
     instance.lastTat = json.lastTat || '';
     instance.software = json.software;
+    instance.softwareVersion = json['software-version'] || '';
+    instance.softwareReleaseDate = json['software-release-date'] || '';
     instance.codeSystems = json.terminologies || [];
     instance.valueSets = json.valuesets || [];
     return instance;
@@ -561,6 +567,13 @@ class ServerRegistryUtilities {
       } else {
         // Otherwise do exact matching on both full and base URL
         ok = vurl === cs || vurl === baseCs;
+        // A SNOMED CT version URI extends its edition URI, so a server hosting
+        // sct|.../{edition}/version/{date} hosts the edition sct|.../{edition}. Servers
+        // list only full versions, so without this an edition-level request would never
+        // find the servers that host the edition.
+        if (!ok && noVersionIndependentMatching && cs.includes('|')) {
+          ok = vurl.startsWith(cs + '/');
+        }
       }
       if (ok && content) {
         content.content = item.content;

package/registry/openapi.js

@@ -0,0 +1,13 @@
+//
+// Copyright 2026, Health Intersections Pty Ltd (http://www.healthintersections.com.au)
+//
+// Licensed under BSD-3: https://opensource.org/license/bsd-3-clause
+//
+
+// The terminology registry's OpenAPI description. openapi.yaml is the source of truth; see
+// the comment at its top.
+
+const path = require('path');
+const { createOpenApiDoc } = require('../library/openapi-doc');
+
+module.exports = createOpenApiDoc(path.join(__dirname, 'openapi.yaml'), '/tx-reg');