fhirsmith 0.17.0 → 0.18.0

package/CHANGELOG.md

@@ -5,6 +5,89 @@ All notable changes to Health Intersections FHIRsmith will be documented in this
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [0.18.0] - 2026-10-23
+
+A big release: a new IPS module (prototype), a secured and documented registry with uptime reporting, a syndication feed for the terminology server, a daily email report, ICF qualified codes, SNOMED CT edition to edition translation and much more ECL, and the ICD-11 representation agreed at the October 2026 test-case review meetings. Plus the outcome of security reviews of the registry, IPS and token modules. The README now says what state each module is in.
+
+Note: We are heading towards a 1.0.0 release.
+
+- **Upgrading** - check these before deploying:
+  - ICD-11 (MMS and ICF): the code is now the bare entity id, not the short code (see Changed). Clients that send or store short codes still work, but get `CODE_NOT_IN_NORMAL_FORM`; full entity URIs as codes are no longer accepted
+  - Token: the JSON API key moves from the URL path to a header (see Security)
+  - LOINC: `exists` filters now mean what they say, so value sets that relied on the old behaviour expand differently (see Fixed)
+  - VSAC: the default base URL is now https; a configured http `baseUrl` is logged as a warning
+  - The SNOMED CT test edition cache is now `sct_test_20250909c` (extension concepts added in module TEST1000003) and needs rebuilding for the tx tests
+
+### Security
+
+- Page templates (`library/html-server.js`, every module): placeholders are now substituted in one pass, so a `[%...%]` in a page's content (a submitted IPS narrative, say) stays as it is. Before, `[%context-link%]`, `[%about%]` and the like inside content were replaced with their raw HTML, which inside an attribute let a stranger's narrative become a live element (stored XSS)
+- IPS: the generated narrative escapes dosage timing values (`frequency`, `period` and their maxima) that a document puts where numbers belong, and generated narrative is checked for active content before it is stored or shown
+- IPS: a project's stored AI key is dropped when the provider or base URL changes, so it can't be sent to a host other than the one it was given for. The terminology check no longer sends a coding's display to `$validate-code`: answers are cached by code, so one document with a wrong display could decide the answer for every document after it. Delete confirmations are an escaped attribute, not inline script with a name in it. `setPath`/`getPath` refuse paths that would reach a prototype. Document submissions are read by the module itself (the server's body parsers leave `/ips` alone): the submission token is checked before the body is read, and the body is read no bigger than `maxSize`
+- Registry: markdown documents are rendered without raw HTML (commonmark's safe mode), and a past version of a document keeps the visibility it had then, so a version from when it was private stays private after the document is made public. `/resolve` no longer answers a page request with JSON when the page fails to render. JWTs without `exp` are refused. Changing or removing a user's password ends the user's other sessions. The failed-authentication limit checks the token first, so a valid token is never held back
+- VSAC: the default base URL is now `https://cts.nlm.nih.gov/fhir`. It was http, and VSAC only redirects to https after the request (which carries the API key) has been sent. A configured `baseUrl` that is http is logged as a warning
+- VSAC: a search bundle's `next` link is only followed on the VSAC server (the same host, under `baseUrl`), and always over `baseUrl`'s scheme, so a link can't send the API key elsewhere or over http
+- Token: the JSON API takes the API key in a header - `Authorization: Bearer {key}` or `X-API-Key: {key}` - instead of the URL path (`/token/api/validate`, `/token/api/usage`, `/token/api/stats`), so keys don't end up in access logs. The usage statistics no longer include the IP address of the last request, and the key list's delete button no longer puts the key's name into an inline script
+- Narrative check (`library/narrative-check.js`): the regex that stripped CSS comments from a `style` attribute was quadratic in the attribute's length - a few hundred KB of `/* ` held the event loop for seconds, and a resource with such a narrative could be sent to any module that checks narratives. Comments are now stripped in one pass
+- IPS: narratives are shown as parsed, never as the string submitted, so an unterminated tag at the end of one narrative can't swallow the start of the next on the page and turn its text into attributes of a live element (the active-content check parses each narrative on its own, so it didn't see this). Reference resolution within a document is indexed rather than scanned (a document with tens of thousands of entries took a minute of CPU per page view); index rows per search parameter, sort keys per search, the JSON a search page carries, the terminology-server questions per document and the remembered answers are all bounded; an in-process terminology request has a timeout; error responses and the public document pages no longer carry internal detail (the reason a document's processing failed is shown to its project's managers); the submission form no longer gives every visitor a session; an unknown login takes as long as a known one; verdicts in a submitted document's own extensions are never shown as analysis
+
+### Added
+
+- Terminology server: a syndication feed (`modules.tx.syndication.enabled`, off by default) - the content the library loads, published at `{endpoint}/syndication` as a [Terminology Syndication Feed](https://build.fhir.org/ig/FHIR/fhir-syndication-ig/en/) (Atom, NCTS ASF profile) for Ontoserver, other FHIRsmith servers and people. As the library loads, each source is recorded in a store in `terminology-cache/syndication`, from the time syndication is turned on: npm, url and core packages (linked at npm.fhir.org, `FHIR_Package`), the versioned resources in file and folder sources (copied into the store), and the binaries (SNOMED caches, LOINC databases etc, linked at `base.url`, in a FHIRsmith category scheme, with `<rights>`) that are published there - not internal sources, OCL, `#current` packages or the full CPT database. Every link has a SHA-256. The store only grows: no retracts, and a later update to the same content is a new entry (so `contentItemVersion` is not unique, deliberately). The feed is paged as RFC 5005 archived feeds of 30 entries; a browser gets it as a page, and the tx pages have a Syndication nav item and feed autodiscovery link
+- Registry: the software page shows one row per server rather than one per endpoint - the endpoints of a server (its FHIR versions) that run the same software and version are collapsed under their common base URL, with the FHIR versions listed and any unreachable endpoint named. Endpoints running different software keep their own rows
+- Registry: the API can be secured (`modules.registry.security`), in any combination of three ways: a single `apiToken` every client sends; `users` the administrator manages on `/tx-reg/admin/users`, each with their own tokens (256 random bits, stored as a SHA-256 hash, shown once, labelled, optionally expiring, revocable; usage counted per token) and optionally a password to manage their own tokens on `/tx-reg/account`; and `oidc`, JWTs from an OpenID Connect provider verified against its published keys for issuer, audience, expiry and a required scope or role. Only the JSON API is closed - the HTML pages, the OpenAPI description and the login page stay open - and a request without a valid token gets 401 with `WWW-Authenticate: Bearer`. Failed authentications are limited per address (`security.failedAuthPerWindow`, default 60 in 15 minutes, then 429; successful requests aren't counted). Tokens are sent as `Authorization: Bearer {token}` or `Api-Key: {token}`, the headers the HL7 Java tooling already sends for a server in fhir-settings.json. Without `security` the registry is open, as before. New dependency: `jose`
+- Registry: documents (`/tx-reg/docs`, when `security.users` is configured) - markdown or JSON documents the registry stores and serves, each with a name (its address), a type, whether it is public, and when and by whom it was last edited. Public documents are readable by anyone, the others by any logged in user (a browser session, never an API token); administrators create, edit, rename and delete them in a plain text editor. A browser gets the document rendered (markdown as written, raw HTML included - the authors are the administrators; JSON pretty-printed), anything else the raw file with its media type. Every change is logged with the full content, and the history page shows each past version
+- Registry: users can be administrators (a flag the administrator sets): an administrator user manages users and documents as the configured administrator does, and keeps its own account page
+- Registry: uptime. Each scan records whether every endpoint answered (`[data]/registry/registry-uptime.json`, kept for `uptimeRetentionDays`, default 90), and the software page reports each server's uptime over the last `uptimeWindowDays` (default 30) as the share of its endpoint probes that answered. Scans the registry itself missed count against no one
+- IPS module (`/ips`): collections ("projects") of International Patient Summary documents, each with its own FHIR endpoint (`POST`/`GET {project}/Bundle`, `Bundle/{id}`, `metadata`) and web pages - a filterable list, the document's assembled narrative, and a submission form. The administrator (login in the configuration) creates projects and users and assigns managers; managers set a project's submission token and delete documents. Searches support Bundle's parameters and chain to the Composition's and Patient's (`composition.date`, `composition.patient.name`, ...), indexed with fhirpath.js. A document's page has narrative, terminology (every CodeableConcept, with code systems linked to the configured terminology server or, for SNOMED CT, LOINC etc., to their browsers), validation and JSON views. Received documents are queued for a background processing pipeline, of which six steps exist: translation of every CodeableConcept that fails the terminology check, by `ConceptMap/$translate` into the element's value set, the matches added as codings (in the analysed copy) with an extension saying which server, concept map and relationship they came from; generation of the document's narrative with an IPS presenter (the Composition's header and patient, then the sections in the IPS order, each a table of its entries with the columns that suit the resource type and the nested parts as bullets beneath, coded values looked up on the terminology server when they have no display), written into the analysed copy; validation with the FHIR validator (as the shl module runs it) against the IPS Bundle profile, the outcome kept with the document; a terminology check of every CodeableConcept for international exchange: against the module's own value sets for its element (`ips/valuesets/`, on SNOMED CT International, LOINC, EDQM routes and HL7 codes), falling back to any SNOMED CT International, LOINC or HL7 code, by `$validate-code` on the configured terminology server, the verdicts written as extensions into a second, analysed copy of the document and shown in the terminology view; a check against each clinical profile (whether the document conforms, and the constraints it doesn't meet, shown in a Clinical Profiles view); and, when a project turns it on, an AI assessment of the document's consistency, completeness and terminology by an OpenAI-compatible or Anthropic service (the project's own account, its key encrypted at rest, or the server's default; capped per project per day). A document can be shared as a QR code - a SMART Health Link, or a Verifiable Health Link signed as the GDHCN does it - through the shl module (`modules.ips.shl`). An editor (`/ips/{code}/compose`) composes a document in the browser - the patient and header, then each IPS section as entries with code lookups, a narrative, or a reason it is empty - and submits it to the project or hands it back; an existing document can be opened in it. Two FHIRPath functions for IPS documents, `section(system, code [, min, max])` and `entry(system, code [, min, max])`, find sections (nested too) and their resolved entries by code, within cardinality bounds, and `resources(system, code [, type])` finds the document's resources with a code, with `entryIn()` and `resourcesIn()` doing the same by value set, asking the terminology server as they go, with each answer cached (`ips/fhirpath-ips.js`, `ips/membership.js`). Clinical profiles (`/ips/StructureDefinition`), across all projects: what a kind of clinical document is expected to contain, as a profile on Bundle (the document) whose root element's constraints are the expectations - edited in three parts: required sections and entries (a section that must be there, or a section with entries with a code, or a code in a value set) and rules about resources (`Bundle.resources(system, code, type)` and the rest of the rule), each as a table with code lookups, and other rules as FHIRPath; the administrator makes, edits, downloads and uploads them (an upload gets a new id and url, and anything more than such a profile is rejected), and everyone can see them. See `ips/readme.md`
+- Daily report: a markdown report on the server, emailed at midnight (local time) - or as soon as the server is up again if it was down then - to `dailyReport.to`, titled "{hostName} daily report". A paragraph on the server's stability since the last report (uptime, restarts, the longest outage, any upgrade - from the run history in the statistics database, which now records each run's version), a paragraph from each running module that has one (a module implements `dailyReport({since, now})` to take part; by default none do), a paragraph on the system (version, Node.js and OS, how long this run has been up, the log files and the free disk space, and anything the npm audit found - vulnerable packages and the upgrade advice, or a newer release), and, when any background task's last run failed, a list of them. Nothing is sent without the new top-level `email` block (SMTP: `host`, `port`, `user`, `password`, `from`), which is the server's outbound email for anything else that needs it (`library/mailer.js`). New dependency: `nodemailer`. The terminology server, registry, packages, xig, testing, shl, ips and publisher modules each report (see the README for what); the counts come from the statistics database, which now answers per-operation counts over a period (`stats.countsSince`)
+- Home page and dashboard: the uptime box now also reports the server's overall uptime - `Uptime: 38m 16s (99.3% overall)` - the proportion of the time since the first recorded run that the server has been running. Each run is recorded in the statistics database (`server_runs` in `stats.db`) with a heartbeat every minute, so a crash loses at most a minute. Not shown when statistics are turned off
+- `library/fhir-search-values.js`: the date range, prefix and comma-splitting parts of FHIR search, moved out of the testing module for sharing
+- `library/package-manager.js`: a cached package folder with no `package/package.json` (a download that failed after the folder was made) is fetched again instead of being used
+- ICF (`http://hl7.org/fhir/sid/icf`): qualified codes. A category may be followed by `.` and one qualifier digit per position, the positions depending on the component: extent for body functions (`b`); extent, nature and localization for body structures (`s`); performance, capacity, and optionally capacity with assistance and performance without assistance for activities and participation (`d`). Qualifiers on environmental factors (`e`) are not supported yet. Each digit is checked against its scale in the code system (`q1`-`q4`, added to the ICF code systems in fhir.tx.support with a `kind` property). A qualified code gets a generated display (8, not specified, is left out of it; `d` qualifiers are labelled `performance`, `capacity`, ...), its qualifiers as properties in `$lookup` (`extent`, `nature`, `localization`, `performance`, `capacity`, `capacity-with-assistance`, `performance-without-assistance`), and sits under its category for subsumption and `is-a` filters. Value sets can control qualifiers with an `exists` `true`/`false` filter on any of those properties (8 counts as absent) and `qualified` `=` `false`. ICF is now marked as not closed. The behaviour is in `tx/cs/cs-icf.js`, a subclass of the CodeSystem provider that `FhirCodeSystemProvider.create()` picks by url - the way to give any code system loaded from a package behaviour of its own
+- `$translate`: SNOMED CT edition to edition translation with no map - from an edition to one it depends on (per the module dependency reference set), using the implicit map `<edition>/version/<date>?fhir_cm=edition/<target module>` (the url agreed with the SNOMED committee), found from `targetSystem` or nominated by url. A concept in the target edition's modules is its own translation; an inactive concept translates through its historical associations; a fully defined extension concept translates to its normal form (or the target concept with that normal form), when that uses only target edition content; anything else translates to its proximal target-edition ancestors as `source-is-narrower-than-target`, unless those are only top-level hierarchies, which is reported as the reason there is no translation. The source concept in a match carries its version, as given
+- `$translate`: when the target code a map directs to is not in its code system's normal form, the match has a `normalized-concept`. A source code that has a normal form is looked up both as given and in its normal form (a map entry written in the normal form matches any code with that normal form; the map's own entries are not normalised). A target in a code system the server doesn't know gets a `note` saying it was not validated, and a target code that is not valid gives an `error`
+- `$lookup`: a warning when the `displayLanguage` asked for has no display for the code, and the display is in the default language (`LOOKUP_DISPLAY_LANGUAGE_FALLBACK`)
+- ICD-11: the `classificationCode` extension (provisional url `http://id.who.int/icd/schema/extension/classificationCode`) on a Coding carries the short code for an entity id, saving a lookup. When present it is checked in `$validate-code` (CodeSystem and ValueSet): a wrong value, or the extension on a code with no short code, on a short code, or on a system other than MMS, is an error
+- ICD-11 `$lookup`: postcoordination is reported as one `postcoordination` property per axis (subproperties `axis` and `value` Codings, a `/` member's value with its own nested postcoordination), replacing the WHO `postcoordinationScale` shape. With the new parameter `post-coordination-metadata=true` every declared axis is listed with `required`, `allowMultipleValues` and its value set; otherwise only the valued axes. `source` (the Foundation entity) is a Coding, and asking for `source.details` adds what a `$lookup` on it would return, designations included. A single-valued axis with several values gets a message in `$validate-code`
+- SNOMED CT ECL: member filters (`^ refset {{ M ... }}`) - `effectiveTime` (with a time value or set), `active`, `moduleId` (any sub expression) and the reference set's own fields (`targetComponentId = << X`, `mapTarget = "..."`, `mapGroup >= #2`, ...), comma-separated and repeatable. A member filter anywhere other than directly after a memberOf constraint is a syntax error (`INVALID_ECL`). Description filters (`{{ D }}`), concept filters (`{{ C }}`) and history supplements (`{{ + }}`) are now parsed and rejected as `UNSUPPORTED_ECL` rather than failing as a lexer error
+- SNOMED CT ECL: concept filters (`{{ C ... }}`) - `active`, `definitionStatus` (primitive/defined), `definitionStatusId`, `moduleId` (a concept, a bracketed concept list or any sub expression) and `effectiveTime`
+- SNOMED CT ECL: `AND` and `OR` between refinements - between attributes (`: A = x AND B = y`, `: A = x OR B = y`, with brackets for nesting), within an attribute group (`{ A = x OR B = y }`) and between groups (`{ ... } OR { ... }`). Mixing AND and OR without brackets is rejected, as for compound expressions. Previously only `,` (conjunction) was accepted
+- SNOMED CT ECL: description filters (`{{ D ... }}`) - `term` (match and `wild:` search terms, single or a set), `language`, `typeId` and `type` (syn/fsn/def), `dialectId` and `dialect` (en-us, en-gb and the other aliases in the ECL specification) with acceptability sets, `moduleId`, `effectiveTime`, `active` and `id`. The filters in one block apply to the same description; only active descriptions are considered unless `active` is given
+- SNOMED CT ECL: comments (`/* ... */`), top and bottom of set (`!!> (...)`, `!!< (...)`), and attribute names that are expressions rather than a single concept (`: << 762705008 = ...`, `: (A OR B) = ...`, `: ^ refset = ...`, `: * = ...`, and the same after `.` in dotted expressions)
+- SNOMED CT ECL: concrete values in refinements (`= #5`, `!= "x"`, `= match:"x"`, `= true`, ...) and alternate identifiers (`"ICD10#A01.1"`, `SCTID#123`) and reference set field selection on member-of (`^ [targetComponentId] refset`) are now parsed, so they are reported as `UNSUPPORTED_ECL` with a message saying what is missing (concrete value relationships / alternate identifier reference sets are not loaded) rather than as a syntax error
+- SNOMED CT: `tx/sct/documentation.md` - what the server supports for SNOMED CT: editions, ECL, expressions and the MRCM, implicit concept maps and translation
+- `utilities/make-sct-test-extension.js`: builds the extension concepts (module TEST1000003) in the SNOMED CT test edition used by the tx tests - one of each shape (fully defined ungrouped, fully defined role-grouped, equivalent to an International concept, primitive), plus inactive ones with and without historical associations
+- README: each module's status - in production; in production but still under development; under development / prototyping; or on the back-burner
+
+### Changed
+
+- SNOMED CT expressions: validation now follows the MRCM the way Ontoserver does (from a comparison over every post-coordinated expression in the published FHIR packages). An attribute must be a concept model attribute (a descendant of 410662002) - linkage concepts such as 20401003 |With| and 280452008 |Interpretation| were allowed before - and an attribute the MRCM attribute domain reference set says nothing about (|Has definitional manifestation|, |Subject of information|, ...) is rejected rather than left unchecked. An attribute the MRCM marks as grouped that is written ungrouped (`40468003:363698007=10200004`) is now accepted and put in a relationship group of its own (the "self-grouped attribute" transformation from Close To User Form to Classifiable Form), as Ontoserver and Snowstorm do; it was rejected. The grouped form is what is rendered and used for the normal form
+- ICD-11 (MMS and ICF): following the October 2026 test-case review meetings, the code is the bare entity id (`257068234`; the full uri is the system url + `/` + the id), and that is the normal form. The short code (`1A00`) is still accepted, with `CODE_NOT_IN_NORMAL_FORM` and the normalized code; the full uri is no longer accepted as a code. The short code is reported as `classificationCode`: a `$lookup` property (the old `code` property; the old `id` property is now `code`), a property on every `$expand` contains entry, and a `$validate-code` output when the request used the id. Parent, child, stem and postcoordination value references are all entity ids, and postcoordinated expressions are written without spaces. The Foundation is unchanged
+- `$lookup`: `name` is the code system's title, translated where possible, falling back to `CodeSystem.name` (a new `lookupName()` on code system providers, so other operations are unaffected). SNOMED CT gives the edition (`SNOMED CT International Edition`), LOINC, UCUM and OMOP their titles; OMOP's `name()` is now `OMOP_Concepts`
+- `$lookup`: a code that is not selectable always returns `abstract` (true) and the `notSelectable` property (true), whatever properties were asked for; a selectable code returns neither unless asked (as agreed for the tx-ecosystem tests)
+- Dependencies: new `jose`, `nodemailer` and `qrcode-generator`; `fhir-validator-wrapper` 1.4.0 to 1.5.0
+- tx.fhir.org configuration: the Swiss and Danish SNOMED CT editions are no longer loaded, and `hl7.fhir.uv.ips` is
+
+### Fixed
+
+- IPS: two runs of the processing queue could overlap at start-up while the terminology server was being looked for, and a run could start before it had answered; the queue now claims itself before it waits, and any answer but a CapabilityStatement (a 5xx while the tx module loads, say) counts as "not there yet". Requeueing a document while it is being processed is refused (it lost the requeue and the results), and a pipeline only finishes a document that is still being processed. A clinical profile rule over a value set the terminology server couldn't answer for is "could not be checked" whatever it evaluates to, and a profile deleted while a document is checked is passed over. The migration of profiles from Composition to Bundle rewrites every `Composition.` in a rule, not just a leading one. FHIRPath string escapes (`\uXXXX`, `\f`, ...) are read in the profile editor's shapes. A display lookup the server couldn't answer isn't cached. The editor refuses two fields for one `value[x]`, reads each `value[x]` field back from its own type, and keeps paragraphs on a round trip. A huge `_offset` is a 400, not a 500
+- Registry: sessions are kept in the auth database (logins survive a restart), and pages a visitor only reads start no session. Documents up to the 2 MB limit can be saved (the form parser stopped at 64 KB). An endpoint missing from one crawl (a sub-registry that failed to fetch, a crawl cut short at shutdown) keeps its uptime history; history only ages out, and a scan cut short isn't recorded as a failure. Usage counts survive a failed write. The main pages' navigation shows who is logged in
+- Server: SIGTERM (systemd, docker stop) shuts the server down as SIGINT does, so modules save what they hold
+- ValueSet `$validate-code`: an include with more than one filter only checked the first. The other filters' checks return a message when the code fails them, and a message was taken as a pass
+- LOINC: the `exists` filter ignored its value. On a relationship property (`METHOD_TYP`, `COMPONENT`, `document-kind`, ...) the value was looked up as a code or display, so `exists true` and `exists false` both returned every code with the relationship (because a code displayed as "True" and one as "False" exist), and on a property (`CLASSTYPE`, `ORDER_OBS`, ...) `exists false` meant `exists true`. `true` now returns the codes that have the property, `false` every other code in the code system - Parts, AnswerLists and Answers included, so add `CLASSTYPE exists true` to restrict to LOINC terms (e.g. LOINC terms with no method: `METHOD_TYP exists false` + `CLASSTYPE exists true`). Any other value is not supported
+- Packages: a package download or page request was counted twice in the statistics (once by the validation step, once by the handler), and tarball downloads were not told from requests for a version's JSON document; downloads are now counted as `download`, version documents as `version`, each once
+- SNOMED CT ECL: a hierarchy operator on a bracketed expression was silently dropped - `< (A OR B)` returned A and B themselves rather than their descendants. It now applies to every member of the set, as do operators on `*` and on `^ refset`, which were previously rejected as unsupported
+- SNOMED CT ECL: `^` on a reference set whose members are descriptions (a language reference set) returned an empty expansion; it is now reported as `UNSUPPORTED_ECL`
+- SNOMED CT ECL: an evaluation error is now `UNSUPPORTED_ECL` unless it is about a concept that does not exist (`INVALID_ECL`); it was classified by matching the error text, which filed `= true` as a syntax error. A concept id that is not 6-18 digits is rejected by the parser
+- SNOMED CT expressions: the MRCM content type taken as "postcoordinated" was 723594008, which is *all precoordinated content* (postcoordinated is 723595009). So the body-structure part-of attributes (|Regional part of| etc., precoordinated only) were allowed in expressions, |Has ingredient| (postcoordinated only) was not checked, and for the attributes with a precoordinated-specific range that range was used in place of the general one
+
+- SHL: every access to a live link (`/shl/access/{uuid}`) hung without a response - the access-logging callback was a plain `function`, so `this.db` was undefined in it and the request was never answered (since 0.3.0). A `GET` on a link is now the U flag (direct file) access the SMART Health Links spec defines: a link with one file returns it as a JWE (`application/jose`), and `recipient` is read from the query and logged (it was always `anonymous`); a link with more than one file, or a VHL, still gets the manifest / VHL bundle. The manifest now has `status` (`finalized`), file `location`s are always https (they followed `req.secure`, so http behind a proxy without trust proxy), and `embedded` is the JWE itself (it was the base64 of it, as stored)
+- IPS: a VHL share no longer puts the U flag in its link payload - a VHL's link answers with the VHL bundle, not the file
+
+### Tx Conformance Statement
+
+FHIRsmith v0.18.0 passed all 3860 HL7 terminology service tests (modes tx.fhir.org+omop+general+snomed+mimetypes+icd-11+icf+closure, tests v1.9.6, runner v7.0.3)
+
 ## [0.17.0] - 2026-10-17
 
 An extensive security review, which hardened many things, but no actual securty notifications needed. Plus improvemnts to text search in terminology filters

package/README.md

@@ -1,22 +1,37 @@
 # ![🔥](static/FHIRsmith64.png) FHIRsmith - FHIR Server toolkit
 
-This server provides a set of server-side services that are useful for the FHIR Community. The set of are two kinds of services:
+This server provides a set of server-side services that are useful for the FHIR Community. Each is a module, turned on and configured in `config.json`. The modules are at different stages, and stability varies between them: a module in production can still be changing in places (and the terminology server's behaviour follows decisions about its content - ICD-11, SNOMED CT - as they are made).
 
-## Modules useful to anyone in the community
+## In production
 
-* (Coming) R4/R6 interconverter
-* [terminology](tx/README.md) server (as running on [tx.fhir.org](http://tx.fhir.org/r4))
-* [SHL Server](shl/readme.md) - SHL/VHL support services (running on [healthintersections.com.au](http://www.healthintersections.com.au), supporting the ICVP demo and and the Patient Document Generator)
+* [Terminology server](tx/README.md) (`tx`) - the reference terminology server for the FHIR community and the terminology ecosystem (as running on [tx.fhir.org](http://tx.fhir.org/r4))
+* [TX Registry](registry/documentation/readme.md) (`registry`) - **Terminology System Registry** as [described by the terminology ecosystem specification](https://build.fhir.org/ig/HL7/fhir-tx-ecosystem-ig) (as running at http://tx.fhir.org/tx-reg, moving to https://ecosystem.hl7.org). Its API is described by an OpenAPI 3.1 spec at `/tx-reg/openapi.json` (and `.yaml`), with a browsable reference at `/tx-reg/openapi`
+* [Package server](packages/readme.md) (`packages`) - **NPM-style FHIR package registry** with search, versioning, and downloads, consistent with the FHIR NPM Specification (as running at http://packages2.fhir.org/packages). Its API is described by an OpenAPI 3.1 spec at `/packages/openapi.json` (and `.yaml`), with a browsable reference at `/packages/openapi`
+* Publisher (`publisher`) - FHIR publishing services (as running at [healthintersections.com.au](http://www.healthintersections.com.au/publisher))
+* XIG server (`xig`) - **Comprehensive FHIR IG analytics** with resource breakdowns by version, authority, and realm (as running at http://packages2.fhir.org/packages)
+* [SHL Server](shl/readme.md) (`shl`) - SHL/VHL support services (running on [healthintersections.com.au](http://www.healthintersections.com.au), supporting the ICVP demo and the Patient Document Generator)
 
-## Services useful the community as a whole
+## In production, but still under development
 
-* [TX Registry](registry/readme.md) - **Terminology System Registry** as [described by the terminology ecosystem specification](https://build.fhir.org/ig/HL7/fhir-tx-ecosystem-ig) (as running at http://tx.fhir.org/tx-reg). Its API is described by an OpenAPI 3.1 spec at `/tx-reg/openapi.json` (and `.yaml`), with a browsable reference at `/tx-reg/openapi`
-* [Package server](packages/readme.md) - **NPM-style FHIR package registry** with search, versioning, and downloads, consistent with the FHIR NPM Specification (as running at http://packages2.fhir.org/packages). Its API is described by an OpenAPI 3.1 spec at `/packages/openapi.json` (and `.yaml`), with a browsable reference at `/packages/openapi`
-* [XIG server](xig/readme.md) -  **Comprehensive FHIR IG analytics** with resource breakdowns by version, authority, and realm (as running at http://packages2.fhir.org/packages)
-* [Publisher](publisher/readme.md) - FHIR publishing services (as running at [healthintersections.com.au](http://www.healthintersections.com.au/publisher))
-* [Testing](testing/readme.md) - **TestReport repository**: receives FHIR TestReports from TxTester and other test tools, with a FHIR API and web pages (as running at https://npm.fhir.org/testing). Its API is described by an OpenAPI 3.1 spec at `/testing/openapi.json` (and `.yaml`), with a browsable reference at `/testing/openapi`
-* [VCL](vcl/readme.md) - **Parse VCL expressions** into FHIR ValueSet resources (as running at http://fhir.org/vcl)
-* (Coming) Token services
+* [Extension tracker](extension-tracker/readme.md) (`ext-tracker`) - tracks extensions, profiles and extension usage across FHIR Implementation Guides, from data the IG Publisher submits
+* [Folder](folder/readme.md) (`folder`) - serves static files from one or more directories, with optional authenticated upload
+* [Testing](testing/readme.md) (`testing`) - **TestReport repository**: receives FHIR TestReports from TxTester and other test tools, with a FHIR API and web pages (as running at https://npm.fhir.org/testing). Its API is described by an OpenAPI 3.1 spec at `/testing/openapi.json` (and `.yaml`), with a browsable reference at `/testing/openapi`
+* VCL (`vcl`) - **Parse VCL expressions** into FHIR ValueSet resources (as running at http://fhir.org/vcl)
+
+## Under development, prototyping
+
+* [IPS](ips/readme.md) (`ips`) - collections of International Patient Summary documents, each with its own FHIR endpoint, processing (validation, terminology checks, translation, narrative, clinical profiles) and web pages
+
+## On the back-burner
+
+These are not being actively developed at present.
+
+* [NpmProjector](npmprojector/readme.md) (`npmprojector`) - watches a local npm package directory and serves its FHIR resources with FHIRPath-based search
+* Token (`token`) - API key services
+
+## Planned
+
+* R4/R6 interconverter
 
 ## Summary Statement
 
@@ -159,6 +174,58 @@ Add a `logging` section to `config.json` to control log behaviour. All fields ar
 
 Log files are written to the `logs/` subdirectory of the data directory as `server-YYYY-MM-DD.log`. A `server.log` symlink always points to the current day's file, so `tail -f data/logs/server.log` tracks the active log without needing to know the date.
 
+### Email and the Daily Report
+
+The server can email a short markdown report on itself once a day. It needs two things in
+`config.json`: an SMTP server to send through, and someone to send to:
+
+```json
+{
+  "email": {
+    "host": "smtp.example.org",
+    "port": 587,
+    "user": "fhirsmith",
+    "password": "...",
+    "from": "FHIRsmith <fhirsmith@example.org>"
+  },
+  "dailyReport": {
+    "to": ["ops@example.org"]
+  }
+}
+```
+
+`user`/`password` are optional (an open relay); `secure` defaults to true for port 465 and
+to STARTTLS otherwise. Without an `email` block nothing is emailed, and without
+`dailyReport.to` there's no report (`dailyReport.enabled: false` turns it off explicitly).
+
+The report, titled "`{hostName}` daily report", goes out at midnight (the server's local
+time), or as soon as the server is up again if it was down at midnight. It has a paragraph
+on the server's stability since the last report - how much of the time it was up, restarts,
+the longest outage, and any upgrade (from the run history in the statistics database, so
+`stats.enabled: false` leaves the uptime unknown); a paragraph from each running module that
+has something to report (by default, none do); and a paragraph on the system - version,
+Node.js and OS, how long this run has been up, the log files and the free disk space they're
+drawing on, plus anything the npm audit check has found (packages with known vulnerabilities and
+what to do about them, or a newer FHIRsmith release). If any background task's last run failed, a
+final section lists them. When the last report was sent is kept in `daily-report.json` in the data
+directory.
+
+A module takes part by implementing `dailyReport({since, now})`, returning its paragraph as
+markdown (or a promise of it); an empty return means it has nothing to say that day. The
+request counts a module quotes come from `stats.countsSince(since, now)`, the per-operation
+counts in the statistics database for the period. The modules that report:
+
+| Module | Reports |
+|---|---|
+| Terminology server | how many of each operation were performed, over all its endpoints |
+| Terminology server registry | client requests answered by the API, servers registered, and any server that answered none of the registry's scans in the period |
+| Packages | packages served (tarball downloads) and new package versions found by the crawler |
+| XIG | pages served, and whether the database could be accessed (and whether the last download failed) |
+| Testing | pages served, test reports received, and the number held |
+| SMART Health Links | QR codes created (SMART Health Links and Verifiable Health Links), link accesses, and the documents stored to serve them |
+| IPS | pages served, IPS documents received, and the number stored |
+| Publisher | publication runs started, and how many finished successfully |
+
 ### Start the Server
 
 ```bash

package/config-template.json

@@ -27,6 +27,24 @@
     "enabled": true,
     "intervalHours": 24
   },
+  // Outbound email, over SMTP. Optional: without it nothing is emailed. user/password are
+  // optional too (an open relay); secure defaults to true for port 465, otherwise STARTTLS
+  "email": {
+    "host": "smtp.example.org",
+    "port": 587,
+    "user": "fhirsmith",
+    "password": "<!-- password -->",
+    "from": "FHIRsmith <fhirsmith@example.org>"
+  },
+  // A markdown report on the server - its uptime since the last report, restarts and
+  // upgrades, a paragraph from any module with something to say, and the version, logs
+  // and free disk space - emailed at midnight (local time), or as soon as the server is
+  // up again if it was down then. Titled "{hostName} daily report". Needs "email" above;
+  // the whole block is optional
+  "dailyReport": {
+    "enabled": true,
+    "to": ["ops@example.org"]
+  },
   "modules": {
     "shl": {
       "enabled": false,
@@ -108,7 +126,17 @@
       "userAgent": "YourServer/1.0",
       "apiKeys": {},
       // the most a crawled server's response can be, once unzipped
-      "maxResponseMB": 100
+      "maxResponseMB": 100,
+      // how far back the software page reports each server's uptime, and how long scan outcomes are kept
+      "uptimeWindowDays": 30,
+      "uptimeRetentionDays": 90
+      // Securing the JSON API (the HTML pages stay open). Omit for an open registry. Any of the
+      // three can be configured, alone or together - see registry/documentation/security.md
+      // "security": {
+      //   "apiToken": "one token every client sends",
+      //   "users": { "database": "registry-auth.db", "adminUsername": "admin", "adminPassword": "...", "sessionSecret": "...", "cookieSecure": true },
+      //   "oidc": { "issuer": "https://login.microsoftonline.com/{tenant}/v2.0", "audience": "api://ecosystem.hl7.org", "requiredScope": "registry.read" }
+      // }
     },
     "publisher": {
       "enabled": true,
@@ -225,6 +253,14 @@
       // several instances can route cached requests back to the instance that holds the
       // cache. Leave unset for a single server. See nginx.md.
       // "instanceCode": "tx1",
+      // The syndication feed (Terminology Syndication Feed IG) at <endpoint>/syndication. When
+      // it is on, the packages, resource files and published binaries the library loads are
+      // recorded in <data>/terminology-cache/syndication as they load - only from the time it
+      // is turned on - and the feed is built from that record. The full CPT database is never
+      // syndicated. Every endpoint shares the one store.
+      "syndication": {
+        "enabled": false
+      },
       // $closure: closure tables are kept on the server, across sessions and restarts, so
       // the operation is off unless it is turned on here. When it is on, only one server
       // instance may use the database - route every $closure request to it (see nginx.md).
@@ -318,6 +354,75 @@
       "rateLimit": { "windowMinutes": 1, "max": 60 }
       // delete reports received more than this many days ago. Leave it out to keep everything
       // , "retentionDays": 365
+    },
+    // International Patient Summary documents, in projects, at /ips. See ips/readme.md
+    "ips": {
+      "enabled": false,
+      // relative paths resolve under the data folder's databases directory
+      "database": "ips/ips-content.db",
+      // the administrator: creates and deletes projects and users, and says who manages
+      // which project. No adminPassword, no administrator
+      "adminUsername": "admin",
+      // "adminPassword": "<!-- secret -->",
+      // signs the login cookie; without it logins don't survive a restart
+      // "sessionSecret": "<!-- random string -->",
+      // the login cookie is HTTPS-only unless this is false (e.g. for local testing over http)
+      // "cookieSecure": true,
+      // failed-login attempts allowed per IP per 15 minutes (default 20)
+      "loginAttemptsPerWindow": 20,
+      // the biggest document accepted, in bytes (default 10MB)
+      "maxSize": 10485760,
+      // documents submitted per IP address per window; max 0 turns the limit off
+      "rateLimit": { "windowMinutes": 1, "max": 60 },
+      // the background loop that processes received documents
+      "processing": { "enabled": true, "intervalSeconds": 5 },
+      // the terminology server, for links to code systems and for analysing documents:
+      // an http(s) URL, or - when the tx module is enabled - one of its endpoint paths
+      // (modules.tx.endpoints[].path, e.g. /r4), to which requests are then routed
+      // in-process. Omit it for no links and no analysis
+      "txServer": "/tx/r4",
+      // "txServer": "https://tx.fhir.org/r4",
+      // (rateLimit also takes lookupMax: the editor's code lookups per window, 600)
+      // the AI assessment (the last processing step; a project turns it on in its settings).
+      // The server's default account, used by a project that has not set its own: provider
+      // is openai (any chat-completions service: OpenAI, OpenRouter, LiteLLM, Groq, Ollama...)
+      // or anthropic (the Messages API); baseUrl is optional (the provider's own by default;
+      // for an OpenAI-compatible service, up to the version, e.g. https://openrouter.ai/api/v1).
+      // dailyCap: assessments per project per day (50); maxBytes: the biggest document sent
+      // (300000); maxTokens: of the answer (16384); timeout: seconds (120); sendNarrative: send
+      // the narratives as submitted (false: the structured data only); a project can override
+      // all but maxBytes, maxTokens and timeout
+      // prices: USD per million tokens by model (or a prefix of the model name), to work out what
+      // each assessment cost, for services that don't say (OpenRouter does). Anthropic's current
+      // models are built in (ips/ai-client.js DEFAULT_PRICES); anything here overrides them
+      // "ai": { "provider": "anthropic", "model": "claude-sonnet-5-5", "apiKey": "...", "dailyCap": 50,
+      //         "prices": { "claude-sonnet-5-5": { "input": 3, "output": 15 } } },
+      // the secret the projects' own AI keys are encrypted with in the database (else the
+      // sessionSecret is used; with neither, projects cannot keep their own accounts)
+      // "aiKeySecret": "<!-- random string -->",
+      // sharing a document as a QR code - a SMART Health Link, or a Verifiable Health Link
+      // signed as the GDHCN does it - through an SHL server: this server's own shl module
+      // ("server": "/shl"; the password is taken from modules.shl when left out) or another
+      // FHIRsmith's ("server": "https://host/shl", "password": its password). days: how long
+      // the shared copy lives (100); viewer: the SHL viewer the QR opens (viewer.tcpdev.org)
+      // "shl": { "server": "/shl", "days": 100 },
+      // the FHIR validator the processing pipeline validates documents with - the same
+      // validator service the shl module uses (fhir-validator-wrapper, the validator CLI jar in
+      // the data folder's bin directory), on its own port. Documents are validated against the
+      // IPS Bundle profile unless "profiles" says otherwise. Omit the block, or set enabled
+      // false, and documents are not validated (processing fails)
+      "validator": {
+        "enabled": true,
+        "version": "4.0.1",
+        "txServer": "http://tx.fhir.org/r4",
+        "txLog": "./ips-validator-tx.log",
+        "port": 8082,
+        "packages": ["hl7.fhir.uv.ips#1.1.0"],
+        "timeout": 120000,
+        // the validator's own IPS code checks (default true)
+        "checkIpsCodes": true
+        // , "profiles": ["http://hl7.org/fhir/uv/ips/StructureDefinition/Bundle-uv-ips"]
+      }
     }
   }
 }