fhirsmith 0.3.0

package/CHANGELOG.md

@@ -0,0 +1,42 @@
+# Changelog
+
+All notable changes to the Health Intersections Node Server will be documented in this file.
+
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
+and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+## [v0.3.0] - 2026-02-05
+### Added
+- Add first draft of publishing engine
+
+### Changed
+- Move all runtime files to a data directory, where an environment variable says. Existing configurations MUST change
+- Finish porting the terminology server
+- Lots of QA related changes, and consistency.
+
+## [v0.2.0] - 2026-01-13
+### Added
+- port tx.fhir.org to FHIRsmith, and pass all the tests
+
+### Changed
+- rework logging, testing, etc infrastructure
+
+## [v0.1.1] - 2025-08-21
+### Added
+- set up ci and release workflows with Docker
+- Add tx-reg implementation
+
+### Changed
+
+- rework logging from scratch 
+
+## [v0.1.0] - 2025-08-20
+
+First Documented Release 
+
+### Added
+- SHL Module: Support services for SHL and VHL implementations
+- VCL Module: Support services for ValueSet Compose Language 
+- XIG Module: The Cross-IG Resource server 
+- Packages Modules: The server for packages2.fhir.org/packages 
+- Testing Infrastructure

package/README.md

@@ -0,0 +1,277 @@
+# ![🔥](static/FHIRsmith64.png) FHIRsmith - FHIR Server toolkit
+
+
+
+This server provides a set of server-side services that are useful for the FHIR Community. The set of are two kinds of services:
+
+## Modules useful to anyone in the community
+
+* (Coming) R4/R6 interconverter
+* [tx.fhir.org](tx/README.md) server
+* [SHL Server](shl/readme.md) - SHL/VHL support services
+
+## Services useful the community as a whole
+
+* [TX Registry](registry/readme.md) - **Terminology System Registry** as [described by the terminology ecosystem specification](https://build.fhir.org/ig/HL7/fhir-tx-ecosystem-ig)(as running at http://tx.fhir.org/tx-reg)
+* [Package server](packages/readme.md) - **NPM-style FHIR package registry** with search, versioning, and downloads, consistent with the FHIR NPM Specification (as running at http://packages2.fhir.org/packages)
+* [XIG server](xig/readme.md) -  **Comprehensive FHIR IG analytics** with resource breakdowns by version, authority, and realm (as running at http://packages2.fhir.org/packages)
+* [Publisher](publisher/readme.md) - FHIR publishing services (coming)
+* [VCL](vcl/readme.md) - **Parse VCL expressions** into FHIR ValueSet resources for http://fhir.org/vcl
+* (Coming) Token services
+
+## Build Status
+![CI Build](https://github.com/HealthIntersections/fhirsmith/actions/workflows/ci.yml/badge.svg)
+[![Release](https://img.shields.io/github/v/release/HealthIntersections/fhirsmith?include_prereleases)](https://github.com/HealthIntersections/fhirsmith/releases)
+[![Docker](https://img.shields.io/badge/docker-ghcr.io-blue)](https://github.com/HealthIntersections/fhirsmith/pkgs/container/fhirsmith)
+
+Note: In production, this server always runs behind an nginx reverse proxy, so there's no
+in-build support for SSL, rate limiting etc.
+
+## Quick Start
+
+There are 4 executable programs:
+* the server (`node server`)
+* the test cases (`npm test`)
+* the terminology importer (`node --max-old-space-size=8192 tx/importers/tx-import XXX`) - see [Doco](tx/importers/readme.md)
+* the test cases generater (`node tx/tests/testcases-generator.js`)
+
+### Data Directory
+
+The server separates code from runtime data. All databases, caches, logs, and downloaded
+files are stored in a single data directory. The location is determined by:
+
+1. The `FHIRSMITH_DATA_DIR` environment variable (if set)
+2. Otherwise, defaults to `./data` relative to the working directory
+
+The data directory contains (depending on which modules are in use):
+* `config.json` — server and module configuration
+* `logs/` — server and nginx log files
+* `terminology-cache/` — downloaded terminology packages and FHIR packages
+* `packages/` — package server database
+* `xig/` — XIG database
+* `shl/` — SHL databases and certificates
+* `registry/` — registry crawler data
+* `publisher/` — publisher database and build workspace
+* `token/` — token database
+
+During development with a cloned repository, the data directory defaults to `[root]/data`
+(the test cases require this setup). When deployed via Docker or npm, the data directory
+is provided by the host — see Deployment below.
+
+### Prerequisites
+- Node.js 16+
+- NPM or Yarn
+- Java 17+ (for FHIR validator, also for the test cases)
+
+### Installation
+
+These instructions are for Development. For deployment, see below.
+
+```bash
+# Clone the repository
+git clone https://github.com/HealthIntersections/FHIRsmith
+cd FHIRsmith
+
+# Install dependencies
+npm install
+
+# Create required directories
+mkdir -p data data/logs
+
+# Copy example configuration
+cp config.example.json data/config.json
+
+# Edit configuration as needed
+nano data/config.json
+```
+
+Each Module has it's own entry in the config, as described by the module
+
+### Basic Configuration
+
+Create a `config.json` file in your data directory (use `config-template.json` as a starting point):
+
+```json
+{
+  "hostName" : "[descriptive name for the server]",
+  "server": {
+    "port": 3000,
+    "cors": {
+      "origin": "*",
+      "credentials": true
+    }
+  },
+  "modules": {
+    // per modules...
+  }
+}
+```
+
+### Start the Server
+
+```bash
+# Development mode
+npm run dev
+
+# Production mode
+npm start
+```
+
+The server will be available at `http://localhost:{port}` using the port specified in the config.
+In the production servers listed above, the server always sits behind an NGINX server which manages
+SSL, security, rate limiting etc.
+
+## Testing
+
+```bash
+npm test
+```
+
+## Deployment
+
+There are three deployment options: npm global install, Docker, or clone-and-run. All three
+use the `FHIRSMITH_DATA_DIR` environment variable to locate the data directory.
+
+### npm Global Install
+
+```bash
+# Install globally
+npm install -g fhirsmith
+
+# Create a data directory
+mkdir -p /var/lib/fhirsmith
+cp node_modules/fhirsmith/config-template.json /var/lib/fhirsmith/config.json
+# Edit config.json as needed
+
+# Set the data directory and run
+export FHIRSMITH_DATA_DIR=/var/lib/fhirsmith
+fhirsmith
+```
+
+Or run it inline:
+
+```bash
+FHIRSMITH_DATA_DIR=/var/lib/fhirsmith fhirsmith
+```
+
+### Docker Installation
+
+The server is available as a Docker image. Mount a host directory as the data directory:
+
+```bash
+# Pull the latest image
+docker pull ghcr.io/healthintersections/fhirsmith:latest
+
+# Create and populate data directory on host
+mkdir -p /path/to/data
+cp config-template.json /path/to/data/config.json
+# Edit config.json as needed
+
+# Run with data directory mounted
+docker run -d --name fhirsmith \
+  -p 3000:3000 \
+  -e FHIRSMITH_DATA_DIR=/app/data \
+  -v /path/to/data:/app/data \
+  ghcr.io/healthintersections/fhirsmith:latest
+```
+
+Available tags:
+- `latest`: Latest stable release
+- `vX.Y.Z`: Specific version (e.g., `v1.0.0`)
+- `cibuild`: Latest build from the main branch
+
+### Environment Variables
+
+| Variable | Description | Default |
+|---|---|---|
+| `FHIRSMITH_DATA_DIR` | Path to the data directory | `./data` |
+| `PORT` | Server port (overrides config) | from config.json |
+| `NODE_ENV` | Node environment | `production` |
+
+### Windows Installation
+
+You can install as a windows service using [windows-install.js](windows-install.js). You might need to 
+hack that. 
+
+## Releases
+
+This project follows [Semantic Versioning](https://semver.org/) and uses a [CHANGELOG.md](CHANGELOG.md) file to track changes.
+
+### What's in a Release
+
+Each GitHub Release includes:
+- **Release notes** extracted from CHANGELOG.md
+- **Source code** archives (zip and tar.gz)
+- **Docker images** pushed to GitHub Container Registry:
+  - `ghcr.io/healthintersections/fhirsmith:latest`
+  - `ghcr.io/healthintersections/fhirsmith:vX.Y.Z`
+  - `ghcr.io/healthintersections/fhirsmith:X.Y.Z`
+- **npm package** published to npmjs.org as `fhirsmith` *(if you add this)*
+
+### Creating a Release
+
+GitHub Actions will automatically:
+- Run tests
+- Create a GitHub Release with notes from CHANGELOG.md
+- Build and publish Docker images with appropriate tags
+
+**Prerequisites:**
+- All tests passing on main branch
+- CHANGELOG.md updated with changes
+
+**Steps:**
+1. Update `CHANGELOG.md` with your changes under a new version section:
+```markdown
+   ## [vX.Y.Z] - YYYY-MM-DD
+   ### Added
+   - New feature description
+   ### Changed
+   - Change description
+   ### Fixed
+   - Bug fix description
+```
+2. Update `package.json` to have the same release version
+
+3. Commit your changes:
+```bash
+   git commit -m "Prepare release vX.Y.Z"
+   git push origin main
+```
+
+4. Tag and push the release:
+```bash
+   git tag vX.Y.Z
+   git push origin vX.Y.Z
+```
+
+5. Monitor the release:
+   - Check [GitHub Actions](https://github.com/HealthIntersections/fhirsmith/actions) for the Release workflow
+   - Verify the [GitHub Release](https://github.com/HealthIntersections/fhirsmith/releases) was created
+   - Confirm Docker images are available at [GHCR](https://github.com/HealthIntersections/fhirsmith/pkgs/container/fhirsmith)
+
+6. Update `package.json` to have the next release version -SNAPSHOT
+
+**If a release fails:**
+- Delete the tag: `git tag -d vX.Y.Z && git push origin :refs/tags/vX.Y.Z`
+- Fix the issue
+- Re-tag and push
+
+### Creating a Release
+
+## License
+
+[BSD-3](https://opensource.org/license/bsd-3-clause)
+
+## Contributing
+
+1. Fork the repository
+2. Create a feature branch
+3. Make your changes
+4. Add tests if applicable
+5. Submit a pull request
+
+## Support
+
+- **Issues:** [GitHub Issues](repository-url/issues)
+- **Documentation:** [Wiki](repository-url/wiki)
+- **FHIR Community:** [chat.fhir.org](https://chat.fhir.org)

package/config-template.json

@@ -0,0 +1,144 @@
+{
+  "server": {
+    "port": 3000,
+    "cors": {
+      "origin": true,
+      "credentials": true
+    }
+  },
+  "modules": {
+    "shl": {
+      "enabled": false,
+      "database": "shl.db",
+      "password": "<!-- Uuid -->",
+      "certificates": {
+        "certFile": "<!-- filename -->",
+        "keyFile": "<!-- filename -->",
+        "kid": "<!-- kid -->"
+      },
+      "vhl": {
+        "issuer": "XXX"
+      },
+      "cleanup": {
+        "schedule": "0 * * * *"
+      },
+      "validator": {
+        "enabled": true,
+        "version": "4.0.1",
+        "txServer": "http://tx.fhir.org/r4",
+        "txLog": "./tx.log",
+        "port": 8081,
+        "packages": [
+          "hl7.fhir.us.core#6.0.0",
+          "hl7.fhir.uv.ips#1.1.0"
+        ],
+        "timeout": 60000
+      }
+    },
+    "vcl": {
+      "enabled": false
+    },
+    "xig": {
+      "enabled": false,
+      "autoUpdate": true
+    },
+    "packages": {
+      "enabled": false,
+      "database": "/absolute/path/to/packages.db",
+      "mirrorPath": "/absolute/path/to/mirror",
+      "crawler": {
+        "enabled": true,
+        "schedule": "0 * * * *"
+      }
+    },
+    "registry": {
+      "enabled": true,
+      "masterUrl": "https://fhir.github.io/ig-registry/tx-servers.json",
+      "crawlInterval": 30,
+      "timeout": 30000,
+      "userAgent": "YourServer/1.0",
+      "apiKeys": {}
+    },
+    "publisher": {
+      "enabled": true,
+      "database": "/absolute/path/to/publisher.db",
+      "sessionSecret": "change-this-to-a-secure-random-string",
+      "workspaceRoot": "/absolute/path/to/workspaces"
+    },
+    "npmprojector": {
+      "enabled": true,
+      "fhirVersion": "r4",  // Options: "r4", "r5", "stu3", "dstu2"
+      "basePath": "/fhir",   // Default: "/npmprojector"
+      "npmPath": "./data/fhir-package",
+      "resourceTypes": null,
+      "searchParametersPath": null,
+      "debounceMs": 500
+    },
+    "token": {
+      "enabled": true,
+      "database": "/absolute/path/to/token.db",
+      "sessionSecret": "change-this-to-a-secure-random-string-for-token-sessions",
+      "oauth": {
+        "google": {
+          "clientId": "your-google-client-id",
+          "clientSecret": "your-google-client-secret",
+          "redirectUri": "http://localhost:3000/token/callback/google",
+          "scope": ["openid", "profile", "email"]
+        },
+        "facebook": {
+          "clientId": "your-facebook-app-id",
+          "clientSecret": "your-facebook-app-secret",
+          "redirectUri": "http://localhost:3000/token/callback/facebook",
+          "scope": ["email"]
+        },
+        "apple": {
+          "clientId": "your-apple-service-id",
+          "teamId": "your-apple-team-id",
+          "keyId": "your-apple-key-id",
+          "privateKey": "/path/to/apple-private-key.p8",
+          "redirectUri": "http://localhost:3000/token/callback/apple",
+          "scope": ["email", "name"]
+        },
+        "github": {
+          "clientId": "your-github-client-id",
+          "clientSecret": "your-github-client-secret",
+          "redirectUri": "http://localhost:3000/token/callback/github",
+          "scope": ["user:email"]
+        }
+      },
+      "apiKeys": {
+        "defaultAllowedRequests": 50,
+        "maxKeysPerUser": 10,
+        "keyExpiration": null
+      },
+      "rateLimit": {
+        "windowMs": 900000,
+        "max": 100
+      }
+    },
+    "tx": {
+      "enabled": false,
+      "librarySource": "/absolute/path/to/library.yml",
+      "cacheTimeout": 30,
+      "expansionCacheSize": 1000,
+      "expansionCacheMemoryThreshold": 0,
+      "endpoints": [
+        {
+          "path": "/tx/r5",
+          "fhirVersion": "5.0",
+          "context": null
+        },
+        {
+          "path": "/tx/r4",
+          "fhirVersion": "4.0",
+          "context": null
+        },
+        {
+          "path": "/tx/r3",
+          "fhirVersion": "3.0",
+          "context": null
+        }
+      ]
+    }
+  }
+}

package/library/folder-setup.js

@@ -0,0 +1,58 @@
+const path = require('path');
+const fs = require('fs');
+
+class FolderSetup {
+  constructor() {
+    this._dataDir = null;
+    this._initialized = false;
+  }
+
+  init(dataDir = null) {
+    if (this._initialized) {
+      return this;
+    }
+
+    this._dataDir = dataDir || process.env.FHIRSMITH_DATA_DIR || path.join(__dirname, '..', 'data');
+    fs.mkdirSync(this._dataDir, { recursive: true });
+    this._initialized = true;
+
+    return this;
+  }
+
+  dataDir() {
+    if (!this._initialized) {
+      this.init();
+    }
+    return this._dataDir;
+  }
+
+  subDir(name) {
+    const dir = path.join(this.dataDir(), name);
+    fs.mkdirSync(dir, { recursive: true });
+    return dir;
+  }
+
+  filePath(...relativePath) {
+    return path.join(this.dataDir(), ...relativePath);
+  }
+
+  ensureFilePath(...relativePath) {
+    const filePath = path.join(this.dataDir(), ...relativePath);
+    fs.mkdirSync(path.dirname(filePath), { recursive: true });
+    return filePath;
+  }
+
+  logsDir() {
+    return this.subDir('logs');
+  }
+
+  cacheDir() {
+    return this.subDir('cache');
+  }
+
+  databasesDir() {
+    return this.subDir('databases');
+  }
+}
+
+module.exports = new FolderSetup();

package/library/html-server.js

@@ -0,0 +1,166 @@
+
+//
+// Copyright 2025, Health Intersections Pty Ltd (http://www.healthintersections.com.au)
+//
+// Licensed under BSD-3: https://opensource.org/license/bsd-3-clause
+//
+
+const fs = require('fs');
+const path = require('path');
+
+class HtmlServer {
+  log;
+
+  constructor() {
+    this.templates = new Map(); // templateName -> template content
+  }
+
+  useLog(logv) {
+    this.log = logv;
+  }
+
+  // Template Management
+  loadTemplate(templateName, templatePath) {
+    try {
+      if (fs.existsSync(templatePath)) {
+        const templateContent = fs.readFileSync(templatePath, 'utf8');
+        this.templates.set(templateName, templateContent);
+        return true;
+      } else {
+        this.log.error(`Template file not found: ${templatePath}`);
+        return false;
+      }
+    } catch (error) {
+      this.log.error(`Failed to load template '${templateName}':`, error.message);
+      return false;
+    }
+  }
+
+  getTemplate(templateName) {
+    return this.templates.get(templateName);
+  }
+
+  hasTemplate(templateName) {
+    return this.templates.has(templateName);
+  }
+
+  // HTML Utilities
+  escapeHtml(text) {
+    if (typeof text !== 'string') {
+      return String(text);
+    }
+    
+    const map = {
+      '&': '&',
+      '<': '&lt;',
+      '>': '&gt;',
+      '"': '&quot;',
+      "'": '&#39;'
+    };
+    
+    return text.replace(/[&<>"']/g, function(m) { return map[m]; });
+  }
+
+  // Page Rendering - simple template substitution
+  renderPage(templateName, title, content, options = {}) {
+    const template = this.getTemplate(templateName);
+    if (!template) {
+      throw new Error(`Template '${templateName}' not found`);
+    }
+    
+    // Default options
+    const renderOptions = {
+      version: '4.0.1',
+      downloadDate: 'Unknown',
+      totalResources: 0,
+      totalPackages: 0,
+      processingTime: 0,
+      ...options
+    };
+    
+    // Perform template replacements
+    let html = template
+      .replace(/\[%title%\]/g, this.escapeHtml(title))
+      .replace(/\[%content%\]/g, content) // Content is assumed to be already-safe HTML
+      .replace(/\[%ver%\]/g, this.escapeHtml(renderOptions.version))
+      .replace(/\[%download-date%\]/g, this.escapeHtml(renderOptions.downloadDate))
+      .replace(/\[%total-resources%\]/g, this.escapeHtml(renderOptions.totalResources.toLocaleString()))
+      .replace(/\[%total-packages%\]/g, this.escapeHtml(renderOptions.totalPackages.toLocaleString()))
+      .replace(/\[%endpoint-path%\]/g, this.escapeHtml(renderOptions.endpointpath))
+      .replace(/\[%fhir-version%\]/g, this.escapeHtml(renderOptions.fhirversion))
+      .replace(/\[%ms%\]/g, this.escapeHtml(renderOptions.processingTime.toString()));
+    
+    // Handle any custom template variables
+    if (options.templateVars) {
+      for (const [key, value] of Object.entries(options.templateVars)) {
+        const placeholder = `[%${key}%]`;
+        const escapedValue = typeof value === 'string' ? this.escapeHtml(value) : String(value);
+        html = html.replace(new RegExp(placeholder.replace(/[.*+?^${}()|[\]\\]/g, '\\$&'), 'g'), escapedValue);
+      }
+    }
+    
+    return html;
+  }
+
+  // Express Response Helper
+  sendHtmlResponse(res, templateName, title, content, options = {}) {
+    try {
+      const html = this.renderPage(templateName, title, content, options);
+      res.setHeader('Content-Type', 'text/html');
+      res.send(html);
+    } catch (error) {
+      this.log.error('[HtmlServer] Error rendering page:', error);
+      res.status(500).send(`<h1>Error</h1><p>Failed to render page: ${this.escapeHtml(error.message)}</p>`);
+    }
+  }
+
+  sendErrorResponse(res, templateName, error, statusCode = 500) {
+    const errorContent = `
+      <div class="alert alert-danger">
+        <h4>Error</h4>
+        <p>${this.escapeHtml(error.message || error)}</p>
+      </div>
+    `;
+    
+    try {
+      const html = this.renderPage(templateName, 'Error', errorContent);
+      res.status(statusCode).setHeader('Content-Type', 'text/html');
+      res.send(html);
+    } catch (renderError) {
+      this.log.error('[HtmlServer] Error rendering error page:', renderError);
+      res.status(statusCode).send(`<h1>Error</h1><p>Failed to render error page: ${this.escapeHtml(renderError.message)}</p>`);
+    }
+  }
+
+  // Date Formatting Utility
+  formatDate(dateString) {
+    if (!dateString) return '';
+    try {
+      const date = new Date(dateString);
+      const year = date.getFullYear();
+      const month = String(date.getMonth() + 1).padStart(2, '0');
+      return `${year}-${month}`;
+    } catch (error) {
+      return dateString; // Return original if parsing fails
+    }
+  }
+
+  // Initialize templates from directory
+  loadTemplatesFromDirectory(templatesDir) {
+    if (!fs.existsSync(templatesDir)) {
+      this.log.warn(`Templates directory not found: ${templatesDir}`);
+      return;
+    }
+
+    const templateFiles = fs.readdirSync(templatesDir).filter(file => file.endsWith('.html'));
+    
+    templateFiles.forEach(file => {
+      const templateName = path.basename(file, '.html');
+      const templatePath = path.join(templatesDir, file);
+      this.loadTemplate(templateName, templatePath);
+    });
+  }
+}
+
+// Export singleton instance
+module.exports = new HtmlServer();