fhir-persistence 0.1.0

package/CHANGELOG.md

@@ -0,0 +1,77 @@
+# Changelog
+
+All notable changes to this project will be documented in this file.
+
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
+and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+## [0.1.0] - 2025-03-13
+
+### Added
+
+#### Storage Layer
+- `StorageAdapter` interface — unified async database abstraction
+- `SQLiteAdapter` — sql.js (WASM) implementation for cross-platform use
+- `BetterSqlite3Adapter` — native better-sqlite3 implementation for production Node.js
+- `SQLiteDialect` / PostgreSQL dialect support for DDL generation
+
+#### CRUD & Persistence
+- `FhirStore` — basic CRUD with soft delete and version tracking
+- `FhirPersistence` — end-to-end facade with automatic search indexing
+- `ConditionalService` — conditionalCreate / conditionalUpdate / conditionalDelete
+- `BundleProcessorV2` — FHIR transaction and batch bundle processing
+- Resource versioning with `versionId` auto-increment
+- History tracking via dedicated `_History` tables
+
+#### Search
+- `SearchParameterRegistry` — index FHIR SearchParameter bundles
+- `parseSearchRequest` — parse FHIR search URL query parameters
+- `buildSearchSQLv2` — generate SQL with `?` placeholders (SQLite + PG compatible)
+- `buildWhereClauseV2` — chain search support (`subject:Patient.birthdate=...`)
+- `SearchPlanner` — filter reordering, chain depth validation, two-phase recommendation
+- `buildTwoPhaseSearchSQLv2` — id-first query for large table performance
+- `SearchExecutor` — `_include`, `_revinclude`, recursive `_include:iterate` (max depth 3)
+- `SearchBundleBuilder` — FHIR Bundle response construction with pagination
+
+#### Indexing
+- `IndexingPipeline` — automatic search column population on CRUD
+- `RuntimeProvider` bridge — FHIRPath-driven extraction via `fhir-runtime`
+- `buildSearchColumns` — fallback property-path extraction
+- `extractReferencesV2` — reference extraction and indexing
+- `LookupTableWriter` — HumanName, Address, ContactPoint, Identifier lookup tables
+- Column strategy, token-column strategy, lookup-table strategy
+
+#### Schema & Migration
+- `StructureDefinitionRegistry` — register and resolve FHIR StructureDefinitions
+- `buildResourceTableSet` — StructureDefinition + SearchParameter → table schema
+- DDL generators for Main, History, References tables + indexes
+- `compareSchemas` — schema diff between old and new table sets
+- `generateMigration` — diff → DDL migration statements
+- `MigrationRunnerV2` — execute migrations with `StorageAdapter`
+- `IGPersistenceManager` — three-way branch (new / upgrade / consistent)
+- `PackageRegistryRepo` — multi-version package tracking with checksum
+- `ReindexScheduler` — schedule reindex jobs for SP expression changes
+
+#### Terminology
+- `TerminologyCodeRepo` — code system concept storage
+- `ValueSetRepo` — value set expansion storage
+
+#### Platform IG
+- Built-in platform resource types: User, Bot, Project, Agent, ClientApplication
+- `initializePlatformIG` — auto-register platform search parameters
+
+#### Provider Bridges (for fhir-engine)
+- `FhirDefinitionBridge` — wraps `fhir-definition` `DefinitionRegistry` → `DefinitionProvider`
+- `FhirRuntimeProvider` — wraps `fhir-runtime` `FhirRuntimeInstance` → `RuntimeProvider`
+- `FhirSystem` — end-to-end startup orchestrator
+
+#### Production
+- `ResourceCacheV2` — in-memory resource cache with TTL
+- `SearchLogger` — search query logging and diagnostics
+- `reindexResourceTypeV2` / `reindexAllV2` — CLI reindex utilities
+
+### Dependencies
+- `fhir-definition` ^0.5.0
+- `fhir-runtime` ^0.8.1
+- `better-sqlite3` ^12.6.2
+- `sql.js` ^1.14.1

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 jason fang
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -0,0 +1,225 @@
+# fhir-persistence
+
+Embedded FHIR R4 persistence layer — CRUD, search, indexing, and schema migration over SQLite and PostgreSQL.
+
+[![npm version](https://img.shields.io/npm/v/fhir-persistence)](https://www.npmjs.com/package/fhir-persistence)
+[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](./LICENSE)
+
+## Features
+
+- **StorageAdapter abstraction** — unified async interface for SQLite and PostgreSQL
+- **Two SQLite adapters** — `BetterSqlite3Adapter` (native, production) + `SQLiteAdapter` (sql.js WASM, cross-platform)
+- **3-table-per-resource pattern** — Main + History + References tables per FHIR resource type
+- **Automatic search indexing** — column, token-column, and lookup-table strategies
+- **FHIRPath-driven extraction** — optional `RuntimeProvider` bridge for `fhir-runtime` powered indexing
+- **Chain search** — `subject:Patient.birthdate=1990-01-15`
+- **\_include / \_revinclude** — with recursive `_include:iterate` support (max depth 3)
+- **Search planner** — filter reordering, chain depth validation, two-phase SQL recommendation
+- **Two-phase SQL** — id-first query for large table performance
+- **IG-driven schema** — StructureDefinition + SearchParameter → DDL (SQLite + PostgreSQL dialects)
+- **Migration engine** — SchemaDiff → MigrationGenerator → MigrationRunnerV2
+- **Conditional operations** — conditionalCreate / conditionalUpdate / conditionalDelete
+- **Bundle processing** — transaction and batch bundle support
+- **Terminology** — TerminologyCodeRepo + ValueSetRepo
+- **FhirSystem orchestrator** — end-to-end startup flow for `fhir-engine` integration
+- **Provider bridges** — `FhirDefinitionBridge` + `FhirRuntimeProvider` for `fhir-definition` / `fhir-runtime`
+
+## Install
+
+```bash
+npm install fhir-persistence
+```
+
+**Peer dependencies:**
+
+```bash
+npm install fhir-definition fhir-runtime
+```
+
+## Quick Start
+
+### Standalone (low-level)
+
+```typescript
+import {
+  SQLiteAdapter,
+  FhirPersistence,
+  SearchParameterRegistry,
+} from "fhir-persistence";
+
+// 1. Create storage adapter
+const adapter = new SQLiteAdapter(":memory:");
+
+// 2. Create search parameter registry
+const spRegistry = new SearchParameterRegistry();
+spRegistry.indexBundle(searchParameterBundle);
+
+// 3. Create persistence facade
+const persistence = new FhirPersistence({
+  adapter,
+  searchParameterRegistry: spRegistry,
+});
+
+// 4. CRUD with automatic indexing
+const patient = await persistence.createResource("Patient", {
+  resourceType: "Patient",
+  name: [{ family: "Smith", given: ["John"] }],
+  birthDate: "1990-01-15",
+  active: true,
+});
+
+const result = await persistence.searchResources({
+  resourceType: "Patient",
+  queryParams: {
+    birthdate: "ge1990-01-01",
+    active: "true",
+    _sort: "-birthdate",
+  },
+});
+```
+
+### With FhirSystem (recommended for fhir-engine)
+
+```typescript
+import { BetterSqlite3Adapter, FhirSystem, FhirDefinitionBridge } from 'fhir-persistence';
+import { loadDefinitionPackages } from 'fhir-definition';
+import { createRuntime } from 'fhir-runtime';
+
+// 1. Load FHIR definitions
+const { registry } = loadDefinitionPackages('./fhir-packages');
+
+// 2. Create runtime with definitions
+const runtime = await createRuntime({ definitions: registry });
+
+// 3. Create adapter + bridges
+const adapter = new BetterSqlite3Adapter({ path: './fhir.db' });
+const definitionBridge = new FhirDefinitionBridge(registry);
+
+// 4. Bootstrap via FhirSystem
+const system = new FhirSystem(adapter, { dialect: 'sqlite' });
+const { persistence, sdRegistry, spRegistry, igResult } =
+  await system.initialize(definitionBridge);
+
+// 5. Use persistence
+const patient = await persistence.createResource('Patient', { resourceType: 'Patient', ... });
+```
+
+## Architecture
+
+```
+StorageAdapter (SQLite / better-sqlite3 / PostgreSQL)
+  └── FhirPersistence (end-to-end facade)
+        ├── FhirStore (basic CRUD + soft delete + versioning)
+        ├── IndexingPipeline
+        │     ├── RuntimeProvider (FHIRPath extraction, optional)
+        │     ├── buildSearchColumns (fallback row indexer)
+        │     ├── extractReferencesV2 (reference indexer)
+        │     └── LookupTableWriter (HumanName/Address/ContactPoint/Identifier)
+        ├── ConditionalService (conditional CRUD)
+        ├── BundleProcessorV2 (transaction / batch)
+        ├── SearchParameterRegistry
+        └── Search Engine
+              ├── WhereBuilder v2 (chain search, ? placeholders)
+              ├── SearchPlanner (filter reorder, two-phase recommendation)
+              ├── SearchSQLBuilder v2 (single-phase + two-phase)
+              ├── SearchExecutor (_include / _revinclude / _include:iterate)
+              └── SearchBundleBuilder
+```
+
+## Storage Adapters
+
+| Adapter                | Backend                 | Use Case                          |
+| ---------------------- | ----------------------- | --------------------------------- |
+| `BetterSqlite3Adapter` | better-sqlite3 (native) | Production Node.js, CLI, Electron |
+| `SQLiteAdapter`        | sql.js (WASM)           | Browser, cross-platform, testing  |
+| `PostgresAdapter`      | pg                      | Production server                 |
+
+## Search
+
+```typescript
+import {
+  parseSearchRequest,
+  buildSearchSQLv2,
+  planSearch,
+} from "fhir-persistence";
+
+// Parse search URL
+const request = parseSearchRequest(
+  "Patient",
+  {
+    birthdate: "ge1990-01-01",
+    active: "true",
+    _sort: "-birthdate",
+    _count: "50",
+  },
+  registry,
+);
+
+// Single-phase SQL
+const sql = buildSearchSQLv2(request, registry);
+
+// Two-phase SQL for large tables
+const plan = planSearch(request, registry, { estimatedRowCount: 100_000 });
+if (plan.useTwoPhase) {
+  const { phase1, phase2Template } = buildTwoPhaseSearchSQLv2(
+    plan.request,
+    registry,
+  );
+}
+
+// Chain search
+const chainRequest = parseSearchRequest(
+  "Observation",
+  {
+    "subject:Patient.birthdate": "ge1990-01-01",
+  },
+  registry,
+);
+```
+
+## Schema Migration
+
+The IG persistence manager automatically handles schema evolution:
+
+```typescript
+import { IGPersistenceManager } from "fhir-persistence";
+
+const igManager = new IGPersistenceManager(adapter, "sqlite");
+const result = await igManager.initialize({
+  name: "hl7.fhir.r4.core",
+  version: "4.0.1",
+  checksum: contentChecksum,
+  tableSets: generatedTableSets,
+});
+// result.action: 'new' | 'upgrade' | 'consistent'
+```
+
+## Integration with fhir-engine
+
+`fhir-persistence` is designed to be bootstrapped by `fhir-engine`:
+
+```typescript
+import { createFhirEngine } from "fhir-engine";
+
+const engine = await createFhirEngine({
+  database: { type: "sqlite", url: "./fhir.db" },
+  packages: { path: "./fhir-packages" },
+});
+
+// engine.persistence — FhirPersistence instance
+// engine.definitions — DefinitionRegistry
+// engine.runtime — FhirRuntimeInstance
+```
+
+## Dependencies
+
+| Package           | Role                                                                   |
+| ----------------- | ---------------------------------------------------------------------- |
+| `fhir-definition` | StructureDefinition, SearchParameter, ValueSet, CodeSystem definitions |
+| `fhir-runtime`    | FHIRPath evaluation, validation, search value extraction               |
+| `better-sqlite3`  | Native SQLite bindings (production)                                    |
+| `sql.js`          | WebAssembly SQLite (cross-platform)                                    |
+
+## License
+
+[MIT](./LICENSE)