@jarenjs/db 0.85.0 → 0.87.0

package/ARCHITECTURE.md

@@ -1186,7 +1186,9 @@ both hosts, including transaction failures.
 
 `dialects/sqlite-relational.js` owns the explicit SQLite expression and statement
 compiler; `dialects/sqlite-schema.js` reuses it for ordered tables, indexes and
-triggers. `table-migration.js` owns reviewed source/target guards, native copying
+triggers, sharing its column/reference renderer with native ADD COLUMN.
+`table-migration.js` owns reviewed source/settings guards for additive/object
+changes and source/target guards for native copying
 and preservation checks, with catalog/pragma spellings in the SQLite dialect.
 The supplied driver owns transactions and cursors. These programs are explicitly
 SQLite-semantic and never enter the JSON residual evaluator. See
@@ -1196,3 +1198,12 @@ SQLite-semantic and never enter the JSON residual evaluator. See
 does not import runtime owners. `/query`, `/model` and `/entity` expose those
 mechanisms without the root store import. `compileEntityModel` performs one model
 normalization for both entities and mapping; openStore reuses that result.
+`mutation.js` keeps bounded statement reuse keyed by emitted SQL; bindings,
+projections and limits are call-local, so cached plans do not retain old payloads.
+
+`drivers/worker-client.js` and `drivers/sqlite-endpoint.js` own the shared bounded
+RPC contract. Thread and process entries supply their transports. The process
+driver retains owner credits until OS exit, fences lost generations, and combines
+observed native transaction state with pending statement metadata to distinguish
+rollback from an unknown commit. Supervision never serializes callbacks or replays
+transactions; Store ownership remains in the existing driver/Store gates.

package/README.md

@@ -888,7 +888,8 @@ oracle run on both. What differs is declared rather than discovered:
 | `ALTER TABLE` | additive only | full, so a rebuild is never needed |
 
 Neither backend has a **statement timeout** (`capabilities.statementTimeout`
-is `false` on both) or **row estimates**. There is no replication.
+is `false` on both) or **row estimates**. Portable replication uses SQLite
+change capture; PostgreSQL capture and replication remain unqualified.
 Cross-process concurrency on SQLite is its own story — WAL plus a busy
 timeout, both set and visible on `store.capabilities`; on PostgreSQL it
 is the server's, and a serialization failure or a deadlock arrives as
@@ -1211,16 +1212,18 @@ same report rows — with the one difference the PostgreSQL mapping
 states: `numeric` carries both JSON number types, so an `integer`
 member reads back as `number`.
 
-## Sync-readiness — what exists and what does not
+## Replication and external-write boundaries
 
-The change stream is an ordered log of RFC 6902 patches with a
-monotonic sequence, and SQLite's own changeset/conflict primitives are
-available — which is what a replication protocol would be *built
-from*. **No replication is shipped.** There is no conflict resolution,
-no site identity, no causal ordering across writers, and no capture of
-writes made by another connection (the coarse `dataVersion()` signal
-is the honest mitigation, not a pretend fine-grained one). Building
-replication on these primitives is a roadmap item, not a hint.
+The portable replication engine supplies replica identities, causal frontiers,
+bounded logical envelopes, durable replay receipts, conflict evidence and snapshot
+resets. Data and acknowledgements commit together; hosts supply transport and any
+conflict resolver. The default preserves both contenders and rejects the write.
+See [REPLICATION-FORMAT](docs/REPLICATION-FORMAT.md) for the current contract.
+
+Capture observes participating Store writes on supported SQLite hosts. Arbitrary
+writes from another connection are not fine-grained capture events; `dataVersion()`
+provides coarse invalidation. Adopted triggers, unsupported layouts and PostgreSQL
+retain the explicit capture and replication qualification limits.
 
 ## Operating a store — configuration, maintenance, backup, cancellation, the queue
 
@@ -1311,10 +1314,11 @@ its side-effect-free status read are in
 
 ## What this is not — every non-claim in one place
 
-- **SQLite only.** One backend (3.45+); the dialect seam is tested
-  against a double but no second dialect ships. No server.
-- **No replication or sync engine** (see above). No cross-connection
-  change capture — another connection's writes are invisible locally.
+- **SQLite and PostgreSQL have different capabilities.** Both backends ship;
+  PostgreSQL does not provide the SQLite capture, replication, job queue or pragma
+  maintenance capabilities. See the dialect table and normative host contracts.
+- **Replication transport is supplied by the host.** Portable logical replication
+  ships; arbitrary external writes are not captured as local row events.
 - **No statement timeout** on SQLite (the drivers expose no interrupt;
   the capability slot is honestly `false`), no row estimates.
 - **Not safe for mutually hostile tenants** without the profile's
@@ -1359,6 +1363,7 @@ Every subpath a consumer can import, derived from the manifest by
 <!--fact:exports.db-->
 | Import | Kind | Declarations |
 |---|---|---|
+| `@jarenjs/db/node-process` | JavaScript | declared |
 | `@jarenjs/db/search` | JavaScript | declared |
 | `@jarenjs/db` | JavaScript | declared |
 | `@jarenjs/db/node` | JavaScript | declared |
@@ -1436,9 +1441,19 @@ Native column reads and bounded mutation documents are specified in [NATIVE-PLAN
 
 `@jarenjs/db/search` composes the resident ranker with bounded authoritative entity reads and optional atomic snapshot storage. See [persisted search](docs/SEARCH.md).
 
-Column-first table definitions, guarded same-connection rebuilds, exact matched writes,
+Column-first table definitions, native additive/object DDL, guarded same-connection rebuilds, exact matched writes,
 partial conflicts, raw-text JSON queries and byte-valued operations use
 [`@jarenjs/db/relational`](docs/SQLITE-RELATIONAL.md). Both host entries export
 `snapshotDatabase(connection, newPath)` for a disk-backed committed-WAL copy.
 Existing-connection consumers can import `/query`, `/model` and `/entity`;
 `compileEntityModel` shares one normalization between queries and mapping.
+`sql.call('json_type', ...)` preserves native missing/null/type distinctions.
+`planSchemaChange`/`applySchemaChange` guard a reviewed ADD COLUMN, DROP INDEX,
+RENAME TABLE or DROP TABLE against source drift. Identity-changing policies stay
+explicit; ordinary rebuilds retain key and row preservation. Mutation statements
+are reused by SQL without retaining previous payload-bearing documents.
+
+`@jarenjs/db/node-process` adds supervised native execution with finite owner
+admission, caller deadlines, generation fencing and separate process-exit and
+transaction-fate observations. See [execution hosts](docs/HOSTS.md#supervised-node-processes)
+for Store composition, receipt reconciliation and operating-system limits.

package/docs/HOSTS.md

@@ -77,6 +77,132 @@ run on the caller. Synchronous Store methods and live queries are unavailable on
 these asynchronous connections. Bun can import both subpaths; opening a Node
 SQLite worker there reports the named unavailable-binding failure (`JD0003`).
 
+## Supervised Node processes
+
+`nodeProcessDriver` from `@jarenjs/db/node-process` implements the same asynchronous
+Driver and Connection contracts using one Node child process per connection. It
+shares the worker protocol, cursor credits, transaction scopes and error handling
+with the thread driver. SQLite and its native calls execute in the child; callbacks
+and decoded query residuals still execute in the parent. No function is serialized.
+Node.js 24 or newer is required. Bun can import the entry but `open` refuses with
+`JD0003`. The native-call and process-lifecycle fixtures qualify Node 24.20.0 on
+Linux; other supported operating systems require their own timing qualification.
+
+```js
+import { nodeProcessDriver } from '@jarenjs/db/node-process';
+
+const driver = nodeProcessDriver({ maxOwners: 2, timeoutMs: 250 });
+const owner = await driver.open('application.sqlite', { timeout: 50, queueTimeout: 100 });
+try {
+  const result = await owner.supervise(async (connection) => {
+    return connection.transaction(async (scope) => {
+      const statement = await scope.prepare('SELECT value FROM settings WHERE key = ?');
+      return statement.get(['theme']);
+    });
+  }, { signal: requestSignal, timeoutMs: 250 });
+  useResult(result);
+} catch (error) {
+  if (error.code === 'JD2097' || error.code === 'JD2090') {
+    const observation = owner.settlement();
+    // This snapshot can still say quarantined / safeToReplace:false.
+    recordOwnerObservation(observation);
+    const exited = await owner.settled();
+    // Before repeating an uncertain write, inspect its durable receipt.
+    reconcileBeforeRetry(exited);
+  } else throw error;
+} finally {
+  // Preserve the original operation outcome if closing a fenced generation fails.
+  await owner.close().catch(recordCleanupFailure);
+}
+```
+
+`supervise(body,{signal,timeoutMs})` admits one supervised callback per connection.
+An overlapping callback refuses with `JD2091`; an already-aborted call runs no
+callback and leaves the owner healthy. A deadline or abort rejects with `JD2097`,
+fences all pending calls and signals process termination. Later operations on the
+old generation fail with `JD2090`. A normal callback failure propagates unchanged
+and does not terminate a healthy owner. `cancel(reason)` explicitly fences the
+whole connection and returns its cancellation error. It affects every call sharing
+that owner, so independent request lifetimes should use separate owners.
+
+This is response cancellation and owner termination, not a SQLite statement
+interrupt. `capabilities.process` and `ownerTermination` are true, while
+`capabilities.cancellation.midStatement` remains false. The supervision timer cannot
+preempt synchronous parent JavaScript or bound operating-system scheduling. Keep
+callbacks/residuals bounded and await all admitted work. Native fixtures use a
+short deadline and independently enforce a one-second caller-response ceiling;
+that tested ceiling is not a universal scheduling guarantee.
+
+`settlement()` reports the generation, PID, health, transaction fate and
+`safeToReplace`. `settled()` resolves only after the child's OS exit event. An
+owner awaiting termination is quarantined and still consumes its credit. If the
+OS cannot settle a process promptly, the response can finish while the owner
+remains quarantined; neither capacity nor writer ownership is recycled. `restart()`
+waits for confirmed exit and returns a new generation; old Store handles remain
+invalid. No statement, transaction or callback is automatically replayed.
+
+The transaction observation is conservative:
+
+- A live, acknowledged native transaction is `active`.
+- An acknowledged single-statement completion is `committed`, or `rolled-back`
+  after rollback. A multi-statement program with no open transaction remains
+  `unknown`: an acknowledgement alone cannot distinguish its internal boundaries.
+- Confirmed process death rolls back an observed open transaction only when no
+  pending operation could have committed it.
+- A lost commit reply, multi-statement program or uncertain autocommit mutation
+  is `unknown`. Native transaction-state support is observed, never inferred from
+  a close promise. Reopen, verify integrity and reconcile a durable receipt before
+  retrying a write with an unknown outcome.
+
+Autocommit mutation cursors remain `unknown` until their final native frame is
+acknowledged. Creating an iterator or receiving an intermediate `RETURNING` row
+does not acknowledge completion; early cursor return still requires reconciliation.
+
+Driver `metrics()` reports capacity, owners, healthy owners and quarantine. A
+file path already owned by the same driver cannot be opened again until exit;
+this reservation uses its resolved path spelling, not filesystem inode identity.
+Different driver instances, aliases and external processes remain the host's
+coordination responsibility. Use one supervisor per ownership domain and retain
+finite SQLite busy and transaction-queue timeouts. The host's service manager must
+also own its process group: an uncatchable parent-process death is not a promise
+that JavaScript can reap every descendant. Child crashes and restart are covered
+by the file/receipt fixtures; power loss and service-manager policy are separate.
+
+The ordinary Store composition uses `openStore(model,{driver,path})`. To supervise
+Store calls, retain the connection in a small driver wrapper's `open` method, then
+call `owner.supervise(() => store.transaction(body), options)`. This uses the same
+Store and transaction APIs. After owner loss, close the invalid Store, await owner
+exit and explicitly reopen/reconcile. A failed-generation `close()` may reject;
+its rejection never substitutes for the separate exit observation.
+
+All options below are positive safe integers. Row/byte/statement/cursor limits
+retain the thread worker meanings. `maxOwners` includes startup and quarantine;
+`timeoutMs` is the default supervised response deadline. `maxRequestBytes` bounds
+each outgoing request before IPC; parameters must be SQLite scalars or byte arrays.
+The process close deadline
+and startup deadline can reject while termination is still pending. The reserved
+owner credit remains until exit. Direct calls without `supervise`
+retain the ordinary row, queue and step cancellation boundaries.
+
+<!--fact:db.execution-options-->
+
+| Option | Thread worker | Process owner |
+|---|---:|---:|
+| `windowRows` | 64 | 64 |
+| `windowBytes` | 1048576 | 1048576 |
+| `maxPending` | 64 | 64 |
+| `maxStatements` | 1024 | 1024 |
+| `maxCursors` | 64 | 64 |
+| `allMaxRows` | 100000 | 100000 |
+| `allMaxBytes` | 16777216 | 16777216 |
+| `closeTimeoutMs` | 5000 | 1000 |
+| `startupTimeoutMs` | 10000 | 10000 |
+| `maxOwners` | — | 4 |
+| `timeoutMs` | — | 250 |
+| `maxRequestBytes` | — | 1048576 |
+
+<!--/fact-->
+
 ## WAL pool policy
 
 A file pool opens exactly one writer and `readers` read-only workers, verifies WAL,
@@ -283,9 +409,10 @@ sync twin. Physical adoption on PostgreSQL refuses pending a separate mapping
 and codec qualification. Unknown application-trigger effects do not qualify
 capture or replication; those combinations refuse before an adoption claim.
 
-Node backups use the built-in online snapshot. Bun uses `Database.serialize()`
-under the store gate, writes and flushes a sibling temporary, then uses the shared
-atomic publisher. Both include committed WAL. Bun's snapshot holds the whole
-image in memory and cancellation takes effect between phases. Process-kill tests
+Node backups use the built-in online snapshot. Bun uses disk-backed `VACUUM INTO`
+under the store gate, flushes a sibling temporary, then uses the shared atomic
+publisher. Both include committed WAL. Bun allocates no whole-database JavaScript
+image; native SQLite caches and temporary storage govern working memory, and
+cancellation takes effect between phases. Process-kill tests
 cover rebuild copy, table drop, commit and backup publication on both hosts;
 these tests do not establish power-loss durability or native executable packaging.

package/docs/MODEL-FORMAT.md

@@ -1101,6 +1101,7 @@ error.
 | `JD2094` | invalid or uncommitted durable snapshot; reopen the last committed version |
 | `JD2095` | trusted SQL or synchronous callback authority refused |
 | `JD2096` | persistence invariant rejected the mutation; constraint class |
+| `JD2097` | supervised native operation cancelled or past its response deadline; owner exit and transaction fate are reported separately |
 | `JD0060` | a replication envelope or snapshot is invalid |
 | `JD2100` | a replica sequence or causal dependency has a gap |
 | `JD2101` | an envelope identity names different content or an unknown local origin |
@@ -2424,3 +2425,8 @@ physical entity retains adoption-only behavior. Public plans create complete
 ordered declarations; an incomplete generated/default definition or view remains
 adoption metadata. SQLite expressions preserve a separate, explicit semantic
 contract for raw text, bytes, nulls, collations and floating totals.
+`planSchemaChange`/`applySchemaChange` expose explicit native ADD COLUMN, DROP
+INDEX, RENAME TABLE and DROP TABLE with source/settings guards. They do not infer
+identity remapping or row deletion; that policy stays in an explicitly reviewed
+transaction. Entity mutation statement reuse is keyed by emitted SQL, while
+bindings, output projections and resource limits belong to each execution.

package/docs/NATIVE-PLANS.md

@@ -65,9 +65,11 @@ row limits bound fetched results; byte limits bound each decoded result item.
 
 ## Mutations
 
-The asynchronous entity set exposes `mutate(document)`. It compiles and caches a
-closed document into one parameterized SQLite data statement, inside the same
-guarded transaction used by the entity writer. It supports adopted writable
+The asynchronous entity set exposes `mutate(document)`. It compiles each closed
+document into one parameterized SQLite data statement and reuses statements by
+their complete SQL in a bounded cache. Bindings, projections and output limits
+remain local to each call; earlier payload-bearing documents are not retained.
+Execution uses the same guarded transaction as the entity writer. It supports adopted writable
 column layouts. Hybrid entities, PostgreSQL physical layouts, arbitrary SQL,
 store-enforced before/after invariants and unsupported expression shapes refuse
 with `JD0038`. Database constraints and invariant triggers retain enforcement.

package/docs/SQLITE-RELATIONAL.md

@@ -49,9 +49,16 @@ Selections support table and subquery sources, inner/left/cross joins, correlate
 `sql.scalar`/`sql.exists`, CASE, IN/NOT IN, DISTINCT, grouped and distinct
 aggregates, HAVING, UNION/UNION ALL and ordered windows. The declaration file
 lists the closed operators and functions, including trim/coalesce, LIKE, JSON
-extraction, casts and SQLite date functions. Neither arbitrary function names
+extraction/type inspection, casts and SQLite date functions. Neither arbitrary function names
 nor a raw-expression escape hatch is accepted.
 
+`sql.call('json_type', [document, path])` distinguishes a missing path (SQL NULL)
+from JSON null (text `'null'`) and reports SQLite's native scalar/container type
+names. Omitting the path inspects the whole document. SQL-null input stays null;
+malformed JSON raises SQLite's error. It composes with CASE and synchronous
+cursors without decoding or rewriting the original column. See
+[SQLite JSON type inspection](https://www.sqlite.org/json1.html#the_json_type_function).
+
 ## Exact writes and bytes
 
 ```js
@@ -114,7 +121,9 @@ applyTableMigration(connection, migration);
 Columns retain declaration order and exact INTEGER/REAL/TEXT/BLOB/NUMERIC/ANY
 types. Definitions support ordered primary keys, rowid or AUTOINCREMENT identity,
 nullability, database defaults, generated columns, STRICT/WITHOUT ROWID, named
-UNIQUE/CHECK/foreign-key constraints and delete/update actions. Index terms
+UNIQUE/CHECK/foreign-key constraints and delete/update actions. A column can also
+declare `references: { table, columns: [name], onDelete, onUpdate, deferred }`.
+Index terms
 support expressions, direction and collation, with an optional partial predicate.
 Triggers support BEFORE/AFTER, INSERT/UPDATE/DELETE, UPDATE OF, OLD/NEW conditions,
 mutation steps and RAISE. Schema expressions use the same structural emitter,
@@ -132,8 +141,9 @@ physical tables use the live-schema `planTableMigration` API.
 ## Guarded upgrades
 
 Planning inspects the existing schema without modifying it. A differing existing
-table requires `allowRebuild: true`; additions currently use the same guarded
-rebuild. Every removed column needs `dropColumns`; removing an existing explicit
+table requires `allowRebuild: true` when using `planTableMigration`; use the narrow
+schema operations below to append a column without rebuilding. Every removed
+column needs `dropColumns`; removing an existing explicit
 index/trigger requires `dropObjects`. Unmentioned indexes and triggers survive.
 An optional `copy` maps writable non-key target columns to structural expressions;
 new columns otherwise use their defaults. The plan retains the source schema and
@@ -157,6 +167,90 @@ refuses before DDL. Node/Bun regressions cover populated history and references,
 failed copies, repeat reopening, nested rollback, and process death after DROP
 with WAL recovery. These tests do not establish power-loss durability.
 
+## Additive and object operations
+
+`planSchemaChange(connection, operation)` returns a reviewable
+`{ version, operation, sql, source, settings, checksum }`. It reads the main schema
+and relevant connection settings without executing DDL. `applySchemaChange`
+checks that source and settings again under an IMMEDIATE transaction before
+executing the single statement. It returns `{ changed }`, the number of schema
+operations that changed the catalog, rather than the number of affected rows.
+Both methods require the same available synchronous SQLite ownership as rebuilds.
+
+```js
+import { planSchemaChange, applySchemaChange, sql } from '@jarenjs/db/relational';
+const addition = planSchemaChange(connection, {
+  op: 'addColumn', table: 'entries', column: {
+    name: 'revision', type: 'INTEGER', nullable: false, default: 1,
+    check: sql.binary('>=', sql.column('revision'), 1),
+  },
+});
+// Review addition.sql and addition.source before execution.
+applySchemaChange(connection, addition);
+applySchemaChange(connection, planSchemaChange(connection, {
+  op: 'dropIndex', name: 'obsolete_index', ifExists: true,
+}));
+```
+
+The closed operations are `addColumn` (`table`, `column`), `dropIndex` (`name`,
+optional `ifExists`), `renameTable` (`table`, `to`) and `dropTable` (`table`,
+optional `ifExists`). Identifiers address **main** explicitly; a temporary table
+with the same spelling cannot redirect an operation. A dot inside a name is a
+literal character. Attached database operations are not part of this surface.
+The column definition and expression renderer are shared with `planTable`.
+
+ADD COLUMN appends a declaration and preserves existing rowids, values, storage
+classes, column order, indexes and triggers. It does not derive a complete table
+definition, normalize unknown constraints or copy the table. Literal defaults
+are supported; identity columns, STORED generated columns and expression defaults
+refuse. Ordinary NOT NULL additions require a non-null default. REFERENCES
+additions require a NULL default (or no default), and a single referenced column.
+SQLite checks existing rows for a new CHECK or generated NOT NULL constraint;
+those checks can scan the table even though the operation does not copy it.
+See [SQLite ADD COLUMN restrictions](https://www.sqlite.org/lang_altertable.html#alter_table_add_column).
+
+Drop operations fail on absence unless `ifExists: true` is supplied. A missing
+object then returns `changed: 0`. DROP INDEX affects the named index, not its
+table or triggers. DROP TABLE removes the table and its owned indexes/triggers;
+SQLite's active foreign-key actions still apply. RENAME follows SQLite's current
+dependency rewriting behavior and the reviewed `legacy_alter_table` setting.
+See [DROP TABLE](https://www.sqlite.org/lang_droptable.html) and
+[RENAME TABLE](https://www.sqlite.org/lang_altertable.html#alter_table_rename).
+
+These are **single-source plans**, not durable migration receipts. Replan after
+any schema change. Reapplying a successful ADD/RENAME plan refuses as stale;
+it does not infer completion from a same-named object. An ordered migration must
+check its trusted schema/version receipt before planning the next operation and
+record completion in the same transaction. Unknown members or unsupported column
+declarations refuse with `JD0005`; modified/stale plans refuse with `JD0021`.
+Native object, data-constraint and dependency errors are left to SQLite. A
+checksum detects accidental plan edits; it does not authorize untrusted plans.
+
+## Explicit identity-changing upgrades
+
+The ordinary rebuild planner continues to refuse key reassignment or row loss.
+A reviewed upgrade can use the primitive operations under
+`withForeignKeysSuspended`, with its own explicit data policy:
+
+1. Check the durable migration receipt or the exact supported legacy schema.
+2. Create a distinct replacement table with `planTable`.
+3. Use relational insert-select with a scoped correlated selection, deterministic
+   identity choice and explicit exclusion predicate. Assert selected, inserted
+   and excluded counts, and any business-specific preservation requirements.
+4. Plan/apply `dropTable` for the original and then `renameTable` for the
+   replacement. Create the required indexes/triggers and validate dependents.
+5. Record completion inside the same transaction so another opening does no work.
+
+Create the replacement before dropping the original; renaming the original first
+can redirect dependent references. Plan each primitive inside the FK scope, after
+the preceding schema operation. The helper checks references before commit and
+restores connection settings. The caller must review incoming key references,
+views, triggers, immutable neighbors and each row disposition. Conflicts fail and
+roll back; there is no inferred permission to discard rows. The installed native
+[qualification fixture](../../../test/db/fixtures/schema-upgrade.mjs) demonstrates
+scoped minimum-ID selection, explicit exclusions, rollback, repeated reopening
+and recovery after actual process death at DROP.
+
 ## Read-only inspection and disk snapshots
 
 `readSchema` from `@jarenjs/db/model` inventories tables without adopting them,
@@ -188,3 +282,15 @@ calls. No global strong model cache is introduced. Keep connection/query caches
 bounded and reuse compiled metadata. Smaller import graphs alone do not prove
 the complete application meets its RSS budget; measure application memory with
 representative workloads.
+
+Entity mutation engines retain prepared statements by their complete emitted SQL
+in the bounded core LRU cache. Bound values, output projections and row/byte limits
+belong to each execution; changing a payload does not retain another document and
+another copy of the same statement. Distinct SQL stays isolated. This reduces
+retained payload memory, at the cost of rebinding/compiling an identical mutation
+document on each call. The synthetic retention probe at
+`test/db/fixtures/mutation-memory.mjs` reports both varying-payload and identical
+mutation timings, heap and RSS on Node (`--expose-gc`) and Bun. Such samples do not
+replace a complete application's resource gate. Metadata remains caller-owned;
+keep one compiled mapping per used model, one query state per connection, and
+release facade/cache references on close. Driver cursors remain ephemeral.

package/package.json

@@ -1,12 +1,16 @@
 {
   "name": "@jarenjs/db",
   "private": false,
-  "version": "0.85.0",
+  "version": "0.87.0",
   "type": "module",
   "main": "./src/index.js",
   "types": "./types/index.d.ts",
   "sideEffects": false,
   "exports": {
+    "./node-process": {
+      "types": "./types/node-process.d.ts",
+      "default": "./src/drivers/node-process.js"
+    },
     "./search": {
       "types": "./types/search.d.ts",
       "default": "./src/search.js"
@@ -104,9 +108,9 @@
     "prepack": "npm run build:types"
   },
   "dependencies": {
-    "@jarenjs/core": "^0.85.0",
-    "@jarenjs/json": "^0.85.0",
-    "@jarenjs/validate": "^0.85.0"
+    "@jarenjs/core": "^0.87.0",
+    "@jarenjs/json": "^0.87.0",
+    "@jarenjs/validate": "^0.87.0"
   },
   "bin": {
     "jaren-db": "./src/cli.js"

package/src/dialects/sqlite-relational.js

@@ -22,7 +22,7 @@ export function relationalIdentifier(name) {
 }
 const q = relationalIdentifier;
 const binary = new Set(['=', '<>', '<', '<=', '>', '>=', 'IS', 'IS NOT', '+', '-', '*', '/', '%', '||', 'AND', 'OR', 'LIKE', 'NOT LIKE', 'GLOB']);
-const functions = new Set(['coalesce', 'nullif', 'trim', 'ltrim', 'rtrim', 'lower', 'upper', 'length', 'abs', 'round', 'typeof', 'json_extract', 'json_valid', 'count', 'sum', 'total', 'avg', 'min', 'max', 'date', 'time', 'datetime', 'julianday', 'unixepoch', 'strftime']);
+const functions = new Set(['coalesce', 'nullif', 'trim', 'ltrim', 'rtrim', 'lower', 'upper', 'length', 'abs', 'round', 'typeof', 'json_extract', 'json_valid', 'json_type', 'count', 'sum', 'total', 'avg', 'min', 'max', 'date', 'time', 'datetime', 'julianday', 'unixepoch', 'strftime']);
 const types = new Set(['INTEGER', 'REAL', 'TEXT', 'BLOB', 'NUMERIC']);
 const collations = new Set(['BINARY', 'NOCASE', 'RTRIM']);
 const node = (kind, spec) => ({ $sql: kind, ...spec });
@@ -104,6 +104,7 @@ export function relationalEmitter(options = {}) {
         if (value.distinct !== undefined && typeof value.distinct !== 'boolean') fail('DISTINCT must be boolean');
         if (value.distinct && value.args.length !== 1) fail('DISTINCT functions require one argument');
         if (!value.args.length && value.name !== 'count') fail('SQL function requires arguments');
+        if (value.name === 'json_type' && value.args.length > 2) fail('json_type requires one or two arguments');
         return `${value.name.toUpperCase()}(${value.distinct ? 'DISTINCT ' : ''}${value.args.length ? value.args.map(next).join(', ') : '*'})`;
       }
       case 'cast':

package/src/dialects/sqlite-schema.js

@@ -34,37 +34,13 @@ export function planTable(definition) {
   for (const key of ['constraints', 'indexes', 'triggers']) if (definition[key] !== undefined && !Array.isArray(definition[key])) fail(`${key} must be a list`);
   const emitter = relationalEmitter({ inline: true });
   const names = new Set();
-  let inlineKey = false;
   const columns = definition.columns.map((column) => {
-    check(column, ['name', 'type', 'nullable', 'default', 'collation', 'identity', 'check', 'generated', 'stored'], 'column definition');
-    const name = q(column.name);
+    const text = columnSql(column, definition, emitter);
     if (names.has(column.name.toLowerCase())) fail('physical column names must be distinct');
     names.add(column.name.toLowerCase());
-    if (!['INTEGER', 'REAL', 'TEXT', 'BLOB', 'NUMERIC', 'ANY'].includes(column.type)) fail('unsupported SQLite column type');
-    if (definition.strict && column.type === 'NUMERIC') fail('STRICT tables do not support NUMERIC');
-    if (column.nullable !== undefined && typeof column.nullable !== 'boolean') fail('nullable must be boolean');
-    if (column.stored !== undefined && (typeof column.stored !== 'boolean' || column.generated === undefined)) fail('stored requires a generated expression');
-    let out = `${name} ${column.type}`;
-    if (column.identity !== undefined) {
-      if (!['rowid', 'autoincrement'].includes(column.identity) || column.type !== 'INTEGER'
-        || definition.withoutRowid || definition.primaryKey?.length !== 1
-        || definition.primaryKey[0] !== column.name || column.generated !== undefined) fail('identity requires a single INTEGER rowid primary key');
-      inlineKey = true;
-      out += ` PRIMARY KEY${column.identity === 'autoincrement' ? ' AUTOINCREMENT' : ''}`;
-    }
-    if (column.nullable === false) out += ' NOT NULL';
-    if (column.collation !== undefined) {
-      if (!['BINARY', 'NOCASE', 'RTRIM'].includes(column.collation)) fail('unsupported column collation');
-      out += ` COLLATE ${column.collation}`;
-    }
-    if (Object.hasOwn(column, 'default')) out += ` DEFAULT (${emitter.expr(column.default)})`;
-    if (column.check !== undefined) out += ` CHECK (${emitter.expr(column.check)})`;
-    if (column.generated !== undefined) {
-      if (Object.hasOwn(column, 'default')) fail('a generated column cannot have a default');
-      out += ` GENERATED ALWAYS AS (${emitter.expr(column.generated)}) ${column.stored ? 'STORED' : 'VIRTUAL'}`;
-    }
-    return out;
+    return text;
   });
+  const inlineKey = definition.columns.some((column) => column.identity !== undefined);
   const members = (values) => {
     const text = list(values, 'constraint columns');
     if (values.some((name) => !names.has(name.toLowerCase()))) fail('constraint names an undeclared column');
@@ -79,11 +55,12 @@ export function planTable(definition) {
     else if (constraint.kind === 'check') columns.push(`${prefix}CHECK (${emitter.expr(constraint.expression)})`);
     else if (constraint.kind === 'foreignKey') {
       if (constraint.columns?.length !== constraint.references?.length) fail('foreign-key columns must have equal arity');
-      if (constraint.deferred !== undefined && typeof constraint.deferred !== 'boolean') fail('deferred must be boolean');
-      columns.push(`${prefix}FOREIGN KEY (${members(constraint.columns)}) REFERENCES ${q(constraint.table)} (${list(constraint.references, 'references')})`
-        + (constraint.onDelete === undefined ? '' : ` ON DELETE ${action(constraint.onDelete)}`)
-        + (constraint.onUpdate === undefined ? '' : ` ON UPDATE ${action(constraint.onUpdate)}`)
-        + (constraint.deferred ? ' DEFERRABLE INITIALLY DEFERRED' : ''));
+      columns.push(`${prefix}FOREIGN KEY (${members(constraint.columns)}) ${referenceSql({
+        table: constraint.table, columns: constraint.references,
+        ...(constraint.onDelete === undefined ? {} : { onDelete: constraint.onDelete }),
+        ...(constraint.onUpdate === undefined ? {} : { onUpdate: constraint.onUpdate }),
+        ...(constraint.deferred === undefined ? {} : { deferred: constraint.deferred }),
+      })}`);
     }
     else fail('constraint kind is unique, check or foreignKey');
     const keys = constraint.kind === 'unique' ? ['kind', 'name', 'columns']
@@ -140,3 +117,76 @@ export function planTable(definition) {
     indexes: (definition.indexes ?? []).map((i) => ({ name: i.name, unique: i.unique === true, terms: i.terms })),
   } };
 }
+
+/** One REFERENCES clause shared by table constraints and column declarations. */
+function referenceSql(reference) {
+  check(reference, ['table', 'columns', 'onDelete', 'onUpdate', 'deferred'], 'reference');
+  if (reference.deferred !== undefined && typeof reference.deferred !== 'boolean') fail('deferred must be boolean');
+  return `REFERENCES ${q(reference.table)} (${list(reference.columns, 'references')})`
+    + (reference.onDelete === undefined ? '' : ` ON DELETE ${action(reference.onDelete)}`)
+    + (reference.onUpdate === undefined ? '' : ` ON UPDATE ${action(reference.onUpdate)}`)
+    + (reference.deferred ? ' DEFERRABLE INITIALLY DEFERRED' : '');
+}
+
+/** Render one typed column without reconstructing any surrounding schema. */
+function columnSql(column, context, emitter) {
+  check(column, ['name', 'type', 'nullable', 'default', 'collation', 'identity', 'check', 'generated', 'stored', 'references'], 'column definition');
+  const name = q(column.name);
+  if (!['INTEGER', 'REAL', 'TEXT', 'BLOB', 'NUMERIC', 'ANY'].includes(column.type)) fail('unsupported SQLite column type');
+  if (context.strict && column.type === 'NUMERIC') fail('STRICT tables do not support NUMERIC');
+  if (column.nullable !== undefined && typeof column.nullable !== 'boolean') fail('nullable must be boolean');
+  if (column.stored !== undefined && (typeof column.stored !== 'boolean' || column.generated === undefined)) fail('stored requires a generated expression');
+  const hasDefault = Object.hasOwn(column, 'default');
+  if (context.additive) {
+    if (column.identity !== undefined || column.stored === true) fail('ADD COLUMN cannot add an identity or STORED column');
+    const value = column.default?.$sql === 'value' ? column.default.value : column.default;
+    if (hasDefault && !(value === null || typeof value === 'string' || typeof value === 'bigint'
+      || (typeof value === 'number' && Number.isFinite(value)))) fail('ADD COLUMN requires a literal default');
+    if (column.generated === undefined && column.nullable === false && (!hasDefault || value === null)) fail('ADD COLUMN NOT NULL requires a non-null default');
+    if (column.references !== undefined && hasDefault && value !== null) fail('ADD COLUMN REFERENCES requires a NULL default');
+  }
+  let out = `${name} ${column.type}`;
+  if (column.identity !== undefined) {
+    if (!['rowid', 'autoincrement'].includes(column.identity) || column.type !== 'INTEGER'
+      || context.withoutRowid || context.primaryKey?.length !== 1
+      || context.primaryKey[0] !== column.name || column.generated !== undefined) fail('identity requires a single INTEGER rowid primary key');
+    out += ` PRIMARY KEY${column.identity === 'autoincrement' ? ' AUTOINCREMENT' : ''}`;
+  }
+  if (column.nullable === false) out += ' NOT NULL';
+  if (column.collation !== undefined) {
+    if (!['BINARY', 'NOCASE', 'RTRIM'].includes(column.collation)) fail('unsupported column collation');
+    out += ` COLLATE ${column.collation}`;
+  }
+  if (hasDefault) out += context.additive ? ` DEFAULT ${emitter.expr(column.default)}` : ` DEFAULT (${emitter.expr(column.default)})`;
+  if (column.check !== undefined) out += ` CHECK (${emitter.expr(column.check)})`;
+  if (column.generated !== undefined) {
+    if (hasDefault) fail('a generated column cannot have a default');
+    out += ` GENERATED ALWAYS AS (${emitter.expr(column.generated)}) ${column.stored ? 'STORED' : 'VIRTUAL'}`;
+  }
+  if (column.references !== undefined) {
+    if (column.references?.columns?.length !== 1) fail('a column reference requires one referenced column');
+    out += ` ${referenceSql(column.references)}`;
+  }
+  return out;
+}
+
+/** Render one explicit main-schema operation; validation never executes SQL.
+ * @param {any} operation @returns {string} */
+export function schemaChangeSql(operation) {
+  check(operation, ['op', 'table', 'column', 'name', 'to', 'ifExists'], 'schema change');
+  switch (operation.op) {
+    case 'addColumn':
+      check(operation, ['op', 'table', 'column'], 'addColumn');
+      return `ALTER TABLE "main".${q(operation.table)} ADD COLUMN ${columnSql(operation.column, { additive: true }, relationalEmitter({ inline: true }))}`;
+    case 'renameTable':
+      check(operation, ['op', 'table', 'to'], 'renameTable');
+      return `ALTER TABLE "main".${q(operation.table)} RENAME TO ${q(operation.to)}`;
+    case 'dropIndex': case 'dropTable': {
+      const index = operation.op === 'dropIndex';
+      check(operation, ['op', index ? 'name' : 'table', 'ifExists'], operation.op);
+      if (operation.ifExists !== undefined && typeof operation.ifExists !== 'boolean') fail('ifExists must be boolean');
+      return `DROP ${index ? 'INDEX' : 'TABLE'}${operation.ifExists ? ' IF EXISTS' : ''} "main".${q(index ? operation.name : operation.table)}`;
+    }
+    default: return fail('schema change is addColumn, dropIndex, renameTable or dropTable');
+  }
+}