@delali/sirannon-db 0.1.6 → 0.1.8

package/README.md

@@ -8,8 +8,26 @@
 
 Build a networked SQLite service with connection pooling, change data capture, migrations, backups, and a client SDK. Applications reach Sirannon over HTTP or WebSocket, while Sirannon nodes replicate primary-owned changes over gRPC. Coordinator mode adds etcd-backed authority and automatic failover.
 
+The benchmarks compare Sirannon against Postgres 17 on the same OLTP workloads, driving each engine through the client it provides and matching durability on both sides. Every published figure is generated from a recorded run on a disclosed machine, and the page shows where each engine wins. See the full [methodology and results](../../BENCHMARKS.md).
+
+See a three-node cluster keep serving through a primary failure in the [distributed entitlements example](https://github.com/assetcorp/sirannon-db/tree/main/packages/ts/examples/distributed-entitlements), which runs the etcd coordinator, gRPC replication with mutual TLS, and fault injection on your machine.
+
 > *sirannon* means 'gate-stream' in Sindarin.
 
+## Project status
+
+Sirannon has two levels of maturity. The core data layer, the server, the client, and primary-replica replication are stable. Coordinator-backed automatic failover is the newest part and needs more production use before it is stable.
+
+| Part | Status | Details |
+| --- | --- | --- |
+| Core engine (`@delali/sirannon-db`) | Stable | Queries, transactions, connection pooling, change data capture, migrations, backups, hooks, metrics, and multi-tenant lifecycle, covered by more than 130 test files with continuous integration on Node 22 and 24. |
+| Server and client (`@delali/sirannon-db/server`, `@delali/sirannon-db/client`) | Stable | HTTP and WebSocket access with reconnection and subscription restore. The server runs client SQL by design, so read the [security section](#security) before you expose it. |
+| Primary-replica replication (`@delali/sirannon-db/replication`) | Stable | Hybrid Logical Clock stamping, conflict resolvers, first sync, write concerns, and a gRPC transport with mutual TLS. |
+| Coordinator-backed automatic failover (`@delali/sirannon-db/replication/coordinator/etcd`) | Experimental | etcd authority, primary terms, and in-sync sets, verified by a Docker conformance run under fault injection. It is new and not yet proven in production. |
+| Drivers | Stable: better-sqlite3, Node, wa-sqlite. Experimental: Bun, Expo | The Bun and Expo drivers run today but have no TypeScript declarations yet. |
+
+Durability follows SQLite's WAL mode with `synchronous=NORMAL` by default, and you can raise it. The [roadmap](https://github.com/assetcorp/sirannon-db/blob/main/ROADMAP.md) sets out what is next, including a second-language implementation and scaling beyond a single node's disk.
+
 ## Install
 
 ```bash
@@ -47,7 +65,7 @@ await db.execute('INSERT INTO users (name, email) VALUES (?, ?)', ['Ada', 'ada@e
 const users = await db.query<{ id: number; name: string }>('SELECT * FROM users')
 ```
 
-Node.js 22+ users can skip the extra dependency by using the built-in `node:sqlite` module (requires the `--experimental-sqlite` flag):
+Node.js 22+ users can skip the extra dependency by using the built-in `node:sqlite` module. Node enables it by default from 22.13.0 and 23.4.0 onward; earlier 22.x releases need the `--experimental-sqlite` flag. The module is still experimental.
 
 ```ts
 import { nodeSqlite } from '@delali/sirannon-db/driver/node'
@@ -103,7 +121,7 @@ const users = await db.query<{ id: number; name: string }>('SELECT * FROM users'
 
 ### Bun
 
-No extra dependency needed since Bun ships `bun:sqlite` built in.
+You need no extra dependency, because Bun includes `bun:sqlite`.
 
 ```ts
 import { Sirannon } from '@delali/sirannon-db'
@@ -129,7 +147,7 @@ Sirannon-db separates the database engine from the library. You pick the driver
 | Driver | Import | Runtime | Install |
 | --- | --- | --- | --- |
 | better-sqlite3 | `@delali/sirannon-db/driver/better-sqlite3` | Node.js | `pnpm add -E better-sqlite3` |
-| Node built-in | `@delali/sirannon-db/driver/node` | Node.js >= 22 | None (use `--experimental-sqlite` flag) |
+| Node built-in | `@delali/sirannon-db/driver/node` | Node.js >= 22 | None (built in; flag-free from Node 22.13.0 and 23.4.0) |
 | wa-sqlite | `@delali/sirannon-db/driver/wa-sqlite` | Browser | `pnpm add -E wa-sqlite` |
 | Bun | `@delali/sirannon-db/driver/bun` | Bun | None (uses `bun:sqlite`) |
 | Expo | `@delali/sirannon-db/driver/expo` | React Native | `pnpm add -E expo-sqlite` |
@@ -147,18 +165,22 @@ import { waSqlite } from '@delali/sirannon-db/driver/wa-sqlite'
 const driver = waSqlite({ vfs: 'IDBBatchAtomicVFS' })
 ```
 
+You write a custom driver by passing `capabilities` and an `open` function to `defineDriver`. To let it run on a writer worker thread (see [Writer worker](#writer-worker-offload-disk-writes)), add a `worker` entry so the worker can rebuild it: a `specifier` the worker can import, the `exportName` of your factory, and a `config` value that survives a structured clone. The worker imports the module and calls the factory with the config, because the `open` function itself can't cross the thread boundary.
+
 ## Package exports
 
-The package ships independent exports so you only bundle what you need:
+The package provides independent exports so you only bundle what you need:
 
 | Import | What you get |
 | --- | --- |
 | `@delali/sirannon-db` | Core library: queries, transactions, CDC, migrations, backups, hooks, metrics, lifecycle |
 | `@delali/sirannon-db/driver/*` | SQLite driver adapters (see table above) |
 | `@delali/sirannon-db/file-migrations` | Load `.up.sql` / `.down.sql` files from a directory |
+| `@delali/sirannon-db/backup-scheduler` | Cron-scheduled backup runner with file rotation, also re-exported from the core entry |
 | `@delali/sirannon-db/server` | HTTP + WebSocket server powered by uWebSockets.js |
 | `@delali/sirannon-db/client` | Browser/Node.js client SDK with auto-reconnect and subscription restore |
 | `@delali/sirannon-db/replication` | Replication engine, conflict resolvers, topologies, HLC |
+| `@delali/sirannon-db/replication/coordinator/etcd` | etcd-backed cluster coordinator for primary authority and automatic failover |
 | `@delali/sirannon-db/transport/grpc` | gRPC replication transport with TLS support |
 | `@delali/sirannon-db/transport/memory` | In-memory transport for testing |
 
@@ -189,6 +211,39 @@ const total = await db.transaction(async tx => {
 })
 ```
 
+### Bulk load
+
+Loading a large dataset through many small committed transactions is slow and can stall the whole server. At `synchronous = full`, every commit calls `fsync`, and because the engine is synchronous, each `fsync` blocks the event loop until the disk confirms; tens of thousands of blocking `fsync` calls back to back stop the server from answering anything. `db.bulkLoad` runs the whole batch in one transaction under a relaxed durability level, then restores the configured level before it resolves.
+
+```ts
+const summary = await db.bulkLoad(
+  'INSERT INTO events (id, payload) VALUES (?, ?)',
+  rows, // an array of parameter arrays, one per row
+  { durability: 'off' },
+)
+// summary.rowsLoaded, summary.changes
+```
+
+The load holds the single writer for its whole duration, so no other write commits under the relaxed level and no two loads race on the durability setting. On success the WAL is checkpointed at the restored level, so the loaded rows are written into the main database file before the call resolves. That checkpoint runs synchronously and blocks the event loop for the length of the WAL flush, which grows with the size of the load.
+
+`durability` defaults to `'off'`. SQLite sanctions `'off'` for a load that starts from an empty database and that the operator can re-run after a power loss; a crash during an `'off'` load can corrupt the file, so recovery means re-running the load from scratch. Use `'normal'` for a load into a database that already holds data you cannot afford to lose, because it keeps WAL corruption safety while it still drops the per-commit `fsync`. Either way the configured `synchronous` level is restored when the load finishes, and a crash mid-load leaves the configured level in force on the next open, because `PRAGMA synchronous` is connection state that SQLite never stores in the database file.
+
+The result sums the row count and the changes rather than returning one object per row, so a load of millions of rows never holds millions of result objects in memory. Over the server, one load must fit under `maxBodyBytes`; send a larger dataset as several sequential loads, each of which restores durability on its own.
+
+For a dataset that spans more than one request, the client's `db.loadAll` batches it for you. Hand it a synchronous or asynchronous iterable of parameter sets; it splits the rows into batches, runs the one fsyncing WAL checkpoint once after the final batch, and restores the configured durability after every batch:
+
+```ts
+const summary = await db.loadAll(
+  'INSERT INTO events (id, payload) VALUES (?, ?)',
+  rowStream, // an iterable or async iterable of parameter arrays
+  { batchSize: 5000, durability: 'off' },
+)
+```
+
+`batchSize` defaults to 1000, and each batch is one request, so size it to stay under the server's `maxBodyBytes`. `loadAll` runs the finalize itself and keeps the checkpoint flag out of your code. On network-attached disks, where each checkpoint fsync is slow, this collapses hundreds of per-batch flushes into one flush at the end.
+
+The low-level `db.load` exposes the same behaviour through a `checkpoint` flag for callers that batch by hand: pass `{ checkpoint: false }` on every load but the last, so the WAL checkpoint runs once, after the final load. Each load still restores the configured durability, so an import you abandon partway keeps the writer at the configured level, and SQLite's automatic checkpoint keeps the WAL bounded during the import. Prefer `loadAll` unless you need that control, because a forgotten final `checkpoint: true` leaves the last pages in the WAL until the next checkpoint.
+
 ### Connection pooling
 
 Every database opens with 1 dedicated write connection and N read connections (default 4). WAL mode is enabled by default, allowing concurrent reads during writes.
@@ -296,10 +351,13 @@ db.scheduleBackup({
   cron: '0 */6 * * *',      // every 6 hours
   destDir: './backups',
   maxFiles: 10,              // keep the 10 most recent
+  timezone: 'America/New_York', // optional; defaults to the host timezone
   onError: err => console.error('Backup failed:', err),
 })
 ```
 
+The cron expression supports five or six fields (an optional leading seconds field), ranges, steps, lists, month and weekday names, and `@daily`-style nicknames. It runs in `timezone` when you set one, and in the host's local timezone otherwise. When the clocks go forward for daylight saving time, the scheduler skips the missing hour; when they go back, it runs a backup timed for the repeated hour once. The scheduler does not backfill: if the host sleeps or the clock jumps forward past a scheduled time, that run is skipped rather than run late, and a backward clock step repeats nothing.
+
 ### Hooks
 
 Hooks run before or after key operations. Throwing from a before-hook denies the operation.
@@ -379,19 +437,76 @@ await server.listen()
 
 See the [Security](#security) section for authentication, TLS, and CORS configuration.
 
+The server offers three write shapes, on both transports. Reach for each one when:
+
+- **transaction** runs several *different* statements that must all succeed or all fail together, such as a debit on one row and a credit on another.
+- **batch** runs *one* statement many times with different values, such as inserting a thousand rows into the same table. It costs less than a transaction of a thousand near-identical statements, and it stays all-or-nothing.
+- **load** runs a batch for a large, from-scratch import. It relaxes durability while the rows go in and restores it afterward, so it trades power-loss safety during the load for speed; if the process dies mid-load, you re-run it.
+
+### Writer worker (offload disk writes)
+
+Under full durability every commit flushes to disk, and a WAL checkpoint flushes too. While the serving thread runs one of those flushes it can't accept or answer connections, so a burst of writes can push the server into refusing fresh connections. Turn on `writerWorker` to run writes, WAL checkpoints, bulk loads, migrations, and backups on a dedicated worker thread. The serving thread hands the work across and stays free to accept and answer connections. Reads stay on the serving thread, since they're fast and served from the page cache.
+
+```ts
+const db = await sirannon.open('app', './data/app.db', {
+  synchronous: 'full',
+  writerWorker: true,
+})
+```
+
+Full durability holds. A write returns to the client only after its flush completes on the worker, exactly as it does without offload. The built-in `better-sqlite3` and `node` drivers support offload, and a custom driver opts in by declaring a worker entry (see [Pluggable drivers](#pluggable-drivers)). Enabling `writerWorker` on a driver that can't support it fails at open with a clear error rather than quietly keeping writes on the serving thread.
+
+Pass an object instead of `true` to tune backpressure and the worker's lifecycle:
+
+```ts
+const db = await sirannon.open('app', './data/app.db', {
+  synchronous: 'full',
+  writerWorker: {
+    maxPendingWrites: 1024,
+    writeTimeoutMs: 30000,
+    maxRestarts: 5,
+  },
+})
+```
+
+`maxPendingWrites` bounds how many writes may be in flight before the server sheds load. Past it, a write returns HTTP 503 with a `Retry-After` header, and a `WRITE_OVERLOADED` error over WebSocket, so clients back off and retry instead of the server buffering without bound. Size it from your sustainable write rate times your worst-case write latency. `writeTimeoutMs` is the per-operation deadline, so a hung flush fails loudly instead of hanging a client. When it expires, the host asks the worker to cancel the operation; the worker itself keeps running, since a thread inside a synchronous SQLite call cannot be interrupted safely. Work the worker has not started yet is shed and rejected with `WRITE_OVERLOADED`, a known outcome that is safe to retry. A result that arrives within one further deadline is delivered normally, and an operation still unresolved after that grace window is rejected with `WRITER_WORKER_TIMEOUT`, an indeterminate outcome: the write may or may not have committed, so reconcile state before retrying a non-idempotent write. A genuinely dead disk keeps rejecting writes until you restart the process. Raise the deadline only for unusually large single operations, such as dropping a very large table. `maxRestarts` caps how many times the worker is respawned after it crashes on its own before writes fail permanently.
+
 ### HTTP routes
 
 | Method | Path | Description |
 | --- | --- | --- |
 | `POST` | `/db/:id/query` | Execute a SELECT, returns `{ rows }` |
 | `POST` | `/db/:id/execute` | Execute a mutation, returns `{ changes, lastInsertRowId }` |
-| `POST` | `/db/:id/transaction` | Execute a batch of statements atomically, returns `{ results }` |
+| `POST` | `/db/:id/transaction` | Execute many statements atomically in one transaction, returns `{ results }` |
+| `POST` | `/db/:id/batch` | Apply one statement over many parameter sets in one transaction, returns `{ results }` |
+| `POST` | `/db/:id/load` | Bulk-load rows with relaxed durability, returns `{ rowsLoaded, changes }` |
+| `GET` | `/db/:id/cluster` | Cluster status for the database: role, replication group, current primary, primary term, read endpoints, and health; returns 404 when the server has no cluster status source configured |
 | `GET` | `/health` | Liveness check |
 | `GET` | `/health/ready` | Readiness check with per-database status |
 
 ### WebSocket protocol
 
-Connect to `ws://host:port/db/:id` and send JSON messages for queries, executions, and CDC subscriptions. The server dispatches change events to subscribers in real time.
+Connect to `ws://host:port/db/:id` and send JSON messages. Every message carries a `type` and a client-chosen `id`, and every reply echoes that `id`. The server dispatches CDC change events to subscribers in real time.
+
+| Inbound `type` | Fields | Reply |
+| --- | --- | --- |
+| `query` | `sql`, `params?` | `{ type: 'result', data: { rows } }` |
+| `execute` | `sql`, `params?` | `{ type: 'result', data: { changes, lastInsertRowId } }` |
+| `transaction` | `statements`, `writeConcern?` | `{ type: 'result', data: { results } }` |
+| `batch` | `sql`, `paramsBatch`, `writeConcern?` | `{ type: 'result', data: { results } }` |
+| `load` | `sql`, `paramsBatch`, `durability?`, `checkpoint?` | `{ type: 'result', data: { rowsLoaded, changes } }` |
+| `subscribe` | `table`, `filter?`, `sinceSeq?`, `epoch?` | `{ type: 'subscribed', seq?, epoch?, resync? }` then `change` events |
+| `unsubscribe` | - | `{ type: 'unsubscribed' }` |
+
+The `transaction`, `batch`, and `load` messages run every statement server-side in one transaction and reply once. The server never holds the write lock across a network round-trip, so it does not accept an interactive transaction where the client sends `BEGIN`, then more statements, then `COMMIT` over separate messages; a single slow or dead client would otherwise freeze every write to the database.
+
+Each `change` event carries the change `type` (`insert`, `update`, or `delete`), the `table`, the `row`, the `oldRow` for updates and deletes, the `seq` as a decimal string, and a `timestamp` in milliseconds since the epoch.
+
+A subscription can resume after a reconnect. Send `sinceSeq`, the highest `seq` the client has processed, as a decimal string, and the server replays every retained change with a greater `seq` before delivering live events. The `subscribed` reply carries `seq`, the sequence the subscription is live from, and `epoch`, which identifies the sequence space the changes come from; store both and echo `epoch` when resuming, so a cursor carried to a different database forces a resync instead of a silent replay of unrelated rows. When the requested `sinceSeq` falls below the retained history, or the `epoch` does not match, the reply sets `resync: true`: the subscription still starts live from now, and the client must treat its prior state as stale and re-read it. The server's `cdcRetentionMs` option bounds how far back a subscriber can resume. The client SDK handles all of this for you when `autoReconnect` is on.
+
+### Values over the wire
+
+Both transports round-trip every SQLite value, including 64-bit integers and binary blobs, even though the messages are JSON. A binary value crosses the wire as a hex envelope, `{ "__sirannon_blob": "<uppercase hex>" }`. An integer beyond JavaScript's safe range crosses as a decimal-string envelope, `{ "__sirannon_int": "<decimal string>" }`, while an integer inside the safe range narrows to a plain number. A `lastInsertRowId` beyond the safe range is returned as a decimal string. The same envelopes work in bind parameters, and the server rejects a malformed envelope instead of passing it to SQL. The client SDK encodes parameters and decodes query rows and change events for you, so `BigInt` and `Uint8Array` values round-trip with no application code. The normative definition is in the specification's server document, [`packages/spec/05-server.md`](../spec/05-server.md).
 
 ## Client SDK
 
@@ -412,7 +527,7 @@ const users = await db.query<{ id: number; name: string }>('SELECT * FROM users'
 
 await db.execute('INSERT INTO users (name) VALUES (?)', ['Turing'])
 
-const sub = db.subscribe('users', event => {
+const sub = await db.on('users').subscribe(event => {
   console.log('User changed:', event)
 })
 
@@ -440,6 +555,10 @@ httpClient.close()
 
 ## Distributed replication
 
+<p align="center">
+  <img src="../../docs/assets/replication-topology.svg" alt="Sirannon replication topology: application clients reach the primary and read replicas, the primary replicates to replicas over gRPC with mutual TLS, and an etcd coordinator tracks authority, leases, and the in-sync set." width="820">
+</p>
+
 Sirannon can replicate a SQLite database across multiple nodes with change propagation, new-node bootstrapping, write concerns, and coordinator-backed failover. The production path is primary-replica: one primary accepts writes, replicas serve reads and can forward writes, and coordinator mode manages authority when failover is enabled. When replication is not enabled, the replication engine does not run.
 
 ```ts
@@ -555,7 +674,7 @@ Automatic write failover needs at least three voting data-bearing nodes. With fe
 
 Normal writes are serialised through one primary per replication group. When a receiver applies a batch and finds the target row already present, it passes the local and incoming versions to the configured resolver. This is part of normal batch application, not a separate repair command.
 
-Three built-in strategies ship with the replication module:
+The replication module includes three built-in strategies:
 
 | Strategy | Class | Behaviour |
 | --- | --- | --- |
@@ -580,7 +699,7 @@ When a new node joins a running cluster, it needs the full dataset before it can
 
 The state machine is: `pending` -> `syncing` -> `catching-up` -> `ready`. You can monitor it via `engine.status().syncState`.
 
-During `syncing`, `syncState.completedTables` lists the tables the joiner has finished and `syncState.totalTables` holds how many it will receive in all, so `completedTables.length / totalTables` gives you first-sync progress. A source that predates this field leaves `totalTables` at 0 until the sync finishes.
+During `syncing`, `syncState.completedTables` lists the tables the joiner has finished and `syncState.totalTables` records how many it will receive in all, so `completedTables.length / totalTables` gives you first-sync progress. A source that predates this field leaves `totalTables` at 0 until the sync finishes.
 
 For large databases where a network transfer is impractical, the out-of-band path lets you copy the SQLite file directly and start from a known sequence:
 
@@ -612,23 +731,23 @@ In coordinator mode, `majority` is calculated from configured voting data-bearin
 
 #### Is this SQLite over a shared network file system?
 
-No. Each node owns its own SQLite database file. Sirannon moves change batches through its replication transport and exposes database operations through the server and client layers. It does not rely on many machines opening the same SQLite file over NFS or another shared network file system.
+No. Each node has its own SQLite database file. Sirannon moves change batches through its replication transport and exposes database operations through the server and client layers. It does not rely on many machines opening the same SQLite file over NFS or another shared network file system.
 
 #### What is replicated?
 
-Sirannon replicates checksummed batches of `ReplicationChange` records. Each change carries the table, operation, row ID, primary key, HLC timestamp, transaction ID, node ID, old data, new data, and optional DDL statement.
+Sirannon replicates checksummed batches of `ReplicationChange` records. Each change includes the table, operation, row ID, primary key, HLC timestamp, transaction ID, node ID, old data, new data, and an optional DDL statement.
 
 #### Is the protocol row-based, statement-based, operation-log based, or CRDT-like?
 
-It is operation-log based at the Sirannon layer. Data changes carry row images and primary-key metadata. DDL changes carry a validated DDL statement. The current production write path is not CRDT-like; it prevents normal write conflicts with a single writable primary.
+It is operation-log based at the Sirannon layer. Data changes include row images and primary-key metadata. DDL changes include a validated DDL statement. The current production write path is not CRDT-like; it prevents normal write conflicts with a single writable primary.
 
 #### What ordering model does it use?
 
-Each change carries a Hybrid Logical Clock timestamp. The HLC gives deterministic causal ordering across nodes without relying on perfectly synchronised wall clocks. Batches also carry a sequence range, checksum, and, in coordinator mode, `groupId` and `primaryTerm`.
+Each change includes a Hybrid Logical Clock timestamp. The HLC gives deterministic causal ordering across nodes without relying on perfectly synchronised wall clocks. Batches also include a sequence range, checksum, and, in coordinator mode, `groupId` and `primaryTerm`.
 
 #### What happens under partitions?
 
-Static mode does not own failover. If the static primary is lost, writes stay unavailable until an operator or external system promotes another node and reroutes clients.
+Static mode has no automatic failover. If the static primary is lost, writes stay unavailable until an operator or external system promotes another node and reroutes clients.
 
 Coordinator mode uses a cluster coordinator, primary terms, leases, in-sync sets, and fail-closed write behaviour. A primary may accept writes only while it can prove current authority. Replicas reject stale batches, stale sync messages, and stale forwarded writes. Only an in-sync replica can be promoted.
 
@@ -650,7 +769,7 @@ Read concern controls this. `local` reads the selected node's local state. `majo
 
 #### Is this local-first or multi-writer today?
 
-The current production path is primary-replica. Conflict resolvers decide how a receiving node applies a change to an existing row; they do not provide local-first reconciliation or a multi-writer CRDT layer.
+The current production path is primary-replica. Conflict resolvers determine how a receiving node applies a change to an existing row; they do not provide local-first reconciliation or a multi-writer CRDT layer.
 
 ### Transport options
 
@@ -691,7 +810,7 @@ The current production path is primary-replica. Conflict resolvers decide how a
 | `maxBatchChanges` | `number` | `1000` | Maximum accepted changes in one inbound batch |
 | `ackTimeoutMs` | `number` | `5000` | Replication batch ack timeout |
 | `initialSync` | `boolean` | `true` | Pull a full snapshot when joining a cluster |
-| `syncBatchSize` | `number` | `10000` | Rows per sync batch during initial sync |
+| `syncBatchSize` | `number` | `10000` | Rows per sync batch during first sync |
 | `maxConcurrentSyncs` | `number` | `2` | Maximum simultaneous sync sessions on the source |
 | `maxSyncDurationMs` | `number` | `1800000` | Source aborts sync after this duration (30 min) |
 | `maxSyncLagBeforeReady` | `number` | `100` | Catch-up lag threshold (in sequences) to transition to ready |
@@ -699,7 +818,7 @@ The current production path is primary-replica. Conflict resolvers decide how a
 | `catchUpDeadlineMs` | `number` | `600000` | Max time in catch-up phase before transitioning to ready (10 min) |
 | `resumeFromSeq` | `bigint` | - | Start replication from a specific sequence (out-of-band sync) |
 | `snapshotConnectionFactory` | `() => Promise<SQLiteConnection>` | - | Factory for read-only connections used during sync serving |
-| `changeTracker` | `ChangeTracker` | - | CDC trigger manager, required for initial sync |
+| `changeTracker` | `ChangeTracker` | - | CDC trigger manager, required for first sync |
 | `flowControl` | `{ maxLagSeconds?, onLagExceeded? }` | - | Replication lag monitoring callbacks |
 | `onBeforeForwardedQuery` | `(sql, params?) => void` | - | Validation or authorisation hook called before the primary executes each forwarded statement |
 | `coordinator` | `CoordinatorModeConfig` | - | Enables coordinator-backed authority and failover |
@@ -820,7 +939,7 @@ Protocol values must be valid `Sec-WebSocket-Protocol` tokens. If you derive the
 
 ### TLS and transport security
 
-The built-in server binds plain HTTP and WebSocket. For any traffic outside a trusted local network, terminate TLS upstream with a reverse proxy, load balancer, or platform edge and use `https://` and `wss://` client URLs. Without TLS, credentials, SQL text, parameters, and CDC payloads travel in cleartext.
+The built-in server binds plain HTTP and WebSocket. For any traffic outside a trusted local network, terminate TLS upstream with a reverse proxy, load balancer, or platform edge and use `https://` and `wss://` client URLs. Without TLS, credentials, SQL text, parameters, and CDC payloads are sent in cleartext.
 
 ### CORS and browser access
 
@@ -885,6 +1004,17 @@ All errors extend `SirannonError` with a machine-readable `code` property:
 | `MaxDatabasesError` | `MAX_DATABASES` | Capacity limit reached |
 | `ExtensionError` | `EXTENSION_ERROR` | SQLite extension load failure |
 
+The server and the bulk-load path add a few more codes. `createServer` throws `SirannonError` with `INVALID_MAX_BODY_BYTES` when `maxBodyBytes` is not a positive integer or exceeds `4_294_967_295`, the largest value uWebSockets.js can store; a larger value would wrap modulo 2^32 and enforce a limit you never configured, so the server refuses to start instead. `INVALID_WS_BACKPRESSURE` guards `maxWebSocketBackpressureBytes` with the same bounds. A bulk load throws `INVALID_DURABILITY` when `durability` is neither `'off'` nor `'normal'`, and `DURABILITY_RESTORE_FAILED` when the load committed but the writer connection failed before its durability could be restored; treat that last code as 'the load succeeded, do not re-run it'. Over the wire the server also returns `PAYLOAD_TOO_LARGE` when a request or message exceeds `maxBodyBytes`, and `BULK_LOAD_UNSUPPORTED` when the resolved execution target for a database does not implement bulk load.
+
+The [writer worker](#writer-worker-offload-disk-writes) adds its own family of codes:
+
+| Code | When | Retry? |
+| --- | --- | --- |
+| `WRITE_OVERLOADED` | More writes are pending than `maxPendingWrites` allows, or a queued write was shed when an earlier operation's deadline expired. Over HTTP this maps to a 503 with a `Retry-After` header. | Yes; the write was never applied. |
+| `WRITER_WORKER_TIMEOUT` | An in-flight operation did not resolve within `writeTimeoutMs` plus the grace window. | Only after reconciling state; the outcome is indeterminate. |
+| `WRITER_WORKER_UNSUPPORTED` | `writerWorker` was enabled on a driver without a worker entry; the database refuses to open. | No; use a driver with a worker entry or turn the option off. |
+| `INVALID_WRITER_WORKER` | A `writerWorker` option is not an integer in its allowed range. | No; fix the configuration. |
+
 ```ts
 import { QueryError } from '@delali/sirannon-db'
 
@@ -916,8 +1046,10 @@ try {
 | `readOnly` | `boolean` | `false` | Open in read-only mode |
 | `readPoolSize` | `number` | `4` | Number of read connections |
 | `walMode` | `boolean` | `true` | Enable WAL mode |
+| `synchronous` | `'off' \| 'normal' \| 'full' \| 'extra'` | `'normal'` | Writer durability (`PRAGMA synchronous`); this is the level a bulk load restores when it finishes |
 | `cdcPollInterval` | `number` | `50` | CDC polling interval in ms |
 | `cdcRetention` | `number` | `3_600_000` | CDC retention period in ms (1 hour) |
+| `writerWorker` | `boolean \| WriterWorkerOptions` | `false` | Run writes on a dedicated worker thread so disk flushes never block the serving thread; see [Writer worker](#writer-worker-offload-disk-writes) |
 
 ### `ServerOptions`
 
@@ -926,6 +1058,9 @@ try {
 | `host` | `string` | `'127.0.0.1'` | Bind address |
 | `port` | `number` | `9876` | Listen port |
 | `cors` | `boolean \| CorsOptions` | `false` | CORS configuration |
+| `maxBodyBytes` | `number` | `1_048_576` | Maximum HTTP request body and WebSocket message size in bytes; one value governs both transports, and it must be a positive integer no larger than `4_294_967_295` |
+| `maxWebSocketBackpressureBytes` | `number` | larger of `16_777_216` and `maxBodyBytes` | Maximum bytes buffered per WebSocket connection before the server closes it so the client reconnects rather than losing a frame silently; must be at least `maxBodyBytes` so a single frame fits, and no larger than `4_294_967_295` |
+| `cdcRetentionMs` | `number` | `3_600_000` | How long change events are retained for WebSocket CDC subscriptions; bounds both change-log growth and how far back a reconnecting subscriber can resume with `sinceSeq` |
 | `onRequest` | `OnRequestHook` | - | Middleware hook for auth, rate limiting, and request validation |
 
 ### `ClientOptions`
@@ -984,7 +1119,7 @@ pnpm run dev
 
 ## Benchmarks
 
-The benchmark suite compares Sirannon's embedded SQLite performance against Postgres 17 across micro-operations, YCSB, TPC-C, and concurrency scaling. All benchmarks support driver switching via the `BENCH_DRIVER` environment variable (`better-sqlite3` or `node`). See [`benchmarks/BENCHMARKS.md`](benchmarks/BENCHMARKS.md) for setup instructions, configuration, Docker-based fair comparisons, and statistical analysis methodology.
+The benchmark suite compares Sirannon against Postgres 17 on the same OLTP workloads, driving Sirannon over HTTP into its real server and Postgres over its socket, both in resource-capped containers at matched durability, under an open-loop load generator that corrects for coordinated omission. The harness is a Python project at the repository root under `benchmarks/server`. See [`BENCHMARKS.md`](../../BENCHMARKS.md) for the methodology and the latest results.
 
 ## Development
 

package/dist/backup-scheduler/index.d.ts

@@ -1,3 +1,18 @@
-export { a as BackupScheduler } from '../index-CLdNrcPz.js';
-export { m as BackupScheduleOptions } from '../types-D-74JiXb.js';
-import '../types-BFSsG77t.js';
+import { e as SQLiteConnection, f as BackupScheduleOptions } from '../types-D_hQW1hr.js';
+
+declare class BackupManager {
+    backup(conn: SQLiteConnection, destPath: string): Promise<void>;
+    generateFilename(): string;
+    rotate(dir: string, maxFiles: number): void;
+}
+
+/** Runs a backup while the caller's writer lock is held, so it never shares the writer connection with another write. */
+type RunExclusive = (op: () => Promise<void>) => Promise<void>;
+declare class BackupScheduler {
+    private readonly manager;
+    constructor(manager?: BackupManager);
+    schedule(conn: SQLiteConnection, options: BackupScheduleOptions, runExclusive?: RunExclusive): () => void;
+    private run;
+}
+
+export { BackupScheduleOptions, BackupScheduler };

package/dist/backup-scheduler/index.mjs

@@ -1,2 +1,2 @@
-export { BackupScheduler } from '../chunk-PXKAKK2V.mjs';
-import '../chunk-O7BHI3CF.mjs';
+export { BackupScheduler } from '../chunk-VLTICJOD.mjs';
+import '../chunk-YPYVQJ4C.mjs';

package/dist/{change-tracker-CFTQ9TSn.d.ts → change-tracker-CbmaMO-N.d.ts}

@@ -1,5 +1,4 @@
-import { a as SQLiteConnection } from './types-BFSsG77t.js';
-import { C as ChangeEvent } from './types-D-74JiXb.js';
+import { e as SQLiteConnection, C as ChangeEvent } from './types-D_hQW1hr.js';
 
 interface ChangeTrackerOptions {
     retention?: number;
@@ -70,6 +69,17 @@ declare class ChangeTracker {
      */
     pruneDroppedTables(conn: SQLiteConnection, tables: readonly string[]): Promise<void>;
     poll(conn: SQLiteConnection): Promise<ChangeEvent[]>;
+    /** The highest seq already polled; live subscribers receive events beyond it. */
+    get cursor(): bigint;
+    /**
+     * Reads retained changes for one table with seq in `(afterSeq, upToSeq]`,
+     * ordered ascending and capped at `limit`. Used to replay history to a
+     * resuming subscriber without disturbing the shared poll cursor.
+     */
+    readSince(conn: SQLiteConnection, table: string, afterSeq: bigint, upToSeq: bigint, limit: number): Promise<ChangeEvent[]>;
+    /** The lowest retained seq, or `null` when the change log is empty. */
+    getMinSeq(conn: SQLiteConnection): Promise<bigint | null>;
+    private rowToEvent;
     advanceToLatest(conn: SQLiteConnection): Promise<void>;
     cleanup(conn: SQLiteConnection): Promise<number>;
     setPruneBoundary(seq: bigint): void;

package/dist/chunk-4ISB7XMA.mjs

@@ -0,0 +1,21 @@
+// src/core/worker/protocol.ts
+var WORKER_CANCELLED_CODE = "WRITER_WORKER_CANCELLED";
+function serializeError(err) {
+  if (err instanceof Error) {
+    const code = err.code;
+    return {
+      message: err.message,
+      name: err.name,
+      ...typeof code === "string" ? { code } : {}
+    };
+  }
+  return { message: String(err) };
+}
+function deserializeError(error) {
+  const err = new Error(error.message);
+  if (error.name) err.name = error.name;
+  if (error.code) err.code = error.code;
+  return err;
+}
+
+export { WORKER_CANCELLED_CODE, deserializeError, serializeError };

package/dist/chunk-BNUTBHHH.mjs

@@ -0,0 +1,22 @@
+import { SirannonError } from './chunk-YPYVQJ4C.mjs';
+
+// src/core/driver/define.ts
+function defineDriver(config) {
+  if (!config.capabilities || typeof config.open !== "function") {
+    throw new SirannonError("Driver must define capabilities and open()", "INVALID_DRIVER");
+  }
+  if (config.worker !== void 0 && typeof config.worker.specifier !== "string") {
+    throw new SirannonError("Driver worker entry must define a string specifier", "INVALID_DRIVER");
+  }
+  return Object.freeze({
+    capabilities: Object.freeze({ ...config.capabilities }),
+    open: config.open,
+    ...config.worker ? { worker: Object.freeze({ ...config.worker }) } : {},
+    ...config.startWriterHost ? { startWriterHost: config.startWriterHost } : {},
+    ...config.createWriterContext ? { createWriterContext: config.createWriterContext } : {},
+    ...config.createBackupEngine ? { createBackupEngine: config.createBackupEngine } : {},
+    ...config.resolveExtensionPath ? { resolveExtensionPath: config.resolveExtensionPath } : {}
+  });
+}
+
+export { defineDriver };

package/dist/chunk-CJLYFDP5.mjs

@@ -0,0 +1,26 @@
+import { SirannonError } from './chunk-YPYVQJ4C.mjs';
+
+// src/core/driver/synchronous.ts
+var DEFAULT_SYNCHRONOUS = "normal";
+var SYNCHRONOUS_PRAGMA_VALUES = {
+  off: "OFF",
+  normal: "NORMAL",
+  full: "FULL",
+  extra: "EXTRA"
+};
+function synchronousPragmaValue(level) {
+  const resolved = level ?? DEFAULT_SYNCHRONOUS;
+  const value = SYNCHRONOUS_PRAGMA_VALUES[resolved];
+  if (value === void 0) {
+    throw new SirannonError(
+      `Invalid synchronous level '${String(resolved)}': expected 'off', 'normal', 'full', or 'extra'`,
+      "INVALID_SYNCHRONOUS"
+    );
+  }
+  return value;
+}
+function isSynchronousLevel(value) {
+  return value === "off" || value === "normal" || value === "full" || value === "extra";
+}
+
+export { DEFAULT_SYNCHRONOUS, isSynchronousLevel, synchronousPragmaValue };