@kedem/okdb 1.9.1 → 2.0.0

package/docs/index.md

@@ -72,7 +72,7 @@ All environments are LMDB directories on disk. Each has its own clock, changelog
 ## Quick example
 
 ```javascript
-const OKDB = require('src/okdb');
+const OKDB = require('@kedem/okdb');
 
 const okdb = new OKDB('./mydb');
 await okdb.open();
@@ -99,18 +99,30 @@ await okdb.close();
 ## Docs map
 
 - [Getting Started](./getting-started.md) — install, open, first writes
+- [Roles & Deployment](./deployment.md) — role flags, single-process vs multi-process topologies, deferred processing
 - [Data Model](./data-model.md) — types, keys, environments, storage layout
 - [Querying](./querying.md) — reads, ranges, MongoDB-style filters
 - [Indexes](./indexes.md) — secondary, composite, unique, geo
 - [Transactions](./transactions.md) — batching, versioning, optimistic concurrency
 - [Change Log & Events](./change-log.md) — clock, changelog, processor, events
+- [Processors](./processors.md) — the change-subscription primitive: modes, registration, leases
 - [Sync](./sync.md) — multi-node replication
 - [Full-Text Search](./fts.md) — FTS index, tokenizer, query
 - [Embeddings](./embeddings.md) — vector pipeline, semantic search
+- [Materialized Views](./views.md) — reactive reduce/group views over types
+- [Time Machine](./time-machine.md) — per-type field-level history, point-in-time reads
 - [Queue](./queue.md) — durable job queue
 - [Live Subscriptions](./subscriptions.md) — signal-SSE change streams with durable sessions
 - [Files](./files.md) — blob / file storage
+- [TTL](./ttl.md) — per-record time-to-live
 - [Custom Functions](./functions.md) — sandboxed stored JS functions, env/global scope, HTTP execution
+- [Pipelines](./pipelines.md) — declarative content/embedding pipelines
 - [HTTP API](./http-api.md) — built-in REST API
+- [HTTP Clustering](./http-cluster.md) — the CLI's N-capable-nodes cluster
+- [Process Registry](./process-registry.md) — root-wide process visibility and control
+- [Auth & Sync](./auth-and-sync.md) / [Auth Permissions](./auth-permissions.md) — tokens, OAuth, grants
+- [Logging](./logging.md) — structured logs, retention
+- [Diagnostics](./diagnostics.md) — opt-in triage tooling (write-origin ring, fatal report, loop lag)
 - [Plugins](./plugins.md) — extending OKDB before open
 - [Migration](./migration.md) — export, import, backup, blob integrity
+- [Upgrading to 2.0](./upgrade-2.0.md) — the node-model runtime redesign: constructor changes, workers dissolved, claimed responsibilities

package/docs/manifest.json

@@ -1,6 +1,7 @@
 [
     { "file": "index.md", "label": "Overview" },
     { "file": "getting-started.md", "label": "Getting Started" },
+    { "file": "deployment.md", "label": "Roles & Deployment" },
     {
         "label": "Core",
         "children": [
@@ -23,7 +24,8 @@
             { "file": "queue.md", "label": "Queue" },
             { "file": "files.md", "label": "Files" },
             { "file": "functions.md", "label": "Custom Functions" },
-            { "file": "ttl.md", "label": "TTL (Time-to-Live)" }
+            { "file": "ttl.md", "label": "TTL (Time-to-Live)" },
+            { "file": "time-machine.md", "label": "Time Machine" }
         ]
     },
     {
@@ -36,9 +38,10 @@
             { "file": "process-registry.md", "label": "Process Registry" },
             { "file": "plugins.md", "label": "Plugins" },
             { "file": "processors.md", "label": "Processors (modes & registration)" },
-            { "file": "worker-fleet.md", "label": "Workers (population & placement)" },
             { "file": "logging.md", "label": "Logging" },
-            { "file": "migration.md", "label": "Migration" }
+            { "file": "diagnostics.md", "label": "Diagnostics" },
+            { "file": "migration.md", "label": "Migration" },
+            { "file": "upgrade-2.0.md", "label": "Upgrading to 2.0" }
         ]
     }
 ]

package/docs/pipelines.md

@@ -216,7 +216,7 @@ Config notes:
 - `handler.env` — optional env name for the function registry; defaults to the engine owning env
 - `concurrency`, `pollInterval`, `ttl` — forwarded to the underlying queue worker runtime
 
-The function runner pool scales dynamically to match `concurrency` — no pool configuration is needed.
+Each claimed payload's function runs in the executing node's local sandbox thread (lazy, idle-reaped); `concurrency` bounds in-flight invocations — there is no pool to configure.
 
 Behavior:
 

package/docs/process-registry.md

@@ -29,18 +29,18 @@ The primary entity is the **process**, not the processor. Each long-lived proces
   pid, ppid, host,
   supervisorKey,      // nodeKey of the supervising process (null = a root) → builds the tree
   slot, slotKey,      // supervised position; slotKey is the STABLE durable control key (see below)
-  role,               // { engines, asyncProcessors, compaction, http }
+  role,               // { engines, processors, compaction, http } — booleans
   listenAddr,
   caps,               // { controllable, decommissionable, killable, killConfirm? } — see Capabilities
   lifecycle,          // { state: 'running' | 'decommissioning', desiredVersionApplied }
-  processing,         // FACET (owners only): { pools[], processors[], cap, policy }
+  processing,         // FACET (participants only): { participate, processors[] }
   heartbeatAt, expiresAt,
 }
 ```
 
 A processor (FTS, views, materializer, time-machine, …) is **not** a separate entity — it is summarized
-on the owner's `processing` facet. The same is true of process-backend pool workers: they are real OS
-processes but are summarized on `processing.pools[]`, not given their own rows.
+on the owner's `processing` facet (`processing.processors[]`). There are no worker pools in 2.0 —
+drains run on the claiming node's loop, so there is nothing pool-shaped to give its own row.
 
 ### Two stores, split by lifetime
 
@@ -163,7 +163,7 @@ const db = new OKDB('./data', { processes: { register: true, kind: 'embedded' }
 const db = new OKDB('./data', { processes: { kind: 'standalone', listenAddr: ':8484' } });
 ```
 
-`register: true` makes a process publish its own row even when it is not an `asyncProcessors` owner.
+`register: true` makes a process publish its own row even when it is not a `processors`-role owner.
 `bin/okdb` sets `kind` automatically (`primary` / `http-server`); embedders pass it explicitly.
 
 ---

package/docs/processors.md

@@ -9,15 +9,18 @@ A **processor** maintains derived state from a source type's change log. You reg
 
 OKDB's own features (indexes, views, FTS, materializer, embeddings, time-machine) are built on this primitive — and it's a public extension point you can use directly.
 
-> **See also:** [Workers](worker-fleet.md) for _where_ a processor's work runs — the worker
-> population, claiming, and scaling. This doc defines _what_ a processor is and its **modes**
-> (the guarantee); worker-fleet.md covers placement.
+> **What vs. where.** This doc defines _what_ a processor is and its **modes** (the
+> guarantee). _Whether_ a node runs `single` processors is the boolean `processors: true|false`
+> (participate vs. passive); the per-`(env,type)` lease distributes the work across whatever
+> participating nodes share the root (1-of-N, with failover). There is no filtered claim API and
+> no managed "worker population" — see [Upgrading to 2.0](upgrade-2.0.md) for the node-model
+> placement story.
 
 ---
 
 ## Modes — the delivery guarantee
 
-A mode names the **delivery guarantee**: how many instances run the handler. Handlers are always **closures** that run in **this process** — trusted derived work runs on the main event loop, chunked into cooperative quanta so a large drain never blocks the loop monolithically (see `plans/runtime-model/overview.md`). There is no worker-thread or worker-process placement choice: the only off-loop thread in the system is the function sandbox (for untrusted user code), never trusted processor work.
+A mode names the **delivery guarantee**: how many instances run the handler. All drains run on the owner process's **event loop**, chunked into cooperative quanta (`FLUSH_QUANTUM`-bounded batches with a yield between them) so a large drain never blocks the loop monolithically. There is no execution placement to configure — no thread pool, no `runIn`; the only thread in the engine is the **sandbox** that runs untrusted user-function code. Handlers register per process: pass `handler:'<name>'` (named registry), `module:{path,export}`, or — for internal features — a closure paired with a reconstructible `quantum`.
 
 | mode     | cardinality                         | cursor                      | use when                                                                                       |
 | -------- | ----------------------------------- | --------------------------- | ---------------------------------------------------------------------------------------------- |
@@ -69,7 +72,7 @@ The `handler` is always a **closure** (a function). The portable `handler: {modu
 
 ## Snapshot bootstrap and chunked drains
 
-A `single` processor with `bootstrap: 'snapshot'` performs a cooperative snapshot scan on the main loop, yielding every batch so the event loop stays responsive while a large type is bootstrapped. Live drains use the same chunked `_flush` (a `FLUSH_QUANTUM` bounds each on-loop quantum, then yields). This is exactly the path FTS uses: every FTS-indexed type registers a `single`-mode processor whose closure handler hydrates + tokenizes + writes postings, drained on the loop in quanta. There is no shared worker thread and no `workerKey` — those were removed in the runtime-model rework.
+A `single` processor with `bootstrap: 'snapshot'` performs a cooperative snapshot scan, yielding every batch so the event loop stays responsive while a large type is bootstrapped. Live drains run on the loop via chunked `_flush` (a `FLUSH_QUANTUM` bounds each quantum, then yields; heavy per-item features pass a smaller per-processor `flushQuantum`). A feature may supply a reconstructible **`quantum`** module — it re-derives the work for a clock range and commits it durably (optionally advancing the cursor atomically with its output, `selfCursor`) before the cursor moves. This is exactly the path FTS uses — every FTS-indexed type registers a `single`-mode processor (hydrate + tokenize + posting writes) drained on the loop in small bounded quanta.
 
 The cursor only advances on a successful drain, so processing is **at-least-once**; make handlers idempotent. Derived state is eventually-consistent; if you need a deterministic "caught up" point, wait on `stop.status().lastClock` reaching the source clock.
 
@@ -79,8 +82,12 @@ The cursor only advances on a successful drain, so processing is **at-least-once
 
 Each processor takes a lease (the `'proc'` lock). With several processes open on the same env, exactly one runs each `single` processor; the others wait and take over if the holder dies. The lease-holding process drains on its own loop, so there is always a single writer to your derived store.
 
+Under sustained load the lease **load-shares by cooperative hold-window claiming**: the holder drains for up to `OKDB_LEASE_HOLD_MS` (200 ms) per window, and at the window boundary yields the lease **only if a peer has signalled it wants a turn** (`~lock:want`, a decaying marker written by waiting peers with backlog). A lone process therefore keeps its leases with zero lock churn; two loaded processes rotate and split the work ≈evenly. Waiting peers re-contend on a short cadence (~one hold window) while they have backlog and on the slow `ttl/2` failover cadence when idle.
+
 A processor that hits an error **keeps its lease** (in-place `retry()`/`restart()` keeps its slot against standbys in other processes). Re-bootstrap reuses the held lease handle, and `tryAcquire` treats a fresh lease held by the same pid + processorId as a re-acquire — so an errored processor can never deadlock in `waiting` against its own heartbeat.
 
+Cross-process wakes ride the UDP bus (`SYSTEM_POKE`), which is lossy by design. An online holder therefore also runs an **idle catch-up tick** (`OKDB_PROCESSOR_CATCHUP_MS`, default 15 000, `0` disables): every interval it compares each single-mode registration's type head against its cursor and schedules a drain when behind, so a lost poke bounds staleness at one tick instead of "until the next write". The changelog GC in turn **never prunes past the slowest durable processor cursor** — a lagging drain finds its entries intact rather than silently skipping writes.
+
 ---
 
 ## Cookbook
@@ -110,16 +117,26 @@ that safe. The per-write gate costs one boolean compare per write when no flip i
 
 ---
 
-## Instance processing policy
-
-The `processing` constructor option controls where **this instance** drains processors. It is independent of the mode declared at `register()`.
-
-| value    | behavior                                                                                                                |
-| -------- | ----------------------------------------------------------------------------------------------------------------------- |
-| `'auto'` | **default.** Self-process on the loop when no workers are configured; yield to the population when desired workers > 0. |
-| `'main'` | Self-process on the main event loop (diagnostic / single-process environments).                                         |
-| `'none'` | Never drain (passive/read-only instances). Used internally for HTTP cluster workers.                                    |
-
-> **Deprecated:** `processing: 'threads'` / `'processes'` (the old worker-thread/process pool) are accepted with a one-time deprecation warning and mapped to `'auto'` — the pool executor was removed in the runtime-model rework; trusted drains run on the loop.
-
-`'auto'` is the recommended default for application code. When a population is configured via `db.workers.ensure()`, `'auto'` instances automatically stop claiming leases so population workers drain instead — no configuration change required at the embedder.
+## Where a processor runs — claiming
+
+The `processors` constructor option is a **boolean** controlling whether **this node**
+participates in claiming `single` (1-of-N) processors. It is independent of the mode declared at
+`register()`.
+
+| value   | behavior                                                                                    |
+| ------- | ------------------------------------------------------------------------------------------- |
+| `true`  | **default.** Participate — claim every unclaimed `single` lease this node sees.             |
+| `false` | Passive — claim nothing (reads/writes + inline processors still work; another node drains). |
+
+The per-`(env,type)` lease + WAITING-retry is the load balancer — start more participating nodes
+over the same path and the work spreads automatically (1-of-N, with failover when a holder dies);
+there is no population to configure and no per-node dedication.
+
+> **Removed before 2.0 shipped:** the _filtered_ claim form — `processors: '<filter>'` (throws at
+> construction) and `db.processors.process(filter)` / `processRest()` / `processEverything()`
+> (throw with a migration message), plus per-node dedication/intent. Also removed: the
+> `processing:` option family (`'auto'|'main'|'none'`, `'threads'|'processes'`) — there is no
+> execution placement; every drain runs on its claimant's loop in bounded quanta.
+> `asyncProcessors: true/false` is now `processors: true/false`. To dedicate hardware to
+> processing, run that node `processors:true` and the others `processors:false`. See
+> [Upgrading to 2.0](upgrade-2.0.md).

package/docs/queue.md

@@ -22,9 +22,9 @@ async function main() {
     const okdb = new OKDB('./db');
     await okdb.open();
 
-    // process() runs the consumer IN THIS process (it takes a closure). For a consumer that
-    // runs on the worker population or a dedicated child, use worker(type, module) /
-    // spawn(type, module) — those cross a process boundary, so they take a module path.
+    // process() runs the consumer IN THIS process — this is the model.
+    // okdb does not auto-run queue handlers; keeping a consumer process alive is the
+    // launcher's job. Scale by running more processes (pm2, systemd, docker).
     const worker = okdb.queue.process(
         'send-email',
         async (payload, ctx) => {
@@ -175,6 +175,63 @@ const worker = okdb.queue.process(
 );
 ```
 
+### Several types, one lane pool (array form)
+
+Passing an **array of entries** runs several job types through ONE shared lane pool. Each
+entry is exactly the single form folded into an object — `{ type, handler, ...perTypeOptions }`
+— and the second argument becomes the **pool options** shared by every lane:
+
+```javascript
+const worker = okdb.queue.process(
+    [
+        { type: 'sync:forward', handler: forwardSync, weight: 3, ttl: 60_000 },
+        { type: 'sync:backfill', handler: backfillSync, weight: 1, ttl: 300_000 },
+        { type: 'content:page', handler: fetchPage },
+    ],
+    {
+        concurrency: 4, // the PROCESS-WIDE cap: at most 4 handlers in flight across ALL types
+        admission: () => okdb.pressure().score < 1, // load-aware: skip claiming under distress
+    },
+);
+```
+
+Why this exists: stacking N single-type consumers gives N _independent_ lane pools — a process
+consuming 10 types at `concurrency: 2` each could run 20 handlers at once, with no total cap
+and no cross-type fairness. The array form gives you:
+
+- **One concurrency budget** across every type (the pool's `concurrency`).
+- **Weighted fairness** — `weight` (default 1) biases the claim sweep, so under contention a
+  weight-3 type gets ~3× the first-claim attempts of a weight-1 type; an idle type costs
+  nothing.
+- **One admission gate** — `admission: () => boolean` is consulted before every claim sweep;
+  while it returns false the pool claims nothing (in-flight handlers finish normally, and the
+  durable jobs remain claimable by other processes). Re-checked every `admissionInterval` ms
+  (default 500). Combine with [`db.pressure()`](#load-aware-consumers) for a load-aware
+  consumer in one line.
+
+Per-type knobs (`ttl`, `autoHeartbeat`, `onPermanentFail`) live on the entry; cluster-wide
+controls (`max_concurrency`, buckets, priority) keep working unchanged — they're enforced at
+claim time regardless of consumer shape.
+
+### Load-aware consumers
+
+`db.pressure()` returns a composite load signal for this node, cached for 250 ms (safe to
+poll per claim):
+
+```javascript
+{
+    writerStallMs,  // max oldest-pending commit age across envs — THE stall tell
+    writerDepth,    // max in-flight commit depth
+    maxDurableLag,  // max durable processor lag (FTS/views/TM/embeddings behind)
+    queuePending,   // pending+retry jobs across envs (workload, not in the score)
+    loopLagMs,      // mean event-loop delay since the previous read
+    score,          // max of the normalized distress signals; 1 ≈ at the limit
+}
+```
+
+`admission: () => okdb.pressure().score < 1` makes a consumer yield to indexing/write load
+automatically; an external autoscaler can poll the same read to decide when to add nodes.
+
 ### Handler signature
 
 The handler receives:
@@ -198,19 +255,61 @@ These local worker helpers now map to the same public queue API that remote work
 - `okdb.queue.markJobComplete(jobId, claimId, result?)`
 - `okdb.queue.markJobFail(jobId, claimId, error, code?)`
 
-### Placement: `process` vs `worker` vs `spawn`
+### Placement: `process`
 
-The three verbs split on **where the consumer runs** — and therefore on whether they take a closure or a module:
+The queue is a **substrate okdb coordinates but does not place** — handlers are trusted user code
+of unknown nature, so okdb never auto-runs them on a population. The surface is:
 
-| Verb                         | Runs in                                     | 2nd arg                                | Lifetime                   |
-| ---------------------------- | ------------------------------------------- | -------------------------------------- | -------------------------- |
-| `queue.process(type, fn)`    | **this** Node process                       | a **closure**                          | dies with this instance    |
-| `queue.worker(type, module)` | the **worker population** (other processes) | a **module path** / `{module, export}` | durable; the fleet runs it |
-| `queue.spawn(type, module)`  | a **dedicated forked child**                | a **module path**                      | tied to the child          |
+| Verb                      | Runs in               | 2nd arg       | Lifetime                |
+| ------------------------- | --------------------- | ------------- | ----------------------- |
+| `queue.process(type, fn)` | **this** Node process | a **closure** | dies with this instance |
 
-A closure can't cross a process boundary, so `worker()`/`spawn()` require a module. Passing a closure to `worker()` is an error that points you at `process()`.
+`queue.process(type, closure)` is **the model** — single-execution and at-least-once come from
+the CAS claim on the durable job row. Scale by running more independent processes: each opens
+okdb and calls `process()`, and the durable CAS claim hands each job to exactly one consumer.
+The canonical worker pattern is a standalone script (`workers/default.js`) launched by pm2,
+systemd, or docker alongside your main service.
 
-`process(...)` is great for I/O-heavy jobs but gives no isolation from CPU-heavy work — use `spawn()` (dedicated child) or `worker()` (population) for that.
+> **Removed:** `queue.worker(type, module)` — the auto-adopted "shared pool" form — throws
+> `QUEUE_WORKER_REMOVED`. `queue.spawn(type, module)` — the dedicated forked child — was also
+> removed in okdb 2.0. Run independent consumer processes instead; they claim the same durable
+> jobs exactly once via CAS. See docs/upgrade-2.0.md for migration.
+
+---
+
+## The worker pattern
+
+The canonical way to run queue consumers outside your main process:
+
+```javascript
+// workers/default.js — run alongside the main service
+const OKDB = require('okdb');
+
+async function main() {
+    const okdb = new OKDB('./db');
+    await okdb.open();
+
+    okdb.queue.process(
+        'send-email',
+        async (payload, ctx) => {
+            await sendEmail(payload.to, payload.subject);
+        },
+        { concurrency: 4 },
+    );
+
+    // okdb.queue.process('resize-image', async (payload, ctx) => { ... });
+
+    process.on('SIGTERM', async () => {
+        await okdb.close();
+        process.exit(0);
+    });
+}
+
+main().catch(console.error);
+```
+
+Launch it with pm2, systemd, docker, or kubernetes — run N replicas and each drains jobs via CAS
+without coordination. A replica that dies leaves its in-flight jobs to re-queue on claim expiry.
 
 ---
 
@@ -320,35 +419,6 @@ POST /api/queue/job/<id>/fail
 
 ---
 
-## Spawned workers
-
-If you want queue handling in a **separate Node process**, use `spawn(...)`:
-
-```javascript
-const spawned = await okdb.queue.spawn('send-email', './workers/email.js', {
-    concurrency: 4,
-    pollInterval: 100,
-});
-
-// later
-await spawned.stop();
-```
-
-The module must export a function:
-
-```javascript
-module.exports = async function (payload, ctx) {
-    await ctx.okdb.put('email_log', payload.orderId, {
-        sentAt: Date.now(),
-        jobId: ctx.jobId,
-    });
-};
-```
-
-The child process opens the same OKDB path and receives the same `ctx.okdb` semantics as a normal worker.
-
----
-
 ## Job lifecycle
 
 ```text
@@ -630,7 +700,7 @@ The queue-worker engine is a pipeline-compatible engine type that:
 - invokes a stored function for each claimed payload
 - handles completion/failure using normal queue semantics
 
-Worker `concurrency` controls job throughput. The function runner pool scales automatically to support it — no manual pool sizing is needed.
+Worker `concurrency` controls job throughput. Each invoked function runs in the executing node's local sandbox thread (lazy, idle-reaped) — no pool to size.
 
 ---
 

package/docs/sync.md

@@ -96,7 +96,7 @@ Returns up to `delta_limit` (default 500) change records from clock `N` onwards
 
 ### Multi-environment sync
 
-The delta spans **all `sync:true` environments** simultaneously. The response carries a `clocks` map (`{ default: N, queue: N, ... }`), and each environment's cursor advances independently. Custom environments created via `createEnvironment` are included automatically if `sync: true`.
+The delta spans **all changelog-enabled user environments** simultaneously. The response carries a `clocks` map (`{ default: N, queue: N, ... }`), and each environment's cursor advances independently. Custom environments created via `createEnvironment` are included automatically — every user env enables its changelog and participates in sync (the per-env `sync` option was removed in 2.0; internal `~`-prefixed envs never participate).
 
 ---
 
@@ -140,7 +140,11 @@ LWW depends on timestamps. If two nodes have significantly different system cloc
 :::
 
 :::note Non-syncable data
-Environments with `sync: false` (FTS indexes, vector stores) are excluded from sync. They are rebuilt locally from synced data. This is intentional — posting lists and HNSW graphs are large, computable, and node-local.
+Internal `~`-prefixed environments (FTS indexes under `~fts`, vector stores, the `~sub`
+subscriptions env) carry no changelog and are excluded from sync. They are rebuilt locally from
+synced data. This is intentional — posting lists and HNSW graphs are large, computable, and
+node-local. (In 1.9 these were opted out with `sync: false`; that option was removed in 2.0 —
+exclusion is now derived from the env being a rebuildable internal store.)
 :::
 
 ---
@@ -149,8 +153,7 @@ Environments with `sync: false` (FTS indexes, vector stores) are excluded from s
 
 `okdb-bus` sends a UDP multicast datagram on `239.1.2.3:30303` whenever the clock changes. This is how peers on the same LAN discover that new data is available without polling. Loss is acceptable — sync reconciles correctly even if some pokes are dropped.
 
-To disable:
-
-```javascript
-const okdb = new OKDB('./db', { bus: false });
-```
+The bus is **auto-enabled** when the LMDB path is shared (the shmbuf native binding present);
+it is off for a lone single-process node. The `bus` constructor option was removed in 2.0 (it
+threw nothing useful as a manual switch — it is load-bearing for POKE/DRAIN coherence and so is
+derived, not asked). See [Upgrading to 2.0](upgrade-2.0.md).

package/docs/ttl.md

@@ -7,7 +7,7 @@ OKDB supports per-record TTL — set an expiration on any item and it will be au
 ## Quick Start
 
 ```javascript
-const OKDB = require('src/okdb');
+const OKDB = require('@kedem/okdb');
 const db = new OKDB('./my-data');
 await db.open();