@loro-dev/flock-wasm 0.1.2 → 0.1.4

package/README.md

@@ -1,13 +1,1288 @@
 # @loro-dev/flock-wasm
 
-WASM bindings for `flock-rs`.
+WASM bindings for `flock-rs` — a distributed, conflict-free key-value store built on CRDT (Conflict-free Replicated Data Type) principles.
 
-## Runtime mode
+Flock enables multiple replicas to independently read and write data, merge changes without coordination, and converge deterministically to the same state. It is designed for offline-first, peer-to-peer, and decentralized applications where network partitions are expected and no central authority is required.
 
-- `RawFlock` constructors (`new`, `fromJson`, `fromFile`) use buffered mode by default.
-- Buffered mode means memtable is enabled by default.
-- Buffered writes are flushed on `txnCommit` (and storage transaction commit), or when exporting file bytes.
-- If you need strongest durability semantics, explicitly commit transactions more frequently.
+## Table of Contents
+
+- [Install](#install)
+- [Core Concepts](#core-concepts)
+- [Quick Start](#quick-start)
+- [Runtime Modes and Buffering](#runtime-modes-and-buffering)
+- [API Reference](#api-reference)
+  - [Top-Level Exports](#top-level-exports)
+  - [Flock Constructors](#flock-constructors)
+  - [Identity and Diagnostics](#identity-and-diagnostics)
+  - [Data Operations](#data-operations)
+  - [Multi-Value Register (MVR)](#multi-value-register-mvr)
+  - [Scan Operations](#scan-operations)
+  - [Replication and Import/Export](#replication-and-importexport)
+  - [Hooks API](#hooks-api)
+  - [Events and Subscriptions](#events-and-subscriptions)
+  - [Debounced Event Batching](#debounced-event-batching)
+  - [Transactions](#transactions)
+  - [Binary File Export/Import](#binary-file-exportimport)
+- [Type Reference](#type-reference)
+- [Conflict Resolution Deep Dive](#conflict-resolution-deep-dive)
+- [Version Vectors Explained](#version-vectors-explained)
+- [Best Practices](#best-practices)
+- [Common Pitfalls](#common-pitfalls)
+- [Build](#build)
+- [Test](#test)
+- [License](#license)
+
+## Install
+
+```bash
+npm add @loro-dev/flock-wasm
+# or
+pnpm add @loro-dev/flock-wasm
+# or
+yarn add @loro-dev/flock-wasm
+```
+
+## Core Concepts
+
+### CRDT with Last-Writer-Wins (LWW) Semantics
+
+Every entry in Flock carries a **clock** consisting of three components:
+
+1. **Physical time** — wall-clock timestamp in milliseconds.
+2. **Logical counter** — monotonic counter to break ties at the same physical time.
+3. **Peer ID** — unique identifier of the writer.
+
+When two replicas have conflicting values for the same key, Flock resolves deterministically:
+
+1. Higher `physicalTime` wins.
+2. On tie, higher `logicalCounter` wins.
+3. On tie, lexicographically greater `peerId` wins.
+
+All replicas applying the same rule to the same set of entries converge to the identical value — no coordination needed.
+
+### Keys and Values
+
+- **Keys** are arrays of JSON-serializable parts: `["users", 42, "name"]`. Keys are encoded into a memcomparable binary format, enabling efficient sorted storage and prefix-based range scans.
+- **Values** are any JSON-serializable type: strings, numbers, booleans, null, arrays, or nested objects.
+- **Metadata** is an optional `Record<string, unknown>` attached to each entry, useful for signatures, policy hints, or audit information.
+
+### Tombstones
+
+Deletions create **tombstone** entries rather than removing data. A tombstone carries a clock but no data. Tombstones:
+
+- Are visible in `scan()` results (with `value: undefined`).
+- Participate in conflict resolution — a delete with a higher clock wins over an older write.
+- Can be pruned during export via `pruneTombstonesBefore`.
+- Ensure that "deleted" and "never existed" are distinguishable via `getEntry()`.
+
+### Version Vectors
+
+A version vector is a compact summary of the visible state: `{ [peerId]: { physicalTime, logicalCounter } }`. Two types exist:
+
+- **Exclusive version** (`version()`): Only includes peers that currently own at least one visible entry. Use this as the baseline for incremental sync.
+- **Inclusive version** (`inclusiveVersion()`): Tracks max seen clocks across all peers ever encountered. Use this for completeness checks.
+
+## Quick Start
+
+```ts
+import { Flock } from "@loro-dev/flock-wasm";
+
+// Create two independent replicas
+const alice = new Flock("alice");
+const bob = new Flock("bob");
+
+// Alice writes some data
+alice.put(["todos", 1], { title: "Write docs", done: false });
+alice.put(["todos", 2], { title: "Ship feature", done: false });
+
+// Export Alice's state and import into Bob
+const snapshot = alice.exportJson();
+bob.importJson(snapshot);
+
+// Bob sees Alice's data
+console.log(bob.get(["todos", 1])); // { title: "Write docs", done: false }
+
+// Both edit concurrently
+alice.put(["todos", 1], { title: "Write docs", done: true });
+bob.put(["todos", 1], { title: "Write better docs", done: false });
+
+// Merge resolves conflict via LWW (higher timestamp wins)
+alice.merge(bob);
+bob.merge(alice);
+
+// Both converge to the same value
+console.log(alice.get(["todos", 1])); // same result
+console.log(bob.get(["todos", 1]));   // same result
+```
+
+## Runtime Modes and Buffering
+
+All `Flock` constructors (`new`, `fromJson`, `fromFile`) use **buffered mode** by default:
+
+- **Memtable enabled**: Writes are accumulated in an in-memory buffer.
+- **Flushed on commit**: Buffered writes are flushed when `txnCommit()` is called, when the storage transaction commits, or when exporting file bytes.
+- **Implication**: For strongest durability semantics, explicitly commit transactions or call `exportFile()` more frequently.
+
+## API Reference
+
+### Top-Level Exports
+
+```ts
+import {
+  Flock,
+  encodeVersionVector,
+  decodeVersionVector,
+  type VersionVector,
+  type VersionVectorEntry,
+  type ExportBundle,
+  type ImportReport,
+  type ScanOptions,
+  type ScanRow,
+  type ScanBound,
+  type EventBatch,
+  type Event,
+  type EventPayload,
+  type EntryInfo,
+  type EntryClock,
+  type ExportRecord,
+  type MetadataMap,
+  type Value,
+  type KeyPart,
+  type ExportPayload,
+  type ExportHookContext,
+  type ExportHooks,
+  type ImportPayload,
+  type ImportHookContext,
+  type ImportAccept,
+  type ImportSkip,
+  type ImportDecision,
+  type ImportHooks,
+  type PutPayload,
+  type PutHookContext,
+  type PutHooks,
+  type PutWithMetaOptions,
+} from "@loro-dev/flock-wasm";
+```
+
+---
+
+### `encodeVersionVector(vector: VersionVector): Uint8Array`
+
+Encodes a version vector into a compact binary representation.
+
+- **Input**: A `VersionVector` object mapping peer IDs to `{ physicalTime, logicalCounter }`.
+- **Output**: `Uint8Array` with `VEVE` magic prefix, ULEB128-encoded timestamps, and delta-compressed entries.
+- **Use case**: Persist or transmit version vectors efficiently. The binary format is deterministic — same logical content always produces identical bytes.
+- **Skips**: Entries with invalid peer IDs (non-string or >= 128 UTF-8 bytes) or non-finite timestamps are silently ignored.
+
+### `decodeVersionVector(bytes: Uint8Array): VersionVector`
+
+Decodes a binary version vector back into a `VersionVector` object.
+
+- **Input**: `Uint8Array` produced by `encodeVersionVector`.
+- **Output**: `VersionVector` object.
+- **Compatibility**: Handles both the current `VEVE`-prefixed format and the legacy format (no magic prefix) transparently.
+- **Throws**: `TypeError` if peer IDs are invalid or timestamps are non-monotonic.
+
+---
+
+### Flock Constructors
+
+#### `new Flock(peerId?: string)`
+
+Creates a new empty Flock instance.
+
+- **`peerId`** (optional): A UTF-8 string under 128 bytes identifying this peer. If omitted, a cryptographically random 64-character hex string is generated.
+- **Returns**: A new `Flock` instance in buffered mode.
+- **Throws**: `TypeError` if `peerId` is not a valid string or exceeds 127 UTF-8 bytes.
+- **Side effects**: None. The instance starts with empty state and no version history.
+
+```ts
+const named = new Flock("peer-alice");
+const anonymous = new Flock(); // random 64-char hex ID
+```
+
+#### `Flock.fromJson(bundle: ExportBundle, peerId: string): Flock`
+
+Creates a new Flock instance pre-populated with state from a JSON export bundle.
+
+- **`bundle`**: An `ExportBundle` obtained from `exportJson()` on another instance.
+- **`peerId`**: The peer ID for the new instance (required).
+- **Returns**: A new `Flock` with merged state from the bundle.
+- **Throws**: If the bundle format is invalid or import fails.
+- **Use case**: Bootstrap a new replica from a snapshot received over the network.
+
+```ts
+const snapshot = source.exportJson();
+const replica = Flock.fromJson(snapshot, "new-peer");
+```
+
+#### `Flock.fromFile(bytes: Uint8Array, peerId?: string): Flock`
+
+Creates a new Flock instance from a binary file export. This is the **fastest way to load state** — the pre-built B+Tree pages are used directly without re-parsing or re-indexing (O(1) open time vs. O(n log n) for `fromJson`).
+
+- **`bytes`**: `Uint8Array` produced by `exportFile()`.
+- **`peerId`** (optional): Peer ID for the new instance. Random if omitted.
+- **Returns**: A new `Flock` with full state restored from the binary format.
+- **Throws**: If the binary format is invalid (wrong magic bytes, corrupted pages, CRC mismatch).
+- **Use case**: Fast cold-start, restore from disk, or load a persisted checkpoint. Prefer this over `fromJson` when loading full state.
+
+```ts
+const bytes = fs.readFileSync("state.db");
+const restored = Flock.fromFile(new Uint8Array(bytes), "restored-peer");
+```
+
+---
+
+### Identity and Diagnostics
+
+#### `peerId(): string`
+
+Returns the current peer ID of this Flock instance.
+
+- **Returns**: The peer ID string (always valid UTF-8, under 128 bytes).
+- **Throws**: `TypeError` if the internal FFI returns an unexpected value (should not happen in normal operation).
+- **Note**: For anonymous instances, returns the auto-generated 64-character hex string.
+
+#### `setPeerId(peerId: string): void`
+
+Changes the peer ID for subsequent operations.
+
+- **`peerId`**: New peer ID (UTF-8, under 128 bytes).
+- **Effect**: All future `put`, `delete`, and `putWithMeta` calls will be attributed to the new peer ID.
+- **Does NOT retroactively change** previous entries — those remain attributed to the original peer.
+- **Throws**: `TypeError` if the peer ID is invalid.
+
+```ts
+flock.setPeerId("peer-a");
+flock.put(["k"], "by-a");   // attributed to "peer-a"
+flock.setPeerId("peer-b");
+flock.put(["k"], "by-b");   // attributed to "peer-b", overwrites "by-a" if clock is higher
+```
+
+#### `checkInvariants(): void`
+
+Validates the internal CRDT state for consistency.
+
+- **Effect**: Exercises `version()`, `inclusiveVersion()`, and `scan()` to check structural integrity.
+- **Throws**: If any internal invariant is violated.
+- **Use case**: Debugging and testing. Not needed in production.
+
+#### `getMaxPhysicalTime(): number`
+
+Returns the maximum physical timestamp across all entries.
+
+- **Returns**: A `number` (milliseconds). Returns `0` for an empty Flock.
+- **Use case**: Determine a safe threshold for tombstone pruning. For example, `pruneTombstonesBefore: flock.getMaxPhysicalTime() - 86400000` prunes tombstones older than 24 hours.
+- **Note**: This value only increases monotonically during the lifetime of the instance.
+
+---
+
+### Data Operations
+
+#### `put(key: KeyPart[], value: Value, now?: number): void`
+
+Stores a value at the specified key using LWW semantics.
+
+- **`key`**: Array of JSON-serializable parts (e.g., `["users", 42]`).
+- **`value`**: Any JSON-serializable value (string, number, boolean, null, array, object).
+- **`now`** (optional): Explicit physical timestamp in milliseconds. If omitted, `Date.now()` is used.
+- **Effect**: Creates or overwrites the entry at `key`. If the key already exists, the new entry wins only if its clock is higher than the existing entry's clock.
+- **Event**: Emits a change event to all subscribers (subject to debounce/transaction batching).
+- **Throws**: If the key or value is not JSON-serializable.
+
+**Important**: `put` always succeeds locally — the LWW comparison happens during merge with remote state. Locally, `put` always overwrites because the local clock is guaranteed to advance.
+
+```ts
+flock.put(["config", "theme"], "dark");
+flock.put(["scores", 1], 100, Date.now());
+flock.put(["nested"], { a: { b: [1, 2, 3] } });
+```
+
+#### `set(key: KeyPart[], value: Value, now?: number): void`
+
+Alias for `put()`. Identical behavior.
+
+#### `putWithMeta(key: KeyPart[], value: Value, options?: PutWithMetaOptions): void | Promise<void>`
+
+Stores a value with optional metadata and/or transform hooks.
+
+- **`key`**: Key array.
+- **`value`**: Value to store.
+- **`options.metadata`** (optional): A `MetadataMap` (`Record<string, unknown>`) to associate with this entry. Useful for signatures, audit trails, or policy tags.
+- **`options.now`** (optional): Explicit timestamp.
+- **`options.hooks.transform`** (optional): An async function `(context, payload) => payload | void` that can modify the data and metadata before storage. If provided, the method returns a `Promise<void>`.
+- **Returns**: `void` if no hooks; `Promise<void>` if hooks are present.
+- **Throws**: `TypeError` if the transform hook returns a payload without a `data` field.
+- **Event**: Emits a change event after the (possibly async) operation completes.
+
+```ts
+// Simple metadata
+flock.putWithMeta(["doc", 1], "content", {
+  metadata: { author: "alice", signature: "abc123" },
+});
+
+// With transform hook (async)
+await flock.putWithMeta(["doc", 1], "content", {
+  metadata: { sig: "pending" },
+  hooks: {
+    transform: async (ctx, payload) => {
+      payload.metadata = { ...payload.metadata, sig: await sign(payload.data) };
+      return payload;
+    },
+  },
+});
+```
+
+#### `delete(key: KeyPart[], now?: number): void`
+
+Deletes the value at the specified key by creating a tombstone.
+
+- **`key`**: Key array.
+- **`now`** (optional): Explicit timestamp.
+- **Effect**: Creates a tombstone entry. `get(key)` will return `undefined` after deletion.
+- **Important**: The tombstone participates in conflict resolution. A delete with a lower clock than an existing write will be ignored during merge. A delete with a higher clock will win.
+- **No-op on missing keys**: Deleting a key that has never been written is a no-op — no tombstone is created and no event is emitted. Deleting an already-deleted key is also a no-op (the existing tombstone is kept).
+- **Event**: Emits a change event with `value: undefined` when an existing entry is deleted.
+
+```ts
+flock.put(["temp"], "data");
+flock.delete(["temp"]);
+flock.get(["temp"]); // undefined
+```
+
+#### `get(key: KeyPart[]): Value | undefined`
+
+Retrieves the current value at the specified key.
+
+- **`key`**: Key array.
+- **Returns**: The stored value, or `undefined` if the key does not exist or has been deleted.
+- **Note**: Does not distinguish between "never set" and "deleted". Use `getEntry()` for that distinction.
+- **Graceful**: If the key array cannot be decoded (e.g., contains non-serializable types), returns `undefined` instead of throwing.
+
+```ts
+flock.put(["k"], 42);
+flock.get(["k"]);         // 42
+flock.get(["missing"]);   // undefined
+```
+
+#### `getEntry(key: KeyPart[]): EntryInfo | undefined`
+
+Retrieves the full entry including data, metadata, and clock information.
+
+- **`key`**: Key array.
+- **Returns**: An `EntryInfo` object `{ data?, metadata, clock }`, or `undefined` if the key has never been written.
+- **Tombstone distinction**: For deleted keys, returns an object with `data` omitted but `clock` and `metadata` present. For never-set keys, returns `undefined`.
+- **Metadata**: Always present as an object (defaults to `{}` when no metadata was stored).
+- **Use case**: Auditing (who wrote when), tombstone detection, signature verification.
+
+```ts
+flock.putWithMeta(["k"], "value", { metadata: { author: "alice" } });
+const entry = flock.getEntry(["k"]);
+// {
+//   data: "value",
+//   metadata: { author: "alice" },
+//   clock: { physicalTime: 1700000000000, logicalCounter: 0, peerId: "alice" }
+// }
+
+flock.delete(["k"]);
+const tombstone = flock.getEntry(["k"]);
+// {
+//   metadata: {},
+//   clock: { physicalTime: 1700000000001, logicalCounter: 0, peerId: "alice" }
+// }
+// Note: `data` field is absent (not `undefined` — it is omitted)
+
+flock.getEntry(["never-set"]); // undefined
+```
+
+---
+
+### Multi-Value Register (MVR)
+
+MVR is a pattern built on top of Flock's LWW key-value store that allows concurrent writes from different peers to coexist instead of one winning. It works by appending the value to the key and storing `true` as a marker.
+
+#### `putMvr(key: KeyPart[], value: Value, now?: number): void`
+
+Stores a scalar value using the MVR pattern.
+
+- **`key`**: Base key array (the MVR "slot").
+- **`value`**: Must be a **scalar** JSON value: `string`, `number`, or `boolean`. Objects, arrays, and `null` are rejected.
+- **`now`** (optional): Explicit timestamp.
+- **Effect**: Deletes all existing MVR entries under the base key (where `value === true`), regardless of which peer wrote them, then writes `true` at `[...key, value]`.
+- **Throws**: `TypeError` if `value` is null, an array, or an object.
+- **Event**: Emits delete events for removed entries and a put event for the new entry.
+
+**How it works internally**:
+
+```
+putMvr(["mv"], "one") →
+  scan prefix ["mv"] and delete all entries where value === true
+  put(["mv", "one"], true)
+```
+
+#### `getMvr(key: KeyPart[]): Value[]`
+
+Retrieves all concurrent MVR values for a base key.
+
+- **`key`**: Base key array.
+- **Returns**: Array of scalar values. Empty `[]` if no values exist.
+- **Behavior**: Scans all entries with prefix `key`, filters for entries at depth `key.length + 1` with `value === true`, and extracts the appended value part.
+
+```ts
+// Single writer
+flock.putMvr(["color"], "red");
+flock.getMvr(["color"]); // ["red"]
+
+// Concurrent writers merge
+const a = new Flock("a");
+const b = new Flock("b");
+a.putMvr(["color"], "red");
+b.putMvr(["color"], "blue");
+a.merge(b);
+a.getMvr(["color"]); // ["red", "blue"] (order may vary)
+
+// Calling putMvr replaces ALL values (not just the local peer's)
+a.putMvr(["color"], "green");
+a.getMvr(["color"]); // ["green"] — both "red" and "blue" were deleted
+```
+
+---
+
+### Scan Operations
+
+#### `scan(options?: ScanOptions): ScanRow[]`
+
+Scans entries within the specified key range and/or prefix.
+
+- **`options.start`** (optional): Start bound for the scan range.
+- **`options.end`** (optional): End bound for the scan range.
+- **`options.prefix`** (optional): Key prefix to filter by. Only entries whose key starts with this prefix are returned.
+- **Returns**: Array of `ScanRow` objects sorted by key in lexicographic order.
+- **Tombstones**: Included in results with `value: undefined`. Use `row.value === undefined` to detect them.
+- **Empty result**: Returns `[]` when no entries match.
+
+**Bound types**:
+
+```ts
+// Inclusive: includes the boundary key
+{ kind: "inclusive", key: ["b"] }
+
+// Exclusive: excludes the boundary key
+{ kind: "exclusive", key: ["d"] }
+
+// Unbounded: no limit in this direction
+{ kind: "unbounded" }
+```
+
+**Examples**:
+
+```ts
+// Full scan (all entries including tombstones)
+const all = flock.scan();
+
+// Prefix scan
+const users = flock.scan({ prefix: ["users"] });
+
+// Range scan with bounds
+const range = flock.scan({
+  start: { kind: "inclusive", key: ["b"] },
+  end: { kind: "exclusive", key: ["d"] },
+});
+
+// Combine prefix and bounds
+const page = flock.scan({
+  prefix: ["users"],
+  start: { kind: "exclusive", key: ["users", 100] },
+  end: { kind: "inclusive", key: ["users", 200] },
+});
+```
+
+**ScanRow structure**:
+
+```ts
+{
+  key: KeyPart[];      // The full key
+  raw: ExportRecord;   // Raw record with clock string and optional data/metadata
+  value?: Value;       // The data value; undefined for tombstones
+}
+```
+
+---
+
+### Replication and Import/Export
+
+#### `version(): VersionVector`
+
+Returns the **exclusive** (visible) version vector.
+
+- **Returns**: `VersionVector` — only includes peers that currently own at least one visible entry (not overwritten by a later entry from another peer).
+- **Use case**: Pass this to a remote peer as the `from` parameter of `exportJson()` to get only the updates the local replica is missing.
+- **Behavior after overwrites**: If peer A's only entry is overwritten by peer B, peer A is removed from the exclusive version.
+
+```ts
+flock.setPeerId("a");
+flock.put(["k"], "v1");
+flock.version(); // { a: { physicalTime: ..., logicalCounter: ... } }
+
+flock.setPeerId("b");
+flock.put(["k"], "v2"); // overwrites peer a's entry
+flock.version(); // { b: { ... } } — peer a is gone
+```
+
+#### `inclusiveVersion(): VersionVector`
+
+Returns the **inclusive** (max-seen) version vector.
+
+- **Returns**: `VersionVector` — tracks the highest clock seen from every peer during this instance's lifetime, including peers whose entries have been overwritten.
+- **Use case**: Completeness checks, debugging, or determining whether a remote update has been "seen" even if it was subsequently overwritten.
+- **Guarantee**: Always a superset of `version()`.
+
+```ts
+flock.setPeerId("a");
+flock.put(["k"], "v1");
+flock.setPeerId("b");
+flock.put(["k"], "v2");
+
+flock.version();          // { b: { ... } }
+flock.inclusiveVersion(); // { a: { ... }, b: { ... } }
+```
+
+#### `merge(other: Flock): void`
+
+Merges all state from another Flock instance into this one.
+
+- **`other`**: The source Flock instance.
+- **Effect**: Applies all entries from `other` into `this` using LWW conflict resolution. After merge, `this` contains the union of both states.
+- **Idempotent**: Calling `merge(other)` multiple times produces the same result as calling it once.
+- **Events**: Emits an event batch for imported entries.
+- **Note**: This is a one-directional merge. To fully sync two replicas, call `a.merge(b)` and `b.merge(a)`.
+
+#### `exportJson(): ExportBundle`
+
+Exports the full state as a JSON bundle.
+
+- **Returns**: `ExportBundle` containing all entries (including tombstones), keyed by stringified key arrays.
+- **Format**: `{ version: number, entries: { [key: string]: ExportRecord } }`.
+- **Use case**: Full snapshot for backup, initial sync, or cold storage.
+
+#### `exportJson(from: VersionVector): ExportBundle`
+
+Exports only entries newer than the given version vector (incremental/delta export).
+
+- **`from`**: The remote peer's current version vector (obtained via `version()`).
+- **Returns**: Bundle containing only entries the remote peer hasn't seen.
+- **Use case**: Bandwidth-efficient incremental sync. Send your `version()` to the remote, get back only what's new.
+
+```ts
+// Incremental sync pattern
+const remoteVersion = remotePeer.version();
+const delta = localPeer.exportJson(remoteVersion);
+remotePeer.importJson(delta);
+```
+
+#### `exportJson(from: VersionVector, pruneTombstonesBefore: number): ExportBundle`
+
+Exports with tombstone pruning.
+
+- **`pruneTombstonesBefore`**: Millisecond timestamp. Tombstones with `physicalTime` older than this value are excluded from the export.
+- **Use case**: Reduce export size by omitting old tombstones that all replicas have already seen.
+- **Caution**: If a replica imports a pruned export, it may not know about deletions that happened before the prune threshold. Only use when you're confident all replicas have already processed those deletions.
+
+#### `exportJson(options: ExportOptions): Promise<ExportBundle>`
+
+Exports with hooks and/or additional options (async).
+
+- **`options.from`** (optional): Version vector for incremental export.
+- **`options.pruneTombstonesBefore`** (optional): Tombstone prune timestamp.
+- **`options.peerId`** (optional): Export only entries written by this peer.
+- **`options.hooks.transform`** (optional): Async function to transform each entry during export.
+- **Returns**: `Promise<ExportBundle>`.
+- **Use case**: Sign entries, redact sensitive data, or filter entries during export.
+
+```ts
+const signed = await flock.exportJson({
+  hooks: {
+    transform: async (ctx, payload) => {
+      return {
+        ...payload,
+        metadata: { ...payload.metadata, sig: await sign(payload.data) },
+      };
+    },
+  },
+});
+```
+
+#### `importJson(bundle: ExportBundle): ImportReport`
+
+Imports a JSON bundle, merging entries using LWW semantics.
+
+- **`bundle`**: An `ExportBundle` from `exportJson()`.
+- **Returns**: `ImportReport` with `{ accepted: number, skipped: Array<{ key, reason }> }`.
+- **`accepted` counts records processed**, not records that changed state. Re-importing the same bundle may still report a positive `accepted` count even though no state changed and no events are emitted. Do not use `accepted` as a reliable indicator of whether new data was applied — use event listeners for that.
+- **Events**: Emits event batches for newly accepted entries.
+- **Effect on debounce**: If `autoDebounceCommit` is active, any pending local events are flushed before the import events are delivered.
+
+#### `importJson(options: ImportOptions): Promise<ImportReport>`
+
+Imports with preprocessing hooks (async).
+
+- **`options.bundle`**: The export bundle.
+- **`options.hooks.preprocess`**: Async function `(context, payload) => ImportDecision` to validate each entry before import.
+- **Returns**: `Promise<ImportReport>`.
+- **Use case**: Validate signatures, check authorization, or reject untrusted entries.
+
+```ts
+const report = await flock.importJson({
+  bundle,
+  hooks: {
+    preprocess: async (ctx, payload) => {
+      if (!await verify(payload.metadata?.sig, payload.data)) {
+        return { accept: false, reason: "invalid signature" };
+      }
+      return { accept: true };
+    },
+  },
+});
+console.log(report.skipped); // entries rejected by the hook
+```
+
+#### `importJsonStr(bundle: string): ImportReport`
+
+Convenience method that parses a JSON string and imports it.
+
+- **`bundle`**: JSON string representation of an `ExportBundle`.
+- **Returns**: `ImportReport`.
+- **Use case**: Import directly from a network response or file read without manual parsing.
+
+#### `kvToJson(): ExportBundle`
+
+Alias for `exportJson()` with no arguments. Returns the full state as a JSON bundle.
+
+---
+
+### Hooks API
+
+Hooks provide extension points for signing, validating, transforming, or filtering data during put, export, and import operations.
+
+#### Put Hooks (`PutWithMetaOptions`)
+
+```ts
+type PutWithMetaOptions = {
+  metadata?: MetadataMap;
+  now?: number;
+  hooks?: {
+    transform?: (context: PutHookContext, payload: PutPayload) => MaybePromise<PutPayload | void>;
+  };
+};
+```
+
+- **`context`**: `{ key: KeyPart[], now?: number }`.
+- **`payload`**: `{ data?: Value, metadata?: MetadataMap }` — a working copy.
+- **Behavior**: The hook receives a cloned payload. It can mutate it in-place or return a new payload. If it returns `void`, the (possibly mutated) working copy is used.
+- **Constraint**: The final payload must have a `data` field; omitting it throws `TypeError`.
+
+#### Export Hooks (`ExportOptions`)
+
+```ts
+type ExportOptions = {
+  from?: VersionVector;
+  pruneTombstonesBefore?: number;
+  peerId?: string;
+  hooks?: {
+    transform?: (context: ExportHookContext, payload: ExportPayload) => MaybePromise<ExportPayload | void>;
+  };
+};
+```
+
+- **`context`**: `{ key: KeyPart[], clock: EntryClock, raw: ExportRecord }`.
+- **`payload`**: `{ data?: Value, metadata?: MetadataMap }` — a working copy.
+- **Behavior**: Called once per entry in the export. Does not modify stored state; only affects the exported bundle.
+- **Use case**: Encrypt data, redact PII, or append signatures before sharing.
+
+#### Import Hooks (`ImportOptions`)
+
+```ts
+type ImportOptions = {
+  bundle: ExportBundle;
+  hooks?: {
+    preprocess?: (context: ImportHookContext, payload: ImportPayload) => MaybePromise<ImportDecision>;
+  };
+};
+```
+
+- **`context`**: `{ key: KeyPart[], clock: EntryClock, raw: ExportRecord }`.
+- **`payload`**: `{ data?: Value, metadata?: MetadataMap }` — a cloned copy for inspection (not mutation).
+- **`ImportDecision`**: Return `{ accept: true }`, `{ accept: false, reason: string }`, or `void` (treated as accept).
+- **Behavior**: Called once per entry before import. Rejected entries are excluded from the merge and reported in `ImportReport.skipped`.
+- **Use case**: Validate signatures, enforce ACLs, or reject entries from untrusted peers.
+
+---
+
+### Events and Subscriptions
+
+#### `subscribe(listener: (batch: EventBatch) => void): () => void`
+
+Subscribes to change events.
+
+- **`listener`**: Callback receiving `EventBatch` objects.
+- **Returns**: An unsubscribe function. Call it to stop receiving events.
+- **Event delivery**: Events are delivered synchronously after each state-changing operation (unless debounce or transactions are active).
+- **Re-entrancy safe**: You can call `put`, `delete`, `get`, `merge`, or `importJson` from within a listener. New events from re-entrant operations are queued and delivered after the current batch completes.
+- **Error isolation**: If a listener throws, other listeners still receive the event. The error does not propagate to the caller of the operation that triggered the event.
+- **Automatic cleanup**: When all listeners are removed, the native WASM subscription is cleaned up.
+
+**EventBatch structure**:
+
+```ts
+{
+  source: string;   // "local" for put/delete/set, or peer identifier for import/merge
+  events: Event[];  // Array of individual change events
+}
+```
+
+**Event structure**:
+
+```ts
+{
+  key: KeyPart[];          // The affected key
+  value?: Value;           // New value; undefined for deletions
+  metadata?: MetadataMap;  // Associated metadata
+  payload: EventPayload;   // { data?: Value, metadata?: MetadataMap }
+}
+```
+
+```ts
+const unsub = flock.subscribe((batch) => {
+  for (const event of batch.events) {
+    console.log(`${batch.source}: ${event.key} → ${event.value}`);
+  }
+});
+
+flock.put(["x"], 1); // listener fires: source="local", key=["x"], value=1
+unsub(); // stop listening
+```
+
+---
+
+### Debounced Event Batching
+
+Debouncing accumulates local events and delivers them as a single batch after a period of inactivity, reducing event noise during rapid-fire edits.
+
+#### `autoDebounceCommit(timeout: number, options?: { maxDebounceTime?: number }): void`
+
+Enables debounced event batching.
+
+- **`timeout`**: Debounce window in milliseconds. Events are held until `timeout` ms of inactivity pass.
+- **`options.maxDebounceTime`** (default: `10000`): Maximum time in ms that events can be held, even if operations keep coming. This prevents unbounded delays during sustained writes.
+- **Effect**: Local `put`/`delete`/`set` events are buffered instead of immediately delivered. Import and merge events are NOT debounced — they flush pending local events first, then deliver immediately.
+- **Throws**: If a transaction is currently active, or if debounce is already enabled.
+
+```ts
+flock.autoDebounceCommit(100, { maxDebounceTime: 5000 });
+flock.put(["a"], 1); // buffered
+flock.put(["b"], 2); // buffered, timer reset
+// After 100ms of inactivity → single batch with both events
+// OR after 5000ms even if operations keep coming
+```
+
+#### `commit(): void`
+
+Forces immediate emission of any pending debounced events.
+
+- **Effect**: Flushes the debounce buffer and delivers all pending events as a batch.
+- **Does NOT disable** debounce mode — subsequent operations continue to be debounced.
+- **No-op**: If debounce is not active or no events are pending.
+
+#### `disableAutoDebounceCommit(): void`
+
+Disables debounce mode and flushes any pending events.
+
+- **Effect**: Pending events are delivered immediately. Future operations emit events synchronously.
+- **No-op**: If debounce is not active.
+
+#### `isAutoDebounceActive(): boolean`
+
+Returns whether debounce mode is currently enabled.
+
+---
+
+### Transactions
+
+Transactions batch multiple operations into a single atomic event delivery. All put/delete operations inside a transaction are delivered as one `EventBatch` on commit.
+
+#### `txn<T>(callback: () => T): T`
+
+Executes a synchronous callback within a transaction.
+
+- **`callback`**: Synchronous function containing put/delete operations.
+- **Returns**: The return value of the callback.
+- **Event batching**: All events from operations inside the callback are collected and delivered as a single `EventBatch` when the transaction commits.
+- **Error handling**: If the callback throws, the transaction is rolled back (event emission is canceled) and the error re-thrown. **Important caveat**: Data changes (puts/deletes) may NOT be rolled back — only event emission is affected. The underlying writes may have already been applied to the store.
+- **Throws**: `Error` if called while `autoDebounceCommit` is active (mutual exclusion).
+- **Throws**: `Error` if nested transactions are attempted.
+- **Constraint**: The callback must be synchronous. Async operations inside `txn()` are not supported.
+
+```ts
+flock.subscribe((batch) => {
+  console.log(`Received ${batch.events.length} events`);
+});
+
+flock.txn(() => {
+  flock.put(["a"], 1);
+  flock.put(["b"], 2);
+  flock.put(["c"], 3);
+});
+// Listener receives: "Received 3 events" — single batch
+```
+
+#### `isInTxn(): boolean`
+
+Returns whether a transaction is currently active.
+
+---
+
+### Binary File Export/Import
+
+The binary file format is Flock's most efficient serialization. Unlike JSON export (which requires parsing every entry and re-inserting them one by one into a B+Tree), the binary format stores pre-built B+Tree pages directly. This makes **loading from binary dramatically faster** — especially for large datasets.
+
+| Operation | `fromJson` (JSON bundle) | `fromFile` (binary) |
+|-----------|--------------------------|---------------------|
+| Open cost | O(n log n) — parse JSON, encode each key, insert into B+Tree one by one | O(1) — read file header, done |
+| Size | Larger (JSON text, repeated key strings, base-10 numbers) | Smaller (binary-encoded keys, prefix-compressed leaves, compact integers) |
+| Use case | Network sync, incremental delta exchange | Persistence, checkpoints, full-state transfer |
+
+**Rule of thumb**: Use `exportJson`/`importJson` for replication and sync. Use `exportFile`/`fromFile` for persistence and fast cold-start loading.
+
+#### `exportFile(): Uint8Array`
+
+Exports the entire Flock state as a binary file.
+
+- **Returns**: `Uint8Array` containing the complete state in Flock's B+Tree page format.
+- **Format**: Binary format with `FLOK` magic header, B+Tree pages with prefix-compressed keys, and optional WAL data. Significantly more compact than JSON export.
+- **Performance**: The exported bytes contain ready-to-use B+Tree pages. When loaded via `Flock.fromFile()`, the pages are used directly without any re-indexing or re-encoding — opening is O(1) regardless of data size.
+- **Use case**: Persist state to disk, transfer as a file, create compact backups, or achieve the fastest possible cold-start.
+- **Side effect**: Flushes any pending memtable writes before exporting.
+
+```ts
+const bytes = flock.exportFile();
+fs.writeFileSync("state.flock", bytes);
+```
+
+Restoration is done via `Flock.fromFile()`:
+
+```ts
+const bytes = fs.readFileSync("state.flock");
+const flock = Flock.fromFile(new Uint8Array(bytes), "my-peer");
+// Ready instantly — no re-parsing or re-indexing needed
+```
+
+---
+
+## Type Reference
+
+### Version Vector Types
+
+```ts
+type VersionVectorEntry = {
+  physicalTime: number;    // Wall-clock timestamp in milliseconds
+  logicalCounter: number;  // Monotonic tie-breaking counter
+};
+
+interface VersionVector {
+  [peer: string]: VersionVectorEntry | undefined;
+}
+```
+
+### Data Types
+
+```ts
+// Any JSON-serializable value
+type Value = string | number | boolean | null | Array<Value> | { [key: string]: Value };
+
+// A single part of a compound key
+type KeyPart = Value;
+
+// Arbitrary metadata stored alongside an entry
+type MetadataMap = Record<string, unknown>;
+```
+
+### Entry Types
+
+```ts
+type EntryClock = {
+  physicalTime: number;    // When the entry was written
+  logicalCounter: number;  // Tie-breaking counter
+  peerId: string;          // Who wrote the entry
+};
+
+type EntryInfo = {
+  data?: Value;            // Omitted for tombstones
+  metadata: MetadataMap;   // Always present (defaults to {})
+  clock: EntryClock;       // Clock of the winning write
+};
+```
+
+### Export/Import Types
+
+```ts
+type ExportRecord = {
+  c: string;               // Clock string: "physicalTime,logicalCounter,peerId"
+  d?: Value;               // Data value (omitted for tombstones)
+  m?: MetadataMap;         // Metadata (omitted if empty)
+};
+
+type ExportBundle = {
+  version: number;         // Format version number
+  entries: Record<string, ExportRecord>;  // Key (stringified) → record
+};
+
+type ImportReport = {
+  accepted: number;                              // Number of entries processed (not necessarily changed)
+  skipped: Array<{ key: KeyPart[]; reason: string }>;  // Rejected entries with reasons
+};
+```
+
+### Hook Types
+
+```ts
+type ExportPayload = { data?: Value; metadata?: MetadataMap };
+type ExportHookContext = { key: KeyPart[]; clock: EntryClock; raw: ExportRecord };
+type ExportHooks = {
+  transform?: (context: ExportHookContext, payload: ExportPayload) => MaybePromise<ExportPayload | void>;
+};
+
+type ImportPayload = ExportPayload;
+type ImportHookContext = ExportHookContext;
+type ImportAccept = { accept: true };
+type ImportSkip = { accept: false; reason: string };
+type ImportDecision = ImportAccept | ImportSkip | ImportPayload | void;
+type ImportHooks = {
+  preprocess?: (context: ImportHookContext, payload: ImportPayload) => MaybePromise<ImportDecision>;
+};
+
+type PutPayload = ExportPayload;
+type PutHookContext = { key: KeyPart[]; now?: number };
+type PutHooks = {
+  transform?: (context: PutHookContext, payload: PutPayload) => MaybePromise<PutPayload | void>;
+};
+
+type PutWithMetaOptions = {
+  metadata?: MetadataMap;
+  now?: number;
+  hooks?: PutHooks;
+};
+```
+
+### Scan Types
+
+```ts
+type ScanBound =
+  | { kind: "inclusive"; key: KeyPart[] }
+  | { kind: "exclusive"; key: KeyPart[] }
+  | { kind: "unbounded" };
+
+type ScanOptions = {
+  start?: ScanBound;
+  end?: ScanBound;
+  prefix?: KeyPart[];
+};
+
+type ScanRow = {
+  key: KeyPart[];
+  raw: ExportRecord;
+  value?: Value;
+};
+```
+
+### Event Types
+
+```ts
+type EventPayload = { data?: Value; metadata?: MetadataMap };
+
+type Event = {
+  key: KeyPart[];
+  value?: Value;
+  metadata?: MetadataMap;
+  payload: EventPayload;
+};
+
+type EventBatch = {
+  source: string;     // "local" or peer identifier
+  events: Event[];
+};
+```
+
+---
+
+## Conflict Resolution Deep Dive
+
+### How LWW Works in Practice
+
+```ts
+// Both peers start with the same state
+const a = new Flock("peer-a");
+const b = new Flock("peer-b");
+
+// Concurrent writes at the same physical time
+a.put(["key"], "alice-value", 1000);
+b.put(["key"], "bob-value", 1000);
+
+// After merge, both converge
+a.merge(b);
+b.merge(a);
+
+// Who wins? Clock comparison:
+// 1. physicalTime: both 1000 → tie
+// 2. logicalCounter: both 0 → tie
+// 3. peerId: "peer-b" > "peer-a" lexicographically → peer-b wins
+console.log(a.get(["key"])); // "bob-value"
+console.log(b.get(["key"])); // "bob-value"
+```
+
+### Delete vs. Write Conflicts
+
+```ts
+// A writes, B deletes at a later time
+a.put(["key"], "value", 1000);
+b.delete(["key"], 2000);
+
+a.merge(b);
+// Delete wins because physicalTime 2000 > 1000
+a.get(["key"]); // undefined
+
+// A writes again at an even later time
+a.put(["key"], "new-value", 3000);
+a.get(["key"]); // "new-value" — write wins because 3000 > 2000
+```
+
+### Version Vector After Overwrites
+
+When a peer's entries are fully overwritten by another peer, the exclusive version vector drops the original peer:
+
+```ts
+const f = new Flock("a");
+f.put(["k1"], "v1");     // version: { a: {...} }
+f.setPeerId("b");
+f.put(["k1"], "v2");     // version: { b: {...} } — a is dropped
+
+// But inclusive version retains both:
+f.inclusiveVersion();     // { a: {...}, b: {...} }
+```
+
+---
+
+## Version Vectors Explained
+
+### Incremental Sync Protocol
+
+```ts
+// Step 1: Bob sends his version to Alice
+const bobVersion = bob.version();
+
+// Step 2: Alice exports only what Bob hasn't seen
+const delta = alice.exportJson(bobVersion);
+
+// Step 3: Bob imports the delta
+const report = bob.importJson(delta);
+console.log(`Imported ${report.accepted} new entries`);
+
+// Step 4: Reverse direction
+const aliceVersion = alice.version();
+const reverseDelta = bob.exportJson(aliceVersion);
+alice.importJson(reverseDelta);
+
+// Both are now fully synced
+```
+
+### When to Use Which Version
+
+| Method              | Includes overwritten peers | Use for                          |
+|---------------------|---------------------------|----------------------------------|
+| `version()`         | No                        | Incremental sync, delta export   |
+| `inclusiveVersion()`| Yes                       | Completeness checks, debugging   |
+
+---
+
+## Best Practices
+
+### 1. Use Explicit Timestamps for Deterministic Testing
+
+```ts
+// In tests, always provide explicit timestamps for reproducible results
+flock.put(["k"], "v", 1000);
+
+// In production, omit the timestamp to use Date.now()
+flock.put(["k"], "v");
+```
+
+### 2. Use Transactions for Batch Updates
+
+```ts
+// Bad: subscribers receive 100 separate event batches
+for (let i = 0; i < 100; i++) {
+  flock.put(["item", i], data[i]);
+}
+
+// Good: subscribers receive one batch with 100 events
+flock.txn(() => {
+  for (let i = 0; i < 100; i++) {
+    flock.put(["item", i], data[i]);
+  }
+});
+```
+
+### 3. Use Debounce for Interactive Editing
+
+```ts
+// Accumulate rapid keystrokes into periodic batches
+flock.autoDebounceCommit(200, { maxDebounceTime: 2000 });
+
+// On each keystroke
+flock.put(["doc", "content"], currentContent);
+
+// Listeners receive updates at most every 200ms,
+// or at least every 2000ms during sustained typing
+```
+
+### 4. Prune Tombstones Periodically
+
+```ts
+// When syncing, prune tombstones older than 7 days
+const sevenDaysAgo = Date.now() - 7 * 24 * 60 * 60 * 1000;
+const bundle = flock.exportJson(
+  remoteVersion,
+  sevenDaysAgo,
+);
+
+// Only safe when you know all replicas have synced within 7 days
+```
+
+### 5. Use Metadata for Audit Trails
+
+```ts
+flock.putWithMeta(["transaction", txId], amount, {
+  metadata: {
+    author: currentUser,
+    sig: await sign(amount),
+    timestamp: Date.now(),
+  },
+});
+
+// Verify on import
+const report = await flock.importJson({
+  bundle,
+  hooks: {
+    preprocess: async (ctx, payload) => {
+      if (!await verify(payload.metadata?.sig, payload.data)) {
+        return { accept: false, reason: "signature verification failed" };
+      }
+    },
+  },
+});
+```
+
+### 6. Use Incremental Sync for Bandwidth Efficiency
+
+```ts
+// Don't do this (sends everything every time):
+const full = flock.exportJson();
+remotePeer.importJson(full);
+
+// Do this instead:
+const delta = flock.exportJson(remotePeer.version());
+remotePeer.importJson(delta);
+```
+
+### 7. Design Keys for Scan Efficiency
+
+```ts
+// Good: hierarchical keys enable prefix scans
+flock.put(["users", userId, "name"], "Alice");
+flock.put(["users", userId, "email"], "alice@example.com");
+const userFields = flock.scan({ prefix: ["users", userId] });
+
+// Bad: flat keys require full scans
+flock.put(["user_name_" + userId], "Alice");
+flock.put(["user_email_" + userId], "alice@example.com");
+```
+
+---
+
+## Common Pitfalls
+
+### 1. Debounce and Transactions Are Mutually Exclusive
+
+```ts
+flock.autoDebounceCommit(100);
+flock.txn(() => { /* ... */ }); // Throws Error!
+
+// Disable debounce first, or don't use both at the same time
+```
+
+### 2. Transaction Rollback Does NOT Undo Data Changes
+
+```ts
+flock.txn(() => {
+  flock.put(["k"], "value");
+  throw new Error("oops");
+});
+// Error is re-thrown, events are NOT emitted
+// But flock.get(["k"]) may still return "value" — data persists
+```
+
+### 3. MVR Only Accepts Scalar Values
+
+```ts
+flock.putMvr(["k"], "string");  // OK
+flock.putMvr(["k"], 42);        // OK
+flock.putMvr(["k"], true);      // OK
+flock.putMvr(["k"], null);      // Throws TypeError
+flock.putMvr(["k"], [1, 2]);    // Throws TypeError
+flock.putMvr(["k"], { a: 1 }); // Throws TypeError
+```
+
+### 4. merge() Is One-Directional
+
+```ts
+a.merge(b); // a now has everything from b
+// But b does NOT have a's changes — you need:
+b.merge(a);
+```
+
+### 5. Tombstones Appear in Scan Results
+
+```ts
+flock.put(["k"], "value");
+flock.delete(["k"]);
+const rows = flock.scan();
+// rows includes { key: ["k"], value: undefined, raw: {...} }
+// Filter tombstones if you only want live data:
+const live = rows.filter(row => row.value !== undefined);
+```
+
+### 6. Import Flushes Debounced Events First
+
+```ts
+flock.autoDebounceCommit(5000);
+flock.put(["local"], 1);       // Buffered, not emitted yet
+
+flock.importJson(remoteDelta); // Forces flush of local batch FIRST,
+                                // then delivers import batch
+// Result: listener receives 2 batches in sequence
+```
+
+### 7. Peer ID Changes Don't Affect Past Entries
+
+```ts
+flock.setPeerId("a");
+flock.put(["k"], "by-a");
+flock.setPeerId("b");
+// The entry at ["k"] is still attributed to peer "a"
+// Only new operations will use peer "b"
+```
+
+---
 
 ## Build