@optimystic/db-p2p-storage-fs 0.14.1 → 0.16.3

package/README.md

@@ -0,0 +1,118 @@
+# @optimystic/db-p2p-storage-fs
+
+Node.js filesystem storage backend for Optimystic peers. Provides:
+
+- **`FileRawStorage`** — implements `IRawStorage` so a Node peer persists block
+  metadata, revisions, pending transactions, committed transactions, and
+  materialized blocks durably across restarts.
+- **`FileKVStore`** — implements `IKVStore` for the persistent transaction state
+  used to recover crashed two-phase commits.
+
+This package targets plain Node.js. Use the sibling adapter for other
+environments:
+
+- `@optimystic/db-p2p-storage-ns` — NativeScript (iOS/Android, SQLite)
+- `@optimystic/db-p2p-storage-rn` — React Native (MMKV)
+- `@optimystic/db-p2p-storage-web` — Browser (IndexedDB)
+
+## Install
+
+```bash
+yarn add @optimystic/db-p2p-storage-fs @optimystic/db-p2p @optimystic/db-core
+```
+
+## Usage
+
+```ts
+import { FileRawStorage, FileKVStore } from '@optimystic/db-p2p-storage-fs';
+import { createLibp2pNode } from '@optimystic/db-p2p';
+
+const rawStorage = new FileRawStorage('/var/lib/my-peer/data');
+const kvStore    = new FileKVStore('/var/lib/my-peer/data');
+
+const libp2p = await createLibp2pNode({
+  bootstrapNodes: [/* … */],
+  networkName: 'my-network',
+  rawStorage,
+  kvStore,
+});
+```
+
+Both constructors take a single `basePath` — the directory under which all
+data is stored. They can share the same `basePath` in practice because their
+top-level names do not collide: block data lives under `<blockId>/`
+subdirectories (block ids are content-address hashes), while KV data lives
+under directories named by the first segment of each key (a `/`-separated key
+becomes nested subdirectories — e.g. `coordinator/key1` → `coordinator/key1.json`).
+Sharing is safe only as long as no block id equals a KV key's first segment;
+give them separate `basePath`s if you cannot guarantee that.
+
+## On-disk layout
+
+```
+<basePath>/
+  <blockId>/
+    meta.json                — BlockMetadata (JSON)
+    revs/<rev>.json          — ActionId for that revision number
+    pend/<id>.json           — pending Transform (two-phase commit, before promotion)
+    actions/<id>.json        — committed Transform
+    blocks/<id>.json         — materialized IBlock snapshot
+  <key-segment>/
+    <key-segment>.json       — FileKVStore value; key "/" separators become subdirectories
+```
+
+Action ids that contain a colon (e.g. `tx:abcd1234`) are percent-encoded in
+filenames (`tx%3Aabcd1234.json`) for Windows compatibility; the storage layer
+encodes and decodes transparently so callers always work with the canonical
+id form.
+
+## Atomic writes
+
+Every write goes through `atomic-write.ts`: the new content is written to a
+`.tmp` sibling, then renamed over the canonical path. A crash mid-write
+therefore leaves either the complete old file or the complete new file — never
+a partial/torn one.
+
+## Identity
+
+Unlike the `ns`/`rn`/`web` adapters, this package ships **no** identity module
+and no `loadOrCreateFSPeerKey` helper. The reference peer that uses this adapter
+does not persist its libp2p private key, so an fs-backed node gets a fresh
+ephemeral peer id on every restart. Adding durable identity is a separate
+feature — see Known limitations below.
+
+## Tests
+
+```bash
+yarn test          # mocha, minimal reporter
+yarn test:verbose  # mocha, spec reporter
+```
+
+Two layers run here, both over a per-test `fs.mkdtemp` fixture with `afterEach`
+cleanup:
+
+- **Shared conformance suite** — `runRawStorageConformance('FileSystem', …)` from
+  `@optimystic/db-p2p/testing`, the one cross-backend parity target. It proves the
+  fs backend (now `KvRawStorage` over a `FileStoreDriver`) behaves identically to
+  every other backend: round-trips, `listRevisions` ordering, promote atomicity +
+  the exact missing-pend error, clone-on-store/read (structural via the byte
+  boundary), drain-before-yield iteration, and a `BlockStorage` parity slice.
+- **fs-only tests** (`node:assert`) — the behaviors the shared suite can't cover:
+  atomic writes + torn-file corruption tolerance, `readdir` error discrimination,
+  colon-encoded filenames + the POSIX legacy raw-colon fallback, the directory-based
+  `listBlockIds` meta-gate, and `FileKVStore.list`/`delete`.
+
+## Known limitations
+
+**No cross-process lock.** Two separate Node processes pointing at the same
+`basePath` can interleave writes with no coordination. The constructor carries
+a TODO (`file-storage.ts:52`) to integrate
+[`proper-lockfile`](https://www.npmjs.com/package/proper-lockfile) along with
+an explicit `dispose()` pattern. Until that lands, `FileRawStorage` is
+single-process only.
+
+**Ephemeral peer identity.** There is no `loadOrCreateFSPeerKey` equivalent.
+A Node peer backed by `FileRawStorage` gets a new libp2p peer id each restart.
+If your use case requires a stable, restart-surviving identity, a future
+`feat-fs-peer-identity` feature should add the key-persistence helper — it is
+deliberately out of scope here.

package/dist/src/atomic-write.d.ts

@@ -0,0 +1,22 @@
+/**
+ * Atomically write `content` to `filePath`.
+ *
+ * Writes to a unique `*.tmp` sibling, fsyncs the data, then renames it into
+ * place. `rename` over an existing file is atomic on POSIX and NTFS, so a
+ * concurrent reader only ever sees the complete old file or the complete new
+ * file — never a torn/half-written one. After the rename we best-effort fsync
+ * the containing directory so the rename itself survives power loss on POSIX;
+ * that directory fsync is unsupported on win32 and its error is swallowed.
+ *
+ * On any failure the temp file is removed (best-effort) so a crashed write does
+ * not leave the canonical path damaged. A crash *between* the temp write and the
+ * rename leaves only an inert `*.tmp` sibling — never read (reads target
+ * canonical paths) and skipped by `.json` directory scans.
+ *
+ * `content` is `string | Uint8Array`: `FileKVStore` writes strings, while the
+ * `KvRawStorage`-backed `FileStoreDriver` writes the kernel's raw value bytes.
+ * `FileHandle.writeFile` writes either losslessly — a `Uint8Array` is written
+ * byte-for-byte, so non-ASCII JSON round-trips exactly.
+ */
+export declare function atomicWriteFile(filePath: string, content: string | Uint8Array): Promise<void>;
+//# sourceMappingURL=atomic-write.d.ts.map

package/dist/src/atomic-write.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"atomic-write.d.ts","sourceRoot":"","sources":["../../src/atomic-write.ts"],"names":[],"mappings":"AAUA;;;;;;;;;;;;;;;;;;;GAmBG;AACH,wBAAsB,eAAe,CAAC,QAAQ,EAAE,MAAM,EAAE,OAAO,EAAE,MAAM,GAAG,UAAU,GAAG,OAAO,CAAC,IAAI,CAAC,CAkCnG"}

package/dist/src/atomic-write.js

@@ -0,0 +1,82 @@
+import { promises as fs } from 'fs';
+import * as path from 'path';
+// Per-process monotonic counter for unique temp names. Combined with the pid it
+// makes each temp path unique across concurrent writers to the same logical
+// target — even two FileRawStorage/FileKVStore instances in one process, which
+// do not lock (see the proper-lockfile TODO in file-storage.ts). Deterministic
+// and collision-safe, unlike Math.random()/Date.now().
+let tempCounter = 0;
+/**
+ * Atomically write `content` to `filePath`.
+ *
+ * Writes to a unique `*.tmp` sibling, fsyncs the data, then renames it into
+ * place. `rename` over an existing file is atomic on POSIX and NTFS, so a
+ * concurrent reader only ever sees the complete old file or the complete new
+ * file — never a torn/half-written one. After the rename we best-effort fsync
+ * the containing directory so the rename itself survives power loss on POSIX;
+ * that directory fsync is unsupported on win32 and its error is swallowed.
+ *
+ * On any failure the temp file is removed (best-effort) so a crashed write does
+ * not leave the canonical path damaged. A crash *between* the temp write and the
+ * rename leaves only an inert `*.tmp` sibling — never read (reads target
+ * canonical paths) and skipped by `.json` directory scans.
+ *
+ * `content` is `string | Uint8Array`: `FileKVStore` writes strings, while the
+ * `KvRawStorage`-backed `FileStoreDriver` writes the kernel's raw value bytes.
+ * `FileHandle.writeFile` writes either losslessly — a `Uint8Array` is written
+ * byte-for-byte, so non-ASCII JSON round-trips exactly.
+ */
+export async function atomicWriteFile(filePath, content) {
+    const dir = path.dirname(filePath);
+    await fs.mkdir(dir, { recursive: true });
+    // Suffix ends in `.tmp` (not `.json`) so listPendingTransactions / FileKVStore.list,
+    // which filter on `.json`, never surface an in-flight temp file.
+    // NOTE: a crash between open and rename leaves an inert `*.tmp` orphan (never read,
+    // skipped by `.json` scans). No cleanup sweep exists; if a crash-looping writer ever
+    // accumulates many, add a startup sweep of stale `*.tmp` siblings.
+    const tmpPath = path.join(dir, `${path.basename(filePath)}.${process.pid}.${tempCounter++}.tmp`);
+    let handle;
+    try {
+        handle = await fs.open(tmpPath, 'w');
+        await handle.writeFile(content);
+        await handle.sync();
+        await handle.close();
+        handle = undefined;
+        // NOTE: on win32, rename-over-existing can throw EPERM/EACCES/EBUSY if a
+        // concurrent reader holds the target open without FILE_SHARE_DELETE (Node
+        // readIfExists/get open→read→close in a tiny window, so it's rare). Modern
+        // libuv retries some cases; if this ever surfaces as spurious write failures
+        // under concurrent read+write on Windows, add a bounded retry loop here (the
+        // write-file-atomic package does exactly this). Conditional — the adapter is
+        // already last-writer-wins with no cross-process lock (proper-lockfile TODO
+        // in file-storage.ts), so it is not reachable under current single-writer use.
+        await fs.rename(tmpPath, filePath);
+    }
+    catch (err) {
+        if (handle)
+            await handle.close().catch(() => { });
+        await fs.unlink(tmpPath).catch(() => { });
+        throw err;
+    }
+    await fsyncDir(dir);
+}
+/**
+ * Best-effort fsync of a directory so a completed rename is durable. POSIX needs
+ * this; win32 (and platforms that reject opening a directory for fsync) throw,
+ * and we ignore that rather than failing the write.
+ */
+async function fsyncDir(dir) {
+    let handle;
+    try {
+        handle = await fs.open(dir, 'r');
+        await handle.sync();
+    }
+    catch {
+        // Directory fsync unsupported here (e.g. win32) — nothing to do.
+    }
+    finally {
+        if (handle)
+            await handle.close().catch(() => { });
+    }
+}
+//# sourceMappingURL=atomic-write.js.map

package/dist/src/atomic-write.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"atomic-write.js","sourceRoot":"","sources":["../../src/atomic-write.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,QAAQ,IAAI,EAAE,EAAE,MAAM,IAAI,CAAC;AACpC,OAAO,KAAK,IAAI,MAAM,MAAM,CAAC;AAE7B,gFAAgF;AAChF,4EAA4E;AAC5E,+EAA+E;AAC/E,+EAA+E;AAC/E,uDAAuD;AACvD,IAAI,WAAW,GAAG,CAAC,CAAC;AAEpB;;;;;;;;;;;;;;;;;;;GAmBG;AACH,MAAM,CAAC,KAAK,UAAU,eAAe,CAAC,QAAgB,EAAE,OAA4B;IACnF,MAAM,GAAG,GAAG,IAAI,CAAC,OAAO,CAAC,QAAQ,CAAC,CAAC;IACnC,MAAM,EAAE,CAAC,KAAK,CAAC,GAAG,EAAE,EAAE,SAAS,EAAE,IAAI,EAAE,CAAC,CAAC;IAEzC,qFAAqF;IACrF,iEAAiE;IACjE,oFAAoF;IACpF,qFAAqF;IACrF,mEAAmE;IACnE,MAAM,OAAO,GAAG,IAAI,CAAC,IAAI,CAAC,GAAG,EAAE,GAAG,IAAI,CAAC,QAAQ,CAAC,QAAQ,CAAC,IAAI,OAAO,CAAC,GAAG,IAAI,WAAW,EAAE,MAAM,CAAC,CAAC;IAEjG,IAAI,MAAiC,CAAC;IACtC,IAAI,CAAC;QACJ,MAAM,GAAG,MAAM,EAAE,CAAC,IAAI,CAAC,OAAO,EAAE,GAAG,CAAC,CAAC;QACrC,MAAM,MAAM,CAAC,SAAS,CAAC,OAAO,CAAC,CAAC;QAChC,MAAM,MAAM,CAAC,IAAI,EAAE,CAAC;QACpB,MAAM,MAAM,CAAC,KAAK,EAAE,CAAC;QACrB,MAAM,GAAG,SAAS,CAAC;QACnB,yEAAyE;QACzE,0EAA0E;QAC1E,2EAA2E;QAC3E,6EAA6E;QAC7E,6EAA6E;QAC7E,6EAA6E;QAC7E,4EAA4E;QAC5E,+EAA+E;QAC/E,MAAM,EAAE,CAAC,MAAM,CAAC,OAAO,EAAE,QAAQ,CAAC,CAAC;IACpC,CAAC;IAAC,OAAO,GAAG,EAAE,CAAC;QACd,IAAI,MAAM;YAAE,MAAM,MAAM,CAAC,KAAK,EAAE,CAAC,KAAK,CAAC,GAAG,EAAE,GAAqB,CAAC,CAAC,CAAC;QACpE,MAAM,EAAE,CAAC,MAAM,CAAC,OAAO,CAAC,CAAC,KAAK,CAAC,GAAG,EAAE,GAAyC,CAAC,CAAC,CAAC;QAChF,MAAM,GAAG,CAAC;IACX,CAAC;IAED,MAAM,QAAQ,CAAC,GAAG,CAAC,CAAC;AACrB,CAAC;AAED;;;;GAIG;AACH,KAAK,UAAU,QAAQ,CAAC,GAAW;IAClC,IAAI,MAAiC,CAAC;IACtC,IAAI,CAAC;QACJ,MAAM,GAAG,MAAM,EAAE,CAAC,IAAI,CAAC,GAAG,EAAE,GAAG,CAAC,CAAC;QACjC,MAAM,MAAM,CAAC,IAAI,EAAE,CAAC;IACrB,CAAC;IAAC,MAAM,CAAC;QACR,iEAAiE;IAClE,CAAC;YAAS,CAAC;QACV,IAAI,MAAM;YAAE,MAAM,MAAM,CAAC,KAAK,EAAE,CAAC,KAAK,CAAC,GAAG,EAAE,GAAqB,CAAC,CAAC,CAAC;IACrE,CAAC;AACF,CAAC"}

package/dist/src/file-kv-store.d.ts.map

@@ -1 +1 @@
-{"version":3,"file":"file-kv-store.d.ts","sourceRoot":"","sources":["../../src/file-kv-store.ts"],"names":[],"mappings":"AAEA,OAAO,KAAK,EAAE,QAAQ,EAAE,MAAM,oBAAoB,CAAC;AAEnD,+FAA+F;AAC/F,qBAAa,WAAY,YAAW,QAAQ;IAC/B,OAAO,CAAC,QAAQ,CAAC,QAAQ;gBAAR,QAAQ,EAAE,MAAM;IAEvC,GAAG,CAAC,GAAG,EAAE,MAAM,GAAG,OAAO,CAAC,MAAM,GAAG,SAAS,CAAC;IAS7C,GAAG,CAAC,GAAG,EAAE,MAAM,EAAE,KAAK,EAAE,MAAM,GAAG,OAAO,CAAC,IAAI,CAAC;IAM9C,MAAM,CAAC,GAAG,EAAE,MAAM,GAAG,OAAO,CAAC,IAAI,CAAC;IAQlC,IAAI,CAAC,MAAM,EAAE,MAAM,GAAG,OAAO,CAAC,MAAM,EAAE,CAAC;YAS/B,aAAa;IAkB3B,OAAO,CAAC,SAAS;CAGjB"}
+{"version":3,"file":"file-kv-store.d.ts","sourceRoot":"","sources":["../../src/file-kv-store.ts"],"names":[],"mappings":"AAEA,OAAO,KAAK,EAAE,QAAQ,EAAE,MAAM,oBAAoB,CAAC;AAGnD,+FAA+F;AAC/F,qBAAa,WAAY,YAAW,QAAQ;IAC/B,OAAO,CAAC,QAAQ,CAAC,QAAQ;gBAAR,QAAQ,EAAE,MAAM;IAEvC,GAAG,CAAC,GAAG,EAAE,MAAM,GAAG,OAAO,CAAC,MAAM,GAAG,SAAS,CAAC;IAS7C,GAAG,CAAC,GAAG,EAAE,MAAM,EAAE,KAAK,EAAE,MAAM,GAAG,OAAO,CAAC,IAAI,CAAC;IAI9C,MAAM,CAAC,GAAG,EAAE,MAAM,GAAG,OAAO,CAAC,IAAI,CAAC;IAQlC,IAAI,CAAC,MAAM,EAAE,MAAM,GAAG,OAAO,CAAC,MAAM,EAAE,CAAC;YAS/B,aAAa;IAkB3B,OAAO,CAAC,SAAS;CAOjB"}

package/dist/src/file-kv-store.js

@@ -1,5 +1,6 @@
 import { promises as fs } from 'fs';
 import * as path from 'path';
+import { atomicWriteFile } from './atomic-write.js';
 /** Filesystem-backed IKVStore. Keys may contain `/` separators which become subdirectories. */
 export class FileKVStore {
     basePath;
@@ -17,9 +18,7 @@ export class FileKVStore {
         }
     }
     async set(key, value) {
-        const filePath = this.keyToPath(key);
-        await fs.mkdir(path.dirname(filePath), { recursive: true });
-        await fs.writeFile(filePath, value);
+        await atomicWriteFile(this.keyToPath(key), value);
     }
     async delete(key) {
         try {
@@ -59,6 +58,10 @@ export class FileKVStore {
         }
     }
     keyToPath(key) {
+        // NOTE: `/`-separated keys become nested dirs, so a key's first segment shares
+        // the top-level namespace with FileRawStorage's <blockId>/ dirs. Safe today
+        // because block ids are content hashes; if a KV key's first segment could ever
+        // equal a block id, give the two stores separate basePaths (see README Usage).
         return path.join(this.basePath, ...key.split('/')) + '.json';
     }
 }

package/dist/src/file-kv-store.js.map

@@ -1 +1 @@
-{"version":3,"file":"file-kv-store.js","sourceRoot":"","sources":["../../src/file-kv-store.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,QAAQ,IAAI,EAAE,EAAE,MAAM,IAAI,CAAC;AACpC,OAAO,KAAK,IAAI,MAAM,MAAM,CAAC;AAG7B,+FAA+F;AAC/F,MAAM,OAAO,WAAW;IACM;IAA7B,YAA6B,QAAgB;QAAhB,aAAQ,GAAR,QAAQ,CAAQ;IAAG,CAAC;IAEjD,KAAK,CAAC,GAAG,CAAC,GAAW;QACpB,IAAI,CAAC;YACJ,OAAO,MAAM,EAAE,CAAC,QAAQ,CAAC,IAAI,CAAC,SAAS,CAAC,GAAG,CAAC,EAAE,OAAO,CAAC,CAAC;QACxD,CAAC;QAAC,OAAO,GAAG,EAAE,CAAC;YACd,IAAK,GAA6B,EAAE,IAAI,KAAK,QAAQ;gBAAE,OAAO,SAAS,CAAC;YACxE,MAAM,GAAG,CAAC;QACX,CAAC;IACF,CAAC;IAED,KAAK,CAAC,GAAG,CAAC,GAAW,EAAE,KAAa;QACnC,MAAM,QAAQ,GAAG,IAAI,CAAC,SAAS,CAAC,GAAG,CAAC,CAAC;QACrC,MAAM,EAAE,CAAC,KAAK,CAAC,IAAI,CAAC,OAAO,CAAC,QAAQ,CAAC,EAAE,EAAE,SAAS,EAAE,IAAI,EAAE,CAAC,CAAC;QAC5D,MAAM,EAAE,CAAC,SAAS,CAAC,QAAQ,EAAE,KAAK,CAAC,CAAC;IACrC,CAAC;IAED,KAAK,CAAC,MAAM,CAAC,GAAW;QACvB,IAAI,CAAC;YACJ,MAAM,EAAE,CAAC,MAAM,CAAC,IAAI,CAAC,SAAS,CAAC,GAAG,CAAC,CAAC,CAAC;QACtC,CAAC;QAAC,OAAO,GAAG,EAAE,CAAC;YACd,IAAK,GAA6B,EAAE,IAAI,KAAK,QAAQ;gBAAE,MAAM,GAAG,CAAC;QAClE,CAAC;IACF,CAAC;IAED,KAAK,CAAC,IAAI,CAAC,MAAc;QACxB,oEAAoE;QACpE,MAAM,KAAK,GAAG,MAAM,CAAC,KAAK,CAAC,GAAG,CAAC,CAAC,MAAM,CAAC,OAAO,CAAC,CAAC;QAChD,MAAM,OAAO,GAAG,IAAI,CAAC,IAAI,CAAC,IAAI,CAAC,QAAQ,EAAE,GAAG,KAAK,CAAC,CAAC;QACnD,MAAM,OAAO,GAAa,EAAE,CAAC;QAC7B,MAAM,IAAI,CAAC,aAAa,CAAC,OAAO,EAAE,MAAM,EAAE,OAAO,CAAC,CAAC;QACnD,OAAO,OAAO,CAAC;IAChB,CAAC;IAEO,KAAK,CAAC,aAAa,CAAC,OAAe,EAAE,MAAc,EAAE,OAAiB;QAC7E,IAAI,OAAO,CAAC;QACZ,IAAI,CAAC;YACJ,OAAO,GAAG,MAAM,EAAE,CAAC,OAAO,CAAC,OAAO,EAAE,EAAE,aAAa,EAAE,IAAI,EAAE,CAAC,CAAC;QAC9D,CAAC;QAAC,OAAO,GAAG,EAAE,CAAC;YACd,IAAK,GAA6B,EAAE,IAAI,KAAK,QAAQ;gBAAE,OAAO;YAC9D,MAAM,GAAG,CAAC;QACX,CAAC;QACD,KAAK,MAAM,KAAK,IAAI,OAAO,EAAE,CAAC;YAC7B,IAAI,KAAK,CAAC,WAAW,EAAE,EAAE,CAAC;gBACzB,MAAM,SAAS,GAAG,MAAM,GAAG,KAAK,CAAC,IAAI,GAAG,GAAG,CAAC;gBAC5C,MAAM,IAAI,CAAC,aAAa,CAAC,IAAI,CAAC,IAAI,CAAC,OAAO,EAAE,KAAK,CAAC,IAAI,CAAC,EAAE,SAAS,EAAE,OAAO,CAAC,CAAC;YAC9E,CAAC;iBAAM,IAAI,KAAK,CAAC,MAAM,EAAE,IAAI,KAAK,CAAC,IAAI,CAAC,QAAQ,CAAC,OAAO,CAAC,EAAE,CAAC;gBAC3D,OAAO,CAAC,IAAI,CAAC,MAAM,GAAG,KAAK,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,CAAC,CAAC;YAChD,CAAC;QACF,CAAC;IACF,CAAC;IAEO,SAAS,CAAC,GAAW;QAC5B,OAAO,IAAI,CAAC,IAAI,CAAC,IAAI,CAAC,QAAQ,EAAE,GAAG,GAAG,CAAC,KAAK,CAAC,GAAG,CAAC,CAAC,GAAG,OAAO,CAAC;IAC9D,CAAC;CACD"}
+{"version":3,"file":"file-kv-store.js","sourceRoot":"","sources":["../../src/file-kv-store.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,QAAQ,IAAI,EAAE,EAAE,MAAM,IAAI,CAAC;AACpC,OAAO,KAAK,IAAI,MAAM,MAAM,CAAC;AAE7B,OAAO,EAAE,eAAe,EAAE,MAAM,mBAAmB,CAAC;AAEpD,+FAA+F;AAC/F,MAAM,OAAO,WAAW;IACM;IAA7B,YAA6B,QAAgB;QAAhB,aAAQ,GAAR,QAAQ,CAAQ;IAAG,CAAC;IAEjD,KAAK,CAAC,GAAG,CAAC,GAAW;QACpB,IAAI,CAAC;YACJ,OAAO,MAAM,EAAE,CAAC,QAAQ,CAAC,IAAI,CAAC,SAAS,CAAC,GAAG,CAAC,EAAE,OAAO,CAAC,CAAC;QACxD,CAAC;QAAC,OAAO,GAAG,EAAE,CAAC;YACd,IAAK,GAA6B,EAAE,IAAI,KAAK,QAAQ;gBAAE,OAAO,SAAS,CAAC;YACxE,MAAM,GAAG,CAAC;QACX,CAAC;IACF,CAAC;IAED,KAAK,CAAC,GAAG,CAAC,GAAW,EAAE,KAAa;QACnC,MAAM,eAAe,CAAC,IAAI,CAAC,SAAS,CAAC,GAAG,CAAC,EAAE,KAAK,CAAC,CAAC;IACnD,CAAC;IAED,KAAK,CAAC,MAAM,CAAC,GAAW;QACvB,IAAI,CAAC;YACJ,MAAM,EAAE,CAAC,MAAM,CAAC,IAAI,CAAC,SAAS,CAAC,GAAG,CAAC,CAAC,CAAC;QACtC,CAAC;QAAC,OAAO,GAAG,EAAE,CAAC;YACd,IAAK,GAA6B,EAAE,IAAI,KAAK,QAAQ;gBAAE,MAAM,GAAG,CAAC;QAClE,CAAC;IACF,CAAC;IAED,KAAK,CAAC,IAAI,CAAC,MAAc;QACxB,oEAAoE;QACpE,MAAM,KAAK,GAAG,MAAM,CAAC,KAAK,CAAC,GAAG,CAAC,CAAC,MAAM,CAAC,OAAO,CAAC,CAAC;QAChD,MAAM,OAAO,GAAG,IAAI,CAAC,IAAI,CAAC,IAAI,CAAC,QAAQ,EAAE,GAAG,KAAK,CAAC,CAAC;QACnD,MAAM,OAAO,GAAa,EAAE,CAAC;QAC7B,MAAM,IAAI,CAAC,aAAa,CAAC,OAAO,EAAE,MAAM,EAAE,OAAO,CAAC,CAAC;QACnD,OAAO,OAAO,CAAC;IAChB,CAAC;IAEO,KAAK,CAAC,aAAa,CAAC,OAAe,EAAE,MAAc,EAAE,OAAiB;QAC7E,IAAI,OAAO,CAAC;QACZ,IAAI,CAAC;YACJ,OAAO,GAAG,MAAM,EAAE,CAAC,OAAO,CAAC,OAAO,EAAE,EAAE,aAAa,EAAE,IAAI,EAAE,CAAC,CAAC;QAC9D,CAAC;QAAC,OAAO,GAAG,EAAE,CAAC;YACd,IAAK,GAA6B,EAAE,IAAI,KAAK,QAAQ;gBAAE,OAAO;YAC9D,MAAM,GAAG,CAAC;QACX,CAAC;QACD,KAAK,MAAM,KAAK,IAAI,OAAO,EAAE,CAAC;YAC7B,IAAI,KAAK,CAAC,WAAW,EAAE,EAAE,CAAC;gBACzB,MAAM,SAAS,GAAG,MAAM,GAAG,KAAK,CAAC,IAAI,GAAG,GAAG,CAAC;gBAC5C,MAAM,IAAI,CAAC,aAAa,CAAC,IAAI,CAAC,IAAI,CAAC,OAAO,EAAE,KAAK,CAAC,IAAI,CAAC,EAAE,SAAS,EAAE,OAAO,CAAC,CAAC;YAC9E,CAAC;iBAAM,IAAI,KAAK,CAAC,MAAM,EAAE,IAAI,KAAK,CAAC,IAAI,CAAC,QAAQ,CAAC,OAAO,CAAC,EAAE,CAAC;gBAC3D,OAAO,CAAC,IAAI,CAAC,MAAM,GAAG,KAAK,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,CAAC,CAAC;YAChD,CAAC;QACF,CAAC;IACF,CAAC;IAEO,SAAS,CAAC,GAAW;QAC5B,+EAA+E;QAC/E,4EAA4E;QAC5E,+EAA+E;QAC/E,+EAA+E;QAC/E,OAAO,IAAI,CAAC,IAAI,CAAC,IAAI,CAAC,QAAQ,EAAE,GAAG,GAAG,CAAC,KAAK,CAAC,GAAG,CAAC,CAAC,GAAG,OAAO,CAAC;IAC9D,CAAC;CACD"}

package/dist/src/file-storage.d.ts

@@ -1,31 +1,63 @@
-import type { BlockId, IBlock, Transform, ActionId, ActionRev } from "@optimystic/db-core";
-import type { BlockMetadata, IRawStorage } from "@optimystic/db-p2p";
-export declare class FileRawStorage implements IRawStorage {
+import type { BlockId, ActionId } from "@optimystic/db-core";
+import { KvRawStorage, type RawStoreDriver } from "@optimystic/db-p2p";
+/**
+ * Filesystem {@link RawStoreDriver}: the five logical block-storage stores mapped
+ * to five subdirectories under `basePath/<blockId>/`
+ * (`{meta.json,revs/,pend/,actions/,blocks/}`). The directory tree is a
+ * deliberate, human-inspectable/debuggable layout — it is NOT flattened into
+ * encoded-filename KV keys.
+ *
+ * `KvRawStorage` now owns all JSON serialization, so this driver reads/writes raw
+ * `Uint8Array` bytes and never does `JSON.stringify/parse` on values. Everything
+ * else fs-specific lives here: atomic (temp-file + rename) writes, the
+ * corrupt-content-as-missing read guard, colon-encoded action-id filenames with
+ * the legacy raw-colon read fallback + win32 guards, and rename-based promote.
+ */
+export declare class FileStoreDriver implements RawStoreDriver {
     private readonly basePath;
     constructor(basePath: string);
-    getMetadata(blockId: BlockId): Promise<BlockMetadata | undefined>;
-    saveMetadata(blockId: BlockId, metadata: BlockMetadata): Promise<void>;
-    getRevision(blockId: BlockId, rev: number): Promise<ActionId | undefined>;
-    saveRevision(blockId: BlockId, rev: number, actionId: ActionId): Promise<void>;
-    getPendingTransaction(blockId: BlockId, actionId: ActionId): Promise<Transform | undefined>;
-    savePendingTransaction(blockId: BlockId, actionId: ActionId, transform: Transform): Promise<void>;
-    deletePendingTransaction(blockId: BlockId, actionId: ActionId): Promise<void>;
-    listPendingTransactions(blockId: BlockId): AsyncIterable<ActionId>;
-    getTransaction(blockId: BlockId, actionId: ActionId): Promise<Transform | undefined>;
-    listRevisions(blockId: BlockId, startRev: number, endRev: number): AsyncIterable<ActionRev>;
-    saveTransaction(blockId: BlockId, actionId: ActionId, transform: Transform): Promise<void>;
-    getMaterializedBlock(blockId: BlockId, actionId: ActionId): Promise<IBlock | undefined>;
-    saveMaterializedBlock(blockId: BlockId, actionId: ActionId, block?: IBlock): Promise<void>;
-    getApproximateBytesUsed(): Promise<number>;
+    getMetadata(blockId: BlockId): Promise<Uint8Array | undefined>;
+    putMetadata(blockId: BlockId, value: Uint8Array): Promise<void>;
+    getRevision(blockId: BlockId, rev: number): Promise<Uint8Array | undefined>;
+    putRevision(blockId: BlockId, rev: number, value: Uint8Array): Promise<void>;
+    rangeRevisions(blockId: BlockId, lo: number, hi: number, reverse: boolean): AsyncIterable<[number, Uint8Array]>;
+    getPending(blockId: BlockId, actionId: ActionId): Promise<Uint8Array | undefined>;
+    putPending(blockId: BlockId, actionId: ActionId, value: Uint8Array): Promise<void>;
+    deletePending(blockId: BlockId, actionId: ActionId): Promise<void>;
+    listPendingActionIds(blockId: BlockId): AsyncIterable<ActionId>;
+    getTransaction(blockId: BlockId, actionId: ActionId): Promise<Uint8Array | undefined>;
+    putTransaction(blockId: BlockId, actionId: ActionId, value: Uint8Array): Promise<void>;
+    getMaterialized(blockId: BlockId, actionId: ActionId): Promise<Uint8Array | undefined>;
+    putMaterialized(blockId: BlockId, actionId: ActionId, value: Uint8Array): Promise<void>;
+    deleteMaterialized(blockId: BlockId, actionId: ActionId): Promise<void>;
+    promote(blockId: BlockId, actionId: ActionId): Promise<void>;
+    listBlockIds(): AsyncIterable<BlockId>;
+    approximateBytesUsed(): Promise<number>;
     private directoryByteSize;
-    promotePendingTransaction(blockId: BlockId, actionId: ActionId): Promise<void>;
     private getBlockPath;
     private getMetadataPath;
     private getRevisionPath;
     private getPendingActionPath;
     private getActionPath;
     private getMaterializedPath;
-    private readIfExists;
-    private ensureAndWriteFile;
+    private unlinkRawColon;
+    private readActionScopedBytes;
+    private readBytesIfExists;
+}
+/**
+ * Filesystem-backed {@link IRawStorage}, now a thin shell over the shared
+ * {@link KvRawStorage} kernel driven by a {@link FileStoreDriver}. The public
+ * name/constructor (`new FileRawStorage(basePath)`) is unchanged so existing
+ * imports keep resolving; the kernel supplies the `IRawStorage` surface and the
+ * driver supplies fs behavior.
+ *
+ * `listBlockIds`/`getApproximateBytesUsed` are re-declared here as always-present
+ * (the fs driver always implements them, so the kernel constructor always wires
+ * them) — the base declares them optional, but every fs consumer relies on them.
+ */
+export declare class FileRawStorage extends KvRawStorage {
+    listBlockIds: () => AsyncIterable<BlockId>;
+    getApproximateBytesUsed: () => Promise<number>;
+    constructor(basePath: string);
 }
 //# sourceMappingURL=file-storage.d.ts.map

package/dist/src/file-storage.d.ts.map

@@ -1 +1 @@
-{"version":3,"file":"file-storage.d.ts","sourceRoot":"","sources":["../../src/file-storage.ts"],"names":[],"mappings":"AAEA,OAAO,KAAK,EAAE,OAAO,EAAE,MAAM,EAAE,SAAS,EAAE,QAAQ,EAAE,SAAS,EAAE,MAAM,qBAAqB,CAAC;AAC3F,OAAO,KAAK,EAAE,aAAa,EAAE,WAAW,EAAE,MAAM,oBAAoB,CAAC;AAerE,qBAAa,cAAe,YAAW,WAAW;IACrC,OAAO,CAAC,QAAQ,CAAC,QAAQ;gBAAR,QAAQ,EAAE,MAAM;IAIvC,WAAW,CAAC,OAAO,EAAE,OAAO,GAAG,OAAO,CAAC,aAAa,GAAG,SAAS,CAAC;IAIjE,YAAY,CAAC,OAAO,EAAE,OAAO,EAAE,QAAQ,EAAE,aAAa,GAAG,OAAO,CAAC,IAAI,CAAC;IAOtE,WAAW,CAAC,OAAO,EAAE,OAAO,EAAE,GAAG,EAAE,MAAM,GAAG,OAAO,CAAC,QAAQ,GAAG,SAAS,CAAC;IAIzE,YAAY,CAAC,OAAO,EAAE,OAAO,EAAE,GAAG,EAAE,MAAM,EAAE,QAAQ,EAAE,QAAQ,GAAG,OAAO,CAAC,IAAI,CAAC;IAO9E,qBAAqB,CAAC,OAAO,EAAE,OAAO,EAAE,QAAQ,EAAE,QAAQ,GAAG,OAAO,CAAC,SAAS,GAAG,SAAS,CAAC;IAI3F,sBAAsB,CAAC,OAAO,EAAE,OAAO,EAAE,QAAQ,EAAE,QAAQ,EAAE,SAAS,EAAE,SAAS,GAAG,OAAO,CAAC,IAAI,CAAC;IAOjG,wBAAwB,CAAC,OAAO,EAAE,OAAO,EAAE,QAAQ,EAAE,QAAQ,GAAG,OAAO,CAAC,IAAI,CAAC;IAQ5E,uBAAuB,CAAC,OAAO,EAAE,OAAO,GAAG,aAAa,CAAC,QAAQ,CAAC;IAenE,cAAc,CAAC,OAAO,EAAE,OAAO,EAAE,QAAQ,EAAE,QAAQ,GAAG,OAAO,CAAC,SAAS,GAAG,SAAS,CAAC;IAInF,aAAa,CAAC,OAAO,EAAE,OAAO,EAAE,QAAQ,EAAE,MAAM,EAAE,MAAM,EAAE,MAAM,GAAG,aAAa,CAAC,SAAS,CAAC;IAS5F,eAAe,CAAC,OAAO,EAAE,OAAO,EAAE,QAAQ,EAAE,QAAQ,EAAE,SAAS,EAAE,SAAS,GAAG,OAAO,CAAC,IAAI,CAAC;IAO1F,oBAAoB,CAAC,OAAO,EAAE,OAAO,EAAE,QAAQ,EAAE,QAAQ,GAAG,OAAO,CAAC,MAAM,GAAG,SAAS,CAAC;IAIvF,qBAAqB,CAAC,OAAO,EAAE,OAAO,EAAE,QAAQ,EAAE,QAAQ,EAAE,KAAK,CAAC,EAAE,MAAM,GAAG,OAAO,CAAC,IAAI,CAAC;IAc1F,uBAAuB,IAAI,OAAO,CAAC,MAAM,CAAC;YAIlC,iBAAiB;IA2BzB,yBAAyB,CAAC,OAAO,EAAE,OAAO,EAAE,QAAQ,EAAE,QAAQ,GAAG,OAAO,CAAC,IAAI,CAAC;IAgBpF,OAAO,CAAC,YAAY;IAIpB,OAAO,CAAC,eAAe;IAIvB,OAAO,CAAC,eAAe;IAIvB,OAAO,CAAC,oBAAoB;IAI5B,OAAO,CAAC,aAAa;IAIrB,OAAO,CAAC,mBAAmB;YAIb,YAAY;YASZ,kBAAkB;CAIhC"}
+{"version":3,"file":"file-storage.d.ts","sourceRoot":"","sources":["../../src/file-storage.ts"],"names":[],"mappings":"AAEA,OAAO,KAAK,EAAE,OAAO,EAAE,QAAQ,EAAE,MAAM,qBAAqB,CAAC;AAC7D,OAAO,EAAE,YAAY,EAAE,KAAK,cAAc,EAAE,MAAM,oBAAoB,CAAC;AAsCvE;;;;;;;;;;;;GAYG;AACH,qBAAa,eAAgB,YAAW,cAAc;IACzC,OAAO,CAAC,QAAQ,CAAC,QAAQ;gBAAR,QAAQ,EAAE,MAAM;IAMvC,WAAW,CAAC,OAAO,EAAE,OAAO,GAAG,OAAO,CAAC,UAAU,GAAG,SAAS,CAAC;IAI9D,WAAW,CAAC,OAAO,EAAE,OAAO,EAAE,KAAK,EAAE,UAAU,GAAG,OAAO,CAAC,IAAI,CAAC;IAe/D,WAAW,CAAC,OAAO,EAAE,OAAO,EAAE,GAAG,EAAE,MAAM,GAAG,OAAO,CAAC,UAAU,GAAG,SAAS,CAAC;IAI3E,WAAW,CAAC,OAAO,EAAE,OAAO,EAAE,GAAG,EAAE,MAAM,EAAE,KAAK,EAAE,UAAU,GAAG,OAAO,CAAC,IAAI,CAAC;IAI3E,cAAc,CAAC,OAAO,EAAE,OAAO,EAAE,EAAE,EAAE,MAAM,EAAE,EAAE,EAAE,MAAM,EAAE,OAAO,EAAE,OAAO,GAAG,aAAa,CAAC,CAAC,MAAM,EAAE,UAAU,CAAC,CAAC;IAuBhH,UAAU,CAAC,OAAO,EAAE,OAAO,EAAE,QAAQ,EAAE,QAAQ,GAAG,OAAO,CAAC,UAAU,GAAG,SAAS,CAAC;IAOjF,UAAU,CAAC,OAAO,EAAE,OAAO,EAAE,QAAQ,EAAE,QAAQ,EAAE,KAAK,EAAE,UAAU,GAAG,OAAO,CAAC,IAAI,CAAC;IAIlF,aAAa,CAAC,OAAO,EAAE,OAAO,EAAE,QAAQ,EAAE,QAAQ,GAAG,OAAO,CAAC,IAAI,CAAC;IASjE,oBAAoB,CAAC,OAAO,EAAE,OAAO,GAAG,aAAa,CAAC,QAAQ,CAAC;IA8ChE,cAAc,CAAC,OAAO,EAAE,OAAO,EAAE,QAAQ,EAAE,QAAQ,GAAG,OAAO,CAAC,UAAU,GAAG,SAAS,CAAC;IAOrF,cAAc,CAAC,OAAO,EAAE,OAAO,EAAE,QAAQ,EAAE,QAAQ,EAAE,KAAK,EAAE,UAAU,GAAG,OAAO,CAAC,IAAI,CAAC;IAMtF,eAAe,CAAC,OAAO,EAAE,OAAO,EAAE,QAAQ,EAAE,QAAQ,GAAG,OAAO,CAAC,UAAU,GAAG,SAAS,CAAC;IAOtF,eAAe,CAAC,OAAO,EAAE,OAAO,EAAE,QAAQ,EAAE,QAAQ,EAAE,KAAK,EAAE,UAAU,GAAG,OAAO,CAAC,IAAI,CAAC;IAMvF,kBAAkB,CAAC,OAAO,EAAE,OAAO,EAAE,QAAQ,EAAE,QAAQ,GAAG,OAAO,CAAC,IAAI,CAAC;IAWvE,OAAO,CAAC,OAAO,EAAE,OAAO,EAAE,QAAQ,EAAE,QAAQ,GAAG,OAAO,CAAC,IAAI,CAAC;IAqB3D,YAAY,IAAI,aAAa,CAAC,OAAO,CAAC;IA4CvC,oBAAoB,IAAI,OAAO,CAAC,MAAM,CAAC;YAI/B,iBAAiB;IA6B/B,OAAO,CAAC,YAAY;IAIpB,OAAO,CAAC,eAAe;IAIvB,OAAO,CAAC,eAAe;IAQvB,OAAO,CAAC,oBAAoB;IAK5B,OAAO,CAAC,aAAa;IAKrB,OAAO,CAAC,mBAAmB;YASb,cAAc;YAsBd,qBAAqB;YAcrB,iBAAiB;CAc/B;AAED;;;;;;;;;;GAUG;AACH,qBAAa,cAAe,SAAQ,YAAY;IACvC,YAAY,EAAE,MAAM,aAAa,CAAC,OAAO,CAAC,CAAC;IAC3C,uBAAuB,EAAE,MAAM,OAAO,CAAC,MAAM,CAAC,CAAC;gBAE3C,QAAQ,EAAE,MAAM;CAG5B"}