@bod.ee/db 0.11.1 → 0.12.1

package/CLAUDE.md

@@ -72,6 +72,7 @@ config.ts                  — demo instance config (open rules, indexes, fts, v
 - **MCP**: `MCPAdapter` wraps a `BodClient` as a JSON-RPC MCP server (stdio + HTTP). Connects to a running BodDB instance over WebSocket — no embedded DB. Entry point: `mcp.ts`. Tools: CRUD (6), FTS (2), vectors (2), streams (4), MQ (7) = 21 tools. Use `--stdio` for Claude Code/Desktop, `--http` for remote agents.
 - **VFS (Virtual File System)**: `VFSEngine` — files stored outside SQLite via pluggable `VFSBackend` interface. `LocalBackend` stores at `<storageRoot>/<fileId>` using `Bun.file`/`Bun.write`. Metadata at `_vfs/<virtualPath>/` (size, mime, mtime, fileId, isDir) — gets subs/rules/replication for free. `fileId = pushId` so move/rename is metadata-only. REST: `POST/GET/DELETE /files/<path>`, `?stat=1`, `?list=1`, `?mkdir=1`, `PUT ?move=<dst>`. WS chunked fallback: base64-encoded `vfs-upload-init/chunk/done`, `vfs-download-init` → `vfs-download-chunk` push messages. Client: `VFSClient` via `client.vfs()` — `upload/download` (REST) + `uploadWS/downloadWS` (WS) + `stat/list/mkdir/delete/move`.
 - **Replication**: `ReplicationEngine` — single primary + N read replicas + multi-source feed subscriptions. Star topology. Primary emits write events to `_repl` stream via `onWrite` hooks. Replicas bootstrap via `streamMaterialize('_repl', { keepKey: 'path' })`, then subscribe for ongoing events. Write proxy: replica forwards writes to primary via BodClient, primary applies + emits, replica consumes. `_replaying` flag prevents re-emission loops. `_emitting` guard prevents recursion from `db.push('_repl')`. Updates flattened to per-path set events for correct compaction keying. Sweep delete events replicated. Excluded prefixes: `_repl`, `_streams`, `_mq`, `_auth`. **Sources**: `ReplicationSource[]` — subscribe to specific paths from multiple remote DBs. Each source is an independent BodClient that filters `_repl` events by path prefix, with optional `localPrefix` remapping (e.g. remote `users/u1` → local `db-a/users/u1`). Sources connect in parallel; individual failures don't block others. Sources are independent of role — a DB can be primary AND consume sources.
+- **KeyAuth integration guide**: `docs/keyauth-integration.md` — flows for signup, signin, new device, autoAuth, IAM roles, common mistakes.
 - **KeyAuth**: `KeyAuthEngine` — portable Ed25519 identity & IAM. Identity hierarchy: Root (server-level, key on filesystem), Account (portable, password-encrypted private key in DB or device-generated), Device (delegate, linked via password unlock). Challenge-response auth: server sends nonce → client signs with Ed25519 → server verifies + creates session. Self-signed tokens (no JWT lib): `base64url(payload).base64url(Ed25519_sign)`. Data model at `_auth/` prefix (protected from external writes). Device reverse-index at `_auth/deviceIndex/{dfp}` for O(1) lookup. Password change is atomic (single `db.update()`). IAM: roles with path-based permissions, account role assignment. `_auth/` excluded from replication. Transport guards: `auth-link-device` and `auth-change-password` require authenticated session; non-root users can only change own password. **Device registration**: `registerDevice(publicKey)` — client-generated keypair, no password, idempotent; `allowOpenRegistration: false` requires authenticated session. **Browser crypto**: `keyAuth.browser.ts` uses `@noble/ed25519` with DER↔raw key bridge for server compatibility. **BodClient autoAuth**: `autoAuth: true` auto-generates keypair (localStorage), registers, authenticates — zero-config device identity. `client.auth.*` convenience methods for all auth ops. **IAM transport ops**: `auth-create-role`, `auth-delete-role`, `auth-update-roles` (root only), `auth-list-accounts`, `auth-list-roles`. Device accounts (no encrypted key) safely reject `linkDevice`/`changePassword`.
 
 ## MCP Server

package/docs/keyauth-integration.md

@@ -0,0 +1,141 @@
+# KeyAuth Integration Guide
+
+BodDB's auth system has two identity layers — **accounts** (portable, password-protected) and **devices** (device-bound, no password). Understanding which to use is critical.
+
+---
+
+## Identity Model
+
+| Type | Private key lives | Auth method | Portable? |
+|------|------------------|-------------|-----------|
+| **Account** | Encrypted in DB (password-protected) | Password (to unlock/link) | Yes — any device |
+| **Device** | Client only (localStorage / memory) | Challenge-response (Ed25519) | No — tied to this client |
+
+**Root** — first account ever created is auto-elevated to root. Server-level privileges.
+
+---
+
+## When to use what
+
+**Use `autoAuth: true` (device-only)** when:
+- You need zero-config anonymous identity (e.g. telemetry, guest sessions)
+- Identity loss is acceptable (clearing localStorage = new identity)
+- No cross-device portability needed
+
+**Use account + linked device** when:
+- Users have passwords and expect to sign in from multiple devices
+- Identity must survive localStorage wipe
+- You need IAM roles assigned to a persistent identity
+
+---
+
+## Flows
+
+### Signup (new user, first device)
+
+```ts
+const client = new BodClient({ url: 'ws://localhost:4400' });
+await client.connect();
+
+// 1. Generate device keypair (client-side, stored in localStorage)
+const kp = await generateDeviceKeypair(); // your key generation
+
+// 2. Create account — auto-links this device
+const account = await client.auth.createAccount(
+  password,
+  [],           // roles (root assigns later)
+  'alice',      // displayName
+  kp.publicKey  // auto-links device → no separate linkDevice call needed
+);
+// account.isRoot === true if this is the very first account on the server
+
+// 3. Authenticate with the device keypair
+const { token } = await client.auth.authenticate(kp.publicKey, kp.signFn);
+```
+
+### Sign in (same device, returning user)
+
+```ts
+// Device keypair is already in localStorage — just authenticate
+const { token } = await client.auth.authenticate(kp.publicKey, kp.signFn);
+// Send token on subsequent requests (handled automatically by BodClient)
+```
+
+### Sign in from a new device
+
+```ts
+// On the NEW device:
+const newKp = await generateDeviceKeypair();
+
+// Link it to the existing account (requires password)
+await client.auth.linkDeviceByName('alice', password, newKp.publicKey, 'MacBook Pro');
+// or by fingerprint:
+await client.auth.linkDevice(accountFingerprint, password, newKp.publicKey, 'MacBook Pro');
+
+// Now authenticate with the new device — no password needed again
+const { token } = await client.auth.authenticate(newKp.publicKey, newKp.signFn);
+```
+
+### autoAuth (zero-config device identity)
+
+```ts
+// Keypair auto-generated + stored in localStorage, registered + authenticated on connect
+const client = new BodClient({ url: 'ws://localhost:4400', autoAuth: true });
+await client.connect();
+// client.deviceFingerprint and client.authToken are set automatically
+```
+
+> ⚠️ `autoAuth` creates a device-only identity. There is no account behind it — no password, no cross-device portability. Clearing localStorage permanently loses this identity.
+
+---
+
+## Common mistakes
+
+| Mistake | Problem | Fix |
+|---------|---------|-----|
+| Using `autoAuth` for user accounts | Identity lost on localStorage clear | Use `createAccount` + `linkDevice` |
+| Calling `createAccount` without `devicePublicKey` | Have to call `linkDevice` separately | Pass `devicePublicKey` to `createAccount` |
+| Not knowing account fingerprint on new device | Can't call `linkDevice` | Use `linkDeviceByName(displayName, ...)` instead |
+| Assuming `autoAuth` device has an account | Device accounts reject `linkDevice`/`changePassword` | They're standalone — no account behind them |
+
+---
+
+## Server setup
+
+```ts
+import { BodDB } from 'bod-db';
+import { KeyAuthEngine } from 'bod-db/server';
+
+const db = new BodDB({
+  keyAuth: {
+    allowOpenRegistration: false, // require auth session to register devices (recommended)
+    maxAuthFailures: 10,
+    authLockoutSeconds: 300,
+  }
+});
+```
+
+`allowOpenRegistration: true` (default) allows any client to register a device without being authenticated first — fine for open apps, restrict for closed ones.
+
+---
+
+## IAM roles
+
+Roles are assigned to **accounts** (not devices). Devices inherit the account's roles via the linked account fingerprint.
+
+```ts
+// Root only
+await client.auth.createRole({
+  id: 'editor',
+  name: 'Editor',
+  permissions: [
+    { path: 'posts/$postId', read: true, write: true },
+    { path: 'users/$uid/profile', read: true },
+  ]
+});
+
+// Assign to account (root only)
+await client.auth.updateRoles(accountFingerprint, ['editor']);
+```
+
+Rules engine receives `auth.roles`, `auth.fingerprint`, `auth.accountFingerprint` in context.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@bod.ee/db",
-  "version": "0.11.1",
+  "version": "0.12.1",
   "module": "index.ts",
   "type": "module",
   "exports": {

package/src/client/BodClient.ts

@@ -829,6 +829,10 @@ export class VFSClient {
     return await this.client._send('vfs-list', { path }) as FileStat[];
   }
 
+  async tree(path: string, opts?: { hiddenPaths?: string[]; hideDotfiles?: boolean }): Promise<any[]> {
+    return await this.client._send('vfs-tree', { path, ...opts }) as any[];
+  }
+
   async delete(path: string): Promise<void> {
     await this.client._send('vfs-delete', { path });
   }

package/src/client/BodClientCached.ts

@@ -182,7 +182,7 @@ export class BodClientCached {
 
   vfs(): CachedVFSClient {
     if (!this._vfs) {
-      this._vfs = new CachedVFSClient(this.client.vfs(), this.options.maxAge);
+      this._vfs = new CachedVFSClient(this.client.vfs(), this.options.maxAge, this.idb);
       this._vfsUnsub = this.client.onChild('_vfs', () => {
         // _vfs events are coarse (top-level key only), clear all VFS caches
         this._vfs?.clear();
@@ -252,18 +252,34 @@ export class CachedVFSClient {
   constructor(
     private raw: VFSClient,
     private maxAge: number,
+    private idb: IDBDatabase | null = null,
   ) {}
 
   async stat(path: string): Promise<FileStat | null> {
     const cached = this.statCache.get(path);
     if (cached && Date.now() - cached.cachedAt < this.maxAge) {
       this._stats.hits++;
-      this.raw.stat(path).then(r => this.statCache.set(path, { data: r, cachedAt: Date.now() })).catch(() => {});
+      this.raw.stat(path).then(r => {
+        this.statCache.set(path, { data: r, cachedAt: Date.now() });
+        this.idbSetVfs(`vfs:stat:${path}`, { data: r, cachedAt: Date.now() });
+      }).catch(() => {});
       return cached.data;
     }
+    // Check IDB before network
+    const idbEntry = await this.idbGetVfs(`vfs:stat:${path}`);
+    if (idbEntry && Date.now() - idbEntry.cachedAt < this.maxAge) {
+      this._stats.hits++;
+      this.statCache.set(path, idbEntry);
+      this.raw.stat(path).then(r => {
+        this.statCache.set(path, { data: r, cachedAt: Date.now() });
+        this.idbSetVfs(`vfs:stat:${path}`, { data: r, cachedAt: Date.now() });
+      }).catch(() => {});
+      return idbEntry.data as FileStat | null;
+    }
     this._stats.misses++;
     const result = await this.raw.stat(path);
     this.statCache.set(path, { data: result, cachedAt: Date.now() });
+    this.idbSetVfs(`vfs:stat:${path}`, { data: result, cachedAt: Date.now() });
     return result;
   }
 
@@ -271,15 +287,34 @@ export class CachedVFSClient {
     const cached = this.listCache.get(path);
     if (cached && Date.now() - cached.cachedAt < this.maxAge) {
       this._stats.hits++;
-      this.raw.list(path).then(r => this.listCache.set(path, { data: r, cachedAt: Date.now() })).catch(() => {});
+      this.raw.list(path).then(r => {
+        this.listCache.set(path, { data: r, cachedAt: Date.now() });
+        this.idbSetVfs(`vfs:list:${path}`, { data: r, cachedAt: Date.now() });
+      }).catch(() => {});
       return cached.data;
     }
+    // Check IDB before network
+    const idbEntry = await this.idbGetVfs(`vfs:list:${path}`);
+    if (idbEntry && Date.now() - idbEntry.cachedAt < this.maxAge) {
+      this._stats.hits++;
+      this.listCache.set(path, idbEntry as { data: FileStat[]; cachedAt: number });
+      this.raw.list(path).then(r => {
+        this.listCache.set(path, { data: r, cachedAt: Date.now() });
+        this.idbSetVfs(`vfs:list:${path}`, { data: r, cachedAt: Date.now() });
+      }).catch(() => {});
+      return idbEntry.data as FileStat[];
+    }
     this._stats.misses++;
     const result = await this.raw.list(path);
     this.listCache.set(path, { data: result, cachedAt: Date.now() });
+    this.idbSetVfs(`vfs:list:${path}`, { data: result, cachedAt: Date.now() });
     return result;
   }
 
+  async tree(path: string, opts?: { hiddenPaths?: string[]; hideDotfiles?: boolean }): Promise<any[]> {
+    return this.raw.tree(path, opts);
+  }
+
   async upload(path: string, data: Uint8Array, mime?: string): Promise<FileStat> {
     const r = await this.raw.upload(path, data, mime);
     this.invalidatePath(path);
@@ -296,6 +331,7 @@ export class CachedVFSClient {
     this.invalidatePath(path);
     // Also invalidate the dir's own list (was empty/nonexistent before)
     this.listCache.delete(path);
+    this.idbDeleteVfs(`vfs:list:${path}`);
     return r;
   }
 
@@ -313,14 +349,17 @@ export class CachedVFSClient {
   invalidatePath(filePath: string) {
     this._stats.invalidations++;
     this.statCache.delete(filePath);
+    this.idbDeleteVfs(`vfs:stat:${filePath}`);
     const parent = filePath.includes('/') ? filePath.slice(0, filePath.lastIndexOf('/')) : '';
     this.listCache.delete(parent);
+    this.idbDeleteVfs(`vfs:list:${parent}`);
   }
 
   clear() {
     this._stats.pushClears++;
     this.listCache.clear();
     this.statCache.clear();
+    this.idbClearVfs();
   }
 
   /** Cache stats for diagnostics. Use in browser: `__bodCache.vfs().stats` */
@@ -334,4 +373,53 @@ export class CachedVFSClient {
       statCacheSize: this.statCache.size,
     };
   }
+
+  // --- IDB helpers for VFS cache ---
+
+  private idbGetVfs(key: string): Promise<{ data: unknown; cachedAt: number } | null> {
+    if (!this.idb) return Promise.resolve(null);
+    try {
+      const tx = this.idb.transaction('entries', 'readonly');
+      const req = tx.objectStore('entries').get(key);
+      return new Promise(resolve => {
+        req.onsuccess = () => {
+          const entry = req.result;
+          resolve(entry ? { data: entry.data, cachedAt: entry.cachedAt } : null);
+        };
+        req.onerror = () => resolve(null);
+      });
+    } catch { return Promise.resolve(null); }
+  }
+
+  private idbSetVfs(key: string, value: { data: unknown; cachedAt: number }): void {
+    if (!this.idb) return;
+    try {
+      const tx = this.idb.transaction('entries', 'readwrite');
+      tx.objectStore('entries').put({ path: key, data: value.data, cachedAt: value.cachedAt, updatedAt: Date.now() });
+    } catch {}
+  }
+
+  private idbDeleteVfs(key: string): void {
+    if (!this.idb) return;
+    try {
+      const tx = this.idb.transaction('entries', 'readwrite');
+      tx.objectStore('entries').delete(key);
+    } catch {}
+  }
+
+  private idbClearVfs(): void {
+    if (!this.idb) return;
+    try {
+      // Delete all vfs: prefixed entries
+      const tx = this.idb.transaction('entries', 'readwrite');
+      const store = tx.objectStore('entries');
+      const req = store.openCursor();
+      req.onsuccess = () => {
+        const cursor = req.result;
+        if (!cursor) return;
+        if (typeof cursor.key === 'string' && cursor.key.startsWith('vfs:')) cursor.delete();
+        cursor.continue();
+      };
+    } catch {}
+  }
 }

package/src/server/Transport.ts

@@ -753,6 +753,14 @@ export class Transport {
               }
               return reply(self.db.vfs.list(msg.path));
             }
+            case 'vfs-tree': {
+              if (!self.db.vfs) return error('VFS not configured', Errors.INTERNAL);
+              if (self.rules && !self.rules.check('read', `_vfs/${msg.path}`, ws.data.auth)) {
+                return error('Permission denied', Errors.PERMISSION_DENIED);
+              }
+              const hiddenPaths = msg.hiddenPaths ? new Set(msg.hiddenPaths) : undefined;
+              return reply(self.db.vfs.tree(msg.path, { hiddenPaths, hideDotfiles: msg.hideDotfiles }));
+            }
             case 'vfs-delete': {
               if (!self.db.vfs) return error('VFS not configured', Errors.INTERNAL);
               if (self.rules && !self.rules.check('write', `_vfs/${msg.path}`, ws.data.auth)) {

package/src/server/VFSEngine.ts

@@ -183,6 +183,33 @@ export class VFSEngine {
     return results;
   }
 
+  tree(virtualPath: string, opts?: { hiddenPaths?: Set<string>; hideDotfiles?: boolean }): any[] {
+    const hiddenPaths = opts?.hiddenPaths ?? new Set<string>();
+    const hideDotfiles = opts?.hideDotfiles ?? false;
+
+    const walk = (vPath: string): any[] => {
+      const stats = this.list(vPath);
+      stats.sort((a, b) => {
+        if (a.isDir !== b.isDir) return a.isDir ? -1 : 1;
+        return a.name.localeCompare(b.name);
+      });
+      const result: any[] = [];
+      for (const s of stats) {
+        if (hiddenPaths.has(s.name)) continue;
+        if (hideDotfiles && s.name.startsWith('.')) continue;
+        if (s.isDir) {
+          const childPath = vPath ? `${vPath}/${s.name}` : s.name;
+          result.push({ name: s.name, type: 'directory', children: walk(childPath) });
+        } else {
+          result.push({ name: s.name, type: 'file' });
+        }
+      }
+      return result;
+    };
+
+    return walk(normalizePath(virtualPath));
+  }
+
   mkdir(virtualPath: string): FileStat {
     const vp = normalizePath(virtualPath);
     const name = vp.split('/').pop()!;

package/src/shared/protocol.ts

@@ -36,6 +36,7 @@ export type ClientMessage =
   | { id: string; op: 'vfs-download-init'; path: string }
   | { id: string; op: 'vfs-stat'; path: string }
   | { id: string; op: 'vfs-list'; path: string }
+  | { id: string; op: 'vfs-tree'; path: string; hiddenPaths?: string[]; hideDotfiles?: boolean }
   | { id: string; op: 'vfs-delete'; path: string }
   | { id: string; op: 'vfs-mkdir'; path: string }
   | { id: string; op: 'vfs-move'; path: string; dst: string }