@bod.ee/db 0.12.4 → 0.12.8

package/.claude/skills/using-bod-db.md

@@ -27,6 +27,12 @@ import { BodDB, increment, serverTimestamp, arrayUnion, arrayRemove, ref } from
 const db = new BodDB({
   path: './data.db',       // SQLite file (default: ':memory:')
   port: 4400,              // optional — only needed if calling db.serve()
+  log: {                   // optional structured logging
+    enabled: true,
+    level: 'info',         // 'debug' | 'info' | 'warn' | 'error'
+    components: '*',       // '*' or ['storage', 'transport', 'subs', 'replication', 'keyauth']
+    logDir: './logs',      // writes bod-YYYYMMDD-HHmmss.log per run; omit for console-only
+  },
   rules: {                 // inline rules, or path to .json/.ts file
     'users/$uid': {
       read: true,
@@ -497,7 +503,7 @@ await replica.replication!.start();
 - Reads served locally on replica, writes forwarded to primary
 - Replicas bootstrap full state on first connect, then consume live events
 - Push keys preserved across replicas (deterministic)
-- Auto-compaction on `_repl` stream (keepKey: 'path', maxCount: 10000)
+- Auto-compaction on `_repl` stream (keepKey: 'path', maxCount: 500, auto-compact every 500 writes)
 - Excluded prefixes: `_repl`, `_streams`, `_mq`, `_auth` (internal data not replicated)
 
 ### Per-Path Topology

package/CLAUDE.md

@@ -71,7 +71,7 @@ config.ts                  — demo instance config (open rules, indexes, fts, v
 - **BodClientCached**: two-tier cache wrapper around BodClient. Memory (Map, LRU eviction) + IndexedDB persistence. Stale-while-revalidate: subscribed paths always fresh, unsubscribed return stale + background refetch. Writes (`set/update/delete`) invalidate path + ancestors. `init()` opens IDB + sweeps expired. `warmup(paths[])` bulk-loads from IDB. Passthrough for `push/batch/query/search/mq/stream/vfs` via `cachedClient.client`.
 - **MCP**: `MCPAdapter` wraps a `BodClient` as a JSON-RPC MCP server (stdio + HTTP). Connects to a running BodDB instance over WebSocket — no embedded DB. Entry point: `mcp.ts`. Tools: CRUD (6), FTS (2), vectors (2), streams (4), MQ (7) = 21 tools. Use `--stdio` for Claude Code/Desktop, `--http` for remote agents.
 - **VFS (Virtual File System)**: `VFSEngine` — files stored outside SQLite via pluggable `VFSBackend` interface. `LocalBackend` stores at `<storageRoot>/<fileId>` using `Bun.file`/`Bun.write`. Metadata at `_vfs/<virtualPath>/` (size, mime, mtime, fileId, isDir) — gets subs/rules/replication for free. `fileId = pushId` so move/rename is metadata-only. REST: `POST/GET/DELETE /files/<path>`, `?stat=1`, `?list=1`, `?mkdir=1`, `PUT ?move=<dst>`. WS chunked fallback: base64-encoded `vfs-upload-init/chunk/done`, `vfs-download-init` → `vfs-download-chunk` push messages. Client: `VFSClient` via `client.vfs()` — `upload/download` (REST) + `uploadWS/downloadWS` (WS) + `stat/list/mkdir/delete/move`.
-- **Replication**: `ReplicationEngine` — single primary + N read replicas + multi-source feed subscriptions. Star topology. Primary emits write events to `_repl` stream via `onWrite` hooks. Replicas bootstrap via cursor-based `streamMaterialize('_repl', { keepKey: 'path', batchSize: 200 })` pagination (avoids huge single WS frame), then subscribe for ongoing events. Auto-compact on write threshold (`autoCompactThreshold`, default 500) + on startup keeps `_repl` bounded. Write proxy: replica forwards writes to primary via BodClient, primary applies + emits, replica consumes. `_replaying` flag prevents re-emission loops. `_emitting` guard prevents recursion from `db.push('_repl')`. Updates flattened to per-path set events for correct compaction keying. Sweep delete events replicated. Excluded prefixes: `_repl`, `_streams`, `_mq`, `_auth`. **Sources**: `ReplicationSource[]` — subscribe to specific paths from multiple remote DBs. Each source is an independent BodClient that filters `_repl` events by path prefix, with optional `localPrefix` remapping (e.g. remote `users/u1` → local `db-a/users/u1`). Sources connect in parallel; individual failures don't block others. Sources are independent of role — a DB can be primary AND consume sources. **Per-path topology**: `PathTopologyRouter` — when `paths` config is set, each path prefix gets an independent mode: `primary` (local authoritative, emits), `replica` (remote authoritative, proxies writes), `sync` (bidirectional, both emit+apply), `readonly` (pull-only, rejects writes), `writeonly` (push-only, ignores remote). Longest-prefix match resolves mode. `writeProxy: 'proxy'|'reject'` overrides replica write behavior. Bootstrap skips sync paths (ongoing stream only). Auth/rules checked before proxy in all handlers. `shouldProxyPath(path)`/`shouldRejectPath(path)` replace `isReplica` checks. `emitsToRepl`/`pullsFromPrimary` getters for compact/bootstrap decisions. Stable `replicaId` from config hash. Falls back to `role` when `paths` absent (backward compat).
+- **Replication**: `ReplicationEngine` — single primary + N read replicas + multi-source feed subscriptions. Star topology. Primary emits write events to `_repl` stream via `onWrite` hooks. Replicas bootstrap via cursor-based `streamMaterialize('_repl', { keepKey: 'path', batchSize: 200 })` pagination (avoids huge single WS frame), then subscribe for ongoing events. Auto-compact on write threshold (`autoCompactThreshold`, default 500) + on startup keeps `_repl` bounded. Write proxy: replica forwards writes to primary via BodClient, primary applies + emits, replica consumes. `_replaying` flag prevents re-emission loops. `_emitting` guard prevents recursion from `db.push('_repl')`. Updates flattened to per-path set events for correct compaction keying. Sweep delete events replicated. Excluded prefixes: `_repl`, `_streams`, `_mq`, `_admin`, `_auth`. **Sources**: `ReplicationSource[]` — subscribe to specific paths from multiple remote DBs. Each source is an independent BodClient that filters `_repl` events by path prefix, with optional `localPrefix` remapping (e.g. remote `users/u1` → local `db-a/users/u1`). Sources connect in parallel; individual failures don't block others. Sources are independent of role — a DB can be primary AND consume sources. **Per-path topology**: `PathTopologyRouter` — when `paths` config is set, each path prefix gets an independent mode: `primary` (local authoritative, emits), `replica` (remote authoritative, proxies writes), `sync` (bidirectional, both emit+apply), `readonly` (pull-only, rejects writes), `writeonly` (push-only, ignores remote). Longest-prefix match resolves mode. `writeProxy: 'proxy'|'reject'` overrides replica write behavior. Bootstrap skips sync paths (ongoing stream only). Auth/rules checked before proxy in all handlers. `shouldProxyPath(path)`/`shouldRejectPath(path)` replace `isReplica` checks. `emitsToRepl`/`pullsFromPrimary` getters for compact/bootstrap decisions. Stable `replicaId` from config hash. Falls back to `role` when `paths` absent (backward compat).
 - **KeyAuth integration guide**: `docs/keyauth-integration.md` — flows for signup, signin, new device, autoAuth, IAM roles, common mistakes.
 - **Para-chat integration guide**: `docs/para-chat-integration.md` — how para-chat uses BodDB: per-path topology, VFS, KeyAuth, caching, file sync.
 - **KeyAuth**: `KeyAuthEngine` — portable Ed25519 identity & IAM. Identity hierarchy: Root (server-level, key on filesystem), Account (portable, password-encrypted private key in DB or device-generated), Device (delegate, linked via password unlock). Challenge-response auth: server sends nonce → client signs with Ed25519 → server verifies + creates session. Self-signed tokens (no JWT lib): `base64url(payload).base64url(Ed25519_sign)`. Data model at `_auth/` prefix (protected from external writes). Device reverse-index at `_auth/deviceIndex/{dfp}` for O(1) lookup. Password change is atomic (single `db.update()`). IAM: roles with path-based permissions, account role assignment. `_auth/` excluded from replication. Transport guards: `auth-link-device` and `auth-change-password` require authenticated session; non-root users can only change own password. **Device registration**: `registerDevice(publicKey)` — client-generated keypair, no password, idempotent; `allowOpenRegistration: false` requires authenticated session. **Browser crypto**: `keyAuth.browser.ts` uses `@noble/ed25519` with DER↔raw key bridge for server compatibility. **BodClient autoAuth**: `autoAuth: true` auto-generates keypair (localStorage), registers, authenticates — zero-config device identity. `client.auth.*` convenience methods for all auth ops. **IAM transport ops**: `auth-create-role`, `auth-delete-role`, `auth-update-roles` (root only), `auth-list-accounts`, `auth-list-roles`. Device accounts (no encrypted key) safely reject `linkDevice`/`changePassword`.

package/admin/admin.ts

@@ -30,7 +30,7 @@ export function startAdminUI(options?: { port?: number; serverUrl?: string }) {
       const url = new URL(req.url);
 
       // Proxy API calls to the BodDB server
-      if (url.pathname.startsWith('/db/') || url.pathname.startsWith('/files/') || url.pathname.startsWith('/sse/')) {
+      if (url.pathname.startsWith('/db/') || url.pathname.startsWith('/files/') || url.pathname.startsWith('/sse/') || url.pathname.startsWith('/replication')) {
         const target = httpBase + url.pathname + url.search;
         return fetch(target, { method: req.method, headers: req.headers, body: req.body });
       }

package/admin/ui.html

@@ -11,7 +11,6 @@
   #metrics-bar { display: flex; background: #0a0a0a; border-bottom: 1px solid #2a2a2a; flex-shrink: 0; align-items: stretch; width: 100%; }
   .metric-card { display: flex; flex-direction: column; padding: 5px 10px 4px; border-right: 1px solid #181818; min-width: 140px; flex-shrink: 0; gap: 1px; overflow: hidden; }
   .metric-card:last-child { border-right: none; width: auto; }
-  .metric-right { margin-left: auto; }
   .metric-top { display: flex; justify-content: space-between; align-items: baseline; width: 100%; }
   .metric-label { font-size: 9px; color: #555; text-transform: uppercase; letter-spacing: 0.5px; }
   .metric-value { font-size: 13px; color: #4ec9b0; font-weight: bold; min-width: 5ch; text-align: right; font-variant-numeric: tabular-nums; }
@@ -31,15 +30,18 @@
   #tree-header span { color: #555; font-size: 11px; }
   #tree-container { flex: 1; overflow-y: auto; padding: 4px; }
   #tree-container details { margin-left: 12px; }
-  #tree-container summary { cursor: pointer; padding: 2px 4px; border-radius: 3px; white-space: nowrap; overflow: hidden; text-overflow: ellipsis; color: #4ec9b0; list-style: none; }
-  #tree-container summary::before { content: '▶'; font-size: 10px; color: #666; margin-right: 5px; display: inline-block; transition: transform 0.15s; }
+  #tree-container summary { cursor: pointer; padding: 2px 4px; border-radius: 3px; color: #4ec9b0; list-style: none; display: flex; align-items: center; white-space: nowrap; overflow: hidden; }
+  #tree-container summary::before { content: '▶'; font-size: 10px; color: #666; margin-right: 5px; flex-shrink: 0; display: inline-block; transition: transform 0.15s; }
   details[open] > summary::before { transform: rotate(90deg); color: #aaa; }
+  #tree-container summary .tree-label { overflow: hidden; text-overflow: ellipsis; white-space: nowrap; min-width: 0; }
+  #tree-container summary .ttl-badge, #tree-container summary .count-badge { flex-shrink: 0; margin-left: 4px; }
   #tree-container summary:hover { background: #1e1e1e; }
   .tree-leaf { padding: 2px 4px 2px 16px; cursor: pointer; border-radius: 3px; color: #9cdcfe; display: flex; gap: 4px; align-items: baseline; overflow: hidden; }
   .tree-leaf:hover { background: #1e1e1e; }
   .tree-val { color: #ce9178; flex: 1; overflow: hidden; text-overflow: ellipsis; white-space: nowrap; font-size: 11px; }
   .tree-key { color: #4ec9b0; flex-shrink: 0; }
   .ttl-badge { font-size: 9px; padding: 0 4px; border-radius: 3px; background: #4d3519; color: #d4a054; flex-shrink: 0; }
+  .count-badge { font-size: 9px; padding: 0 4px; border-radius: 3px; background: #1e2d3d; color: #569cd6; flex-shrink: 0; }
   @keyframes treeFlash { 0%,100% { background: transparent; } 30% { background: rgba(86,156,214,0.25); } }
   .flash { animation: treeFlash 1.2s ease-out; border-radius: 3px; }
 
@@ -127,16 +129,18 @@
     <div class="metric-top"><span class="metric-label">Ping</span><span class="metric-value" id="s-ping">—</span></div>
     <canvas class="metric-canvas" id="g-ping" width="100" height="28"></canvas>
   </div>
-  <div class="metric-card" id="repl-card" style="border-left:1px solid #282828;display:none;width:180px">
-    <div class="metric-top"><span class="metric-label">Replication</span><span class="metric-value dim" id="s-repl-role">—</span></div>
-    <div style="margin-top:4px;font-size:10px" id="s-repl-sources"></div>
-  </div>
-  <div class="metric-card metric-right" style="border-left:1px solid #282828;justify-content:space-between">
+  <div style="margin-left:auto;display:flex;flex-shrink:0">
+    <div class="metric-card" id="repl-card" style="border-left:1px solid #282828;display:none;width:180px">
+      <div class="metric-top"><span class="metric-label">Replication</span><span class="metric-value dim" id="s-repl-role">—</span></div>
+      <div style="margin-top:4px;font-size:10px" id="s-repl-sources"></div>
+    </div>
+    <div class="metric-card" style="border-left:1px solid #282828;justify-content:space-between">
     <div class="metric-top"><span class="metric-label">Uptime</span><span class="metric-value dim" id="s-uptime">—</span></div>
     <div style="font-size:10px;color:#555;display:flex;justify-content:space-between"><span id="s-ts">—</span><span>v<span id="s-version">—</span></span></div>
     <div><span class="metric-label">WS<span id="ws-dot"></span></span> <span style="font-size:10px;color:#555"><span id="s-clients">0</span> clients · <span id="s-subs">0</span> subs</span></div>
     <div><button id="stats-toggle" class="sm" onclick="toggleStats()" title="Toggle server stats collection">Stats: ON</button></div>
   </div>
+  </div>
 </div>
 
 <div id="main">
@@ -1455,7 +1459,8 @@ function renderChildren(children, parentPath) {
       html += `<div class="tree-leaf" data-path="${escHtml(path)}" onclick="selectPath('${path.replace(/'/g, "\\'")}')"><span class="tree-key">${escHtml(ch.key)}</span>${ttlBadge}<span class="tree-val">${escHtml(String(display ?? ''))}</span></div>`;
     } else {
       const isOpen = _restoredOpenPaths.has(path);
-      html += `<details data-path="${escHtml(path)}"${isOpen ? ' open' : ''}><summary><span onclick="selectPath('${path.replace(/'/g, "\\'")}')">${escHtml(ch.key)}${ttlBadge}</span></summary><div class="tree-children" data-parent="${escHtml(path)}"></div></details>`;
+      const countBadge = ch.count != null ? `<span class="count-badge">${ch.count}</span>` : '';
+      html += `<details data-path="${escHtml(path)}"${isOpen ? ' open' : ''}><summary><span class="tree-label" onclick="selectPath('${path.replace(/'/g, "\\'")}')">${escHtml(ch.key)}</span>${ttlBadge}${countBadge}</summary><div class="tree-children" data-parent="${escHtml(path)}"></div></details>`;
     }
   }
   return html;
@@ -1534,8 +1539,6 @@ async function expandNode(details, isRefresh) {
 }
 
 async function refreshPath(path) {
-  // Re-fetch the nearest loaded ancestor and update its children in-place
-  // For simplicity, find the <details> or root and re-expand
   const parts = path.split('/');
   let target = '';
   // Walk up to find the deepest loaded ancestor
@@ -1545,8 +1548,17 @@ async function refreshPath(path) {
   }
 
   if (!target) {
-    // Refresh root
-    return loadTree([path]);
+    // No loaded ancestor — try to update just the root-level item without full reload
+    const topKey = parts[0];
+    const container = document.getElementById('tree-container');
+    const existing = container.querySelector(`[data-path="${CSS.escape(topKey)}"]`);
+    if (!existing) {
+      // Truly new top-level key — must reload root
+      return loadTree([path]);
+    }
+    // Root item exists but children weren't loaded — just flash it
+    flashPaths([path]);
+    return;
   }
 
   // Re-fetch this node and all its children
@@ -1557,6 +1569,9 @@ async function refreshPath(path) {
   if (det && det.open) {
     await expandNode(det, true);
     flashPaths([path]);
+  } else {
+    // Node exists but is collapsed — just flash it
+    flashPaths([path]);
   }
 }
 

package/docs/para-chat-integration.md

@@ -68,12 +68,13 @@ const db = new BodDB({
     role: 'primary',           // fallback for unconfigured paths
     primaryUrl: repl.remoteUrl, // wss://bod.ee/db
     paths: [
-      { path: '_vfs', mode: 'primary' },      // local files are authoritative
-      { path: '_auth', mode: 'replica' },      // bod.ee is auth authority
-      { path: 'config', mode: 'sync' },        // bidirectional app config
-      { path: 'storage', mode: 'primary' },    // local collections (notifications, etc.)
+      { path: '_vfs', mode: 'primary' },          // local files are authoritative
+      { path: '_auth', mode: 'replica' },          // bod.ee is auth authority
+      { path: '_auth/sessions', mode: 'primary' }, // sessions are local (longest-prefix wins)
+      { path: '_auth/server', mode: 'primary' },   // server keypair is local
+      { path: 'config', mode: 'sync' },            // bidirectional app config
+      { path: 'storage', mode: 'primary' },        // local collections (notifications, etc.)
     ],
-    excludePrefixes: ['_repl', '_streams', '_mq'],
   },
 });
 ```
@@ -93,7 +94,7 @@ const db = new BodDB({
 1. **`_auth` writes are proxied** — `createAccount`, `linkDevice` go through bod.ee automatically. No separate HTTP call needed.
 2. **`_vfs` emits to remote** — file metadata changes push to bod.ee via `_repl` stream. No manual upload sync.
 3. **Bootstrap is selective** — only `_auth` (replica) pulls from remote on connect. `_vfs` (primary) and `storage` (primary) keep local state.
-4. **`_auth/sessions` and `_auth/server`** — automatically excluded from replication (`_auth` prefix excluded by default; replica mode pulls from remote but these internal paths are local-only).
+4. **`_auth/sessions` and `_auth/server`** — kept local via explicit sub-path overrides (`mode: 'primary'`). Longest-prefix match ensures `_auth/sessions/x` resolves to `primary` even though `_auth` is `replica`.
 
 ---
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@bod.ee/db",
-  "version": "0.12.4",
+  "version": "0.12.8",
   "module": "index.ts",
   "type": "module",
   "exports": {
@@ -23,6 +23,7 @@
     "admin:remote": "bun run admin/proxy.ts",
     "serve": "bun run cli.ts",
     "start": "bun run cli.ts config.ts",
+    "start-admin": "bunx concurrently -n server,admin -c cyan,yellow \"bun run cli.ts config.ts\" \"bun run admin/admin.ts\"",
     "publish-lib": "bun publish --access public",
     "mcp": "bun run mcp.ts --stdio",
     "deploy": "bun run deploy/deploy.ts boddb deploy",

package/src/client/BodClient.ts

@@ -203,9 +203,11 @@ export class BodClient {
           this.scheduleReconnect();
         }
 
-        if (!this.connectPromise) return;
-        this.connectPromise = null;
-        // Only reject if this was the initial connect
+        // Reject initial connect promise so callers don't hang forever
+        if (this.connectPromise) {
+          this.connectPromise = null;
+          reject(new Error(`WebSocket connection failed (${this.options.url})`));
+        }
       };
 
       ws.onerror = () => {

package/src/server/BodDB.ts

@@ -541,6 +541,7 @@ export class BodDB {
     this.stop();
     this.subs.clear();
     this.storage.close();
+    this.log.close();
   }
 
   private snapshotExisting(path: string): Set<string> {

package/src/server/ReplicationEngine.ts

@@ -268,6 +268,7 @@ export class ReplicationEngine {
       sc.client.disconnect();
     }
     this.sourceConns = [];
+    this._pendingReplEvents = null;
   }
 
   /** Proxy a write operation to the primary (replica mode) */
@@ -296,7 +297,10 @@ export class ReplicationEngine {
 
   private startPrimary(): void {
     this.unsubWrite = this.db.onWrite((ev: WriteEvent) => {
-      this.emit(ev);
+      // Defer emit — onWrite fires inside the SQLite write transaction,
+      // and db.push('_repl') inside that transaction gets silently dropped.
+      const evCopy = { ...ev };
+      setTimeout(() => { if (this._started) this.emit(evCopy); }, 0);
     });
 
     // Compact on startup
@@ -320,22 +324,31 @@ export class ReplicationEngine {
 
   /** Buffer replication events during transactions, emit immediately otherwise */
   private emit(ev: WriteEvent): void {
-    if (this._emitting) return;
+    // _repl writes must never emit to _repl (infinite loop), regardless of config
+    if (ev.path.startsWith('_repl')) return;
+    if (this._emitting) {
+      this.log.debug('emit: skipped (re-entrant)', { path: ev.path });
+      return;
+    }
     if (this.router) {
-      // Single resolve: check exclusion override + shouldEmit together
       const resolved = this.router.resolve(ev.path);
       const isConfigured = resolved.path !== '';
-      // If in excludePrefixes but not explicitly configured, skip
-      if (!isConfigured && this.options.excludePrefixes.some(p => ev.path.startsWith(p))) return;
+      if (!isConfigured && this.options.excludePrefixes.some(p => ev.path.startsWith(p))) {
+        this.log.debug('emit: skipped (excluded, not configured)', { path: ev.path });
+        return;
+      }
       const mode = resolved.mode;
-      if (mode !== 'primary' && mode !== 'sync' && mode !== 'writeonly') return;
+      if (mode !== 'primary' && mode !== 'sync' && mode !== 'writeonly') {
+        this.log.debug('emit: skipped (mode)', { path: ev.path, mode });
+        return;
+      }
     } else {
       if (this.isExcluded(ev.path)) return;
     }
 
-    // If buffering (transaction in progress), collect events
     if (this._pendingReplEvents) {
       this._pendingReplEvents.push(ev);
+      this.log.debug('emit: buffered (batch)', { path: ev.path });
       return;
     }
 
@@ -349,12 +362,13 @@ export class ReplicationEngine {
       const seq = this._seq++;
       const idempotencyKey = `${replEvent.ts}:${seq}:${ev.path}`;
       this.db.push('_repl', replEvent, { idempotencyKey });
-
+      this.log.info('_repl emit', { seq, op: ev.op, path: ev.path });
+    } catch (e: any) {
+      this.log.error('_repl emit failed', { path: ev.path, error: e.message });
     } finally {
       this._emitting = false;
     }
 
-    // Auto-compact on write threshold (outside _emitting guard so notifications flow normally)
     this._emitCount++;
     const threshold = this.options.autoCompactThreshold;
     if (threshold > 0 && this._emitCount >= threshold) {
@@ -408,12 +422,16 @@ export class ReplicationEngine {
 
     // Subscribe to ongoing events, filter by paths
     const groupId = this.options.replicaId!;
+    this.log.info('Subscribing to _repl stream', { groupId, pathPrefixes });
     this.unsubStream = this.client.stream('_repl', groupId).on((events) => {
+      this.log.info('_repl events received', { count: events.length });
       this.db.setReplaying(true);
       try {
         for (const e of events) {
           const ev = e.val() as ReplEvent;
-          if (this.matchesPathPrefixes(ev.path, pathPrefixes)) {
+          const matched = this.matchesPathPrefixes(ev.path, pathPrefixes);
+          this.log.info('_repl event', { op: ev.op, path: ev.path, matched });
+          if (matched) {
             this.applyEvent(ev);
           }
           this.client!.stream('_repl', groupId).ack(e.key).catch(() => {});
@@ -582,7 +600,7 @@ export class ReplicationEngine {
   private applyEvent(ev: ReplEvent, source?: ReplicationSource): void {
     const path = source ? this.remapPath(ev.path, source) : ev.path;
     // Defense-in-depth: skip events for paths we shouldn't apply (primary/writeonly)
-    if (!source && this.router && !this.router.shouldApply(path)) return;
+    if (this.router && !this.router.shouldApply(path)) return;
     switch (ev.op) {
       case 'set':
         this.db.set(path, ev.value, ev.ttl ? { ttl: ev.ttl } : undefined);

package/src/server/StorageEngine.ts

@@ -137,7 +137,7 @@ export class StorageEngine {
   }
 
   /** Get immediate children of a path (one level deep). Returns { key, isLeaf, value? }[] */
-  getShallow(path?: string): Array<{ key: string; isLeaf: boolean; value?: unknown; ttl?: number }> {
+  getShallow(path?: string): Array<{ key: string; isLeaf: boolean; value?: unknown; ttl?: number; count?: number }> {
     const prefix = path ? path + '/' : '';
     const end = prefix + '\uffff';
     const rows = (prefix
@@ -145,7 +145,7 @@ export class StorageEngine {
       : this.db.prepare('SELECT path, value, expires_at FROM nodes ORDER BY path').all()
     ) as Array<{ path: string; value: string; expires_at: number | null }>;
 
-    const children: Array<{ key: string; isLeaf: boolean; value?: unknown; ttl?: number }> = [];
+    const children: Array<{ key: string; isLeaf: boolean; value?: unknown; ttl?: number; count?: number }> = [];
     const seen = new Set<string>();
     for (const row of rows) {
       const rest = row.path.slice(prefix.length);
@@ -158,10 +158,18 @@ export class StorageEngine {
         if (row.expires_at) entry.ttl = Math.max(0, row.expires_at - Math.floor(Date.now() / 1000));
         children.push(entry);
       } else {
-        // Check if any child in this branch has TTL
+        // Check if any child in this branch has TTL; count direct children
         const branchPrefix = prefix + key + '/';
         const hasTTL = rows.some(r => r.path.startsWith(branchPrefix) && r.expires_at);
-        children.push(hasTTL ? { key, isLeaf: false, ttl: -1 } : { key, isLeaf: false });
+        const directChildren = new Set<string>();
+        for (const r of rows) {
+          if (!r.path.startsWith(branchPrefix)) continue;
+          const seg = r.path.slice(branchPrefix.length).split('/')[0];
+          if (seg) directChildren.add(seg);
+        }
+        const entry: { key: string; isLeaf: boolean; ttl?: number; count?: number } = { key, isLeaf: false, count: directChildren.size };
+        if (hasTTL) entry.ttl = -1;
+        children.push(entry);
       }
     }
     return children;

package/src/server/Transport.ts

@@ -253,6 +253,49 @@ export class Transport {
       })();
     }
 
+    // Replication REST routes
+    if (url.pathname.startsWith('/replication')) {
+      return (async () => {
+        const repl = this.db.replication;
+        if (!repl) {
+          if (req.method === 'GET' && url.pathname === '/replication') {
+            return Response.json({ ok: true, role: 'primary', started: false, seq: 0, topology: null, sources: [], synced: {} });
+          }
+          return Response.json({ ok: false, error: 'Replication not configured' }, { status: 503 });
+        }
+        if (req.method === 'GET' && url.pathname === '/replication') {
+          const s = repl.stats();
+          // Build synced snapshot: read local copies of each source's paths
+          const synced: Record<string, unknown> = {};
+          for (const src of (s.sources ?? [])) {
+            for (const p of (src.paths ?? [])) {
+              try { synced[p] = this.db.get(p); } catch {}
+            }
+          }
+          return Response.json({ ok: true, ...s, synced });
+        }
+        if (req.method === 'POST' && url.pathname === '/replication/source-write') {
+          try {
+            const body = await req.json() as { path: string; value: unknown };
+            await repl.proxyWrite({ op: 'set', path: body.path, value: body.value });
+            return Response.json({ ok: true });
+          } catch (e: any) {
+            return Response.json({ ok: false, error: e.message }, { status: 500 });
+          }
+        }
+        if (req.method === 'DELETE' && url.pathname.startsWith('/replication/source-delete/')) {
+          const path = url.pathname.slice('/replication/source-delete/'.length);
+          try {
+            await repl.proxyWrite({ op: 'delete', path });
+            return Response.json({ ok: true });
+          } catch (e: any) {
+            return Response.json({ ok: false, error: e.message }, { status: 500 });
+          }
+        }
+        return Response.json({ ok: false, error: 'Not found' }, { status: 404 });
+      })();
+    }
+
     // VFS REST routes
     if (this.db.vfs && url.pathname.startsWith('/files/')) {
       const vfsPath = normalizePath(url.pathname.slice(7));
@@ -515,10 +558,11 @@ export class Transport {
             }
 
             case 'batch': {
-              // Upfront rules check before proxy (defense-in-depth)
+              // Upfront auth prefix + rules check before proxy (defense-in-depth)
               for (const batchOp of msg.operations) {
                 const opPaths = batchOp.op === 'update' ? Object.keys(batchOp.updates) : [batchOp.path];
                 for (const p of opPaths) {
+                  if (guardAuthPrefix(p)) return;
                   if (self.rules && !self.rules.check('write', p, ws.data.auth)) {
                     return error(`Permission denied for ${p}`, Errors.PERMISSION_DENIED);
                   }

package/src/shared/logger.ts

@@ -3,6 +3,9 @@
  * Disabled by default. Enable via `log` option in BodDBOptions.
  */
 
+import { createWriteStream, mkdirSync } from 'fs';
+import type { WriteStream } from 'fs';
+
 export type LogLevel = 'debug' | 'info' | 'warn' | 'error';
 
 export interface LogConfig {
@@ -12,6 +15,12 @@ export interface LogConfig {
   level?: LogLevel;
   /** Enable specific components: ['storage', 'transport', 'subs', 'replication', 'stats', 'keyauth'] or '*' for all */
   components?: string[] | '*';
+  /**
+   * Directory to write log files into. Each run creates a new file named
+   * `bod-YYYYMMDD-HHmmss.log`. If omitted, logs go to console only.
+   * Only active when `enabled: true`.
+   */
+  logDir?: string;
 }
 
 const LEVELS: Record<LogLevel, number> = { debug: 0, info: 1, warn: 2, error: 3 };
@@ -21,11 +30,31 @@ export class Logger {
   private minLevel: number;
   private components: Set<string> | '*';
   private _cache = new Map<string, ComponentLogger>();
+  private _stream: WriteStream | null = null;
+  private _streamDead = false;
 
   constructor(config?: LogConfig) {
     this.enabled = config?.enabled ?? false;
     this.minLevel = LEVELS[config?.level ?? 'info'];
     this.components = config?.components === '*' ? '*' : new Set(config?.components ?? []);
+    if (this.enabled && config?.logDir) {
+      try {
+        mkdirSync(config.logDir, { recursive: true });
+        const d = new Date();
+        const ts = `${d.getFullYear()}${pad(d.getMonth() + 1)}${pad(d.getDate())}-${pad(d.getHours())}${pad(d.getMinutes())}${pad(d.getSeconds())}`;
+        const file = `${config.logDir}/bod-${ts}.log`;
+        this._stream = createWriteStream(file, { flags: 'a' });
+        this._stream.on('error', (err) => {
+          if (!this._streamDead) {
+            console.error(`[BodDB logger] File write error, disabling disk logging: ${err.message}`);
+            this._streamDead = true;
+            this._stream = null;
+          }
+        });
+      } catch (err: any) {
+        console.error(`[BodDB logger] Failed to open log file in "${config.logDir}": ${err.message}`);
+      }
+    }
   }
 
   forComponent(name: string): ComponentLogger {
@@ -50,9 +79,20 @@ export class Logger {
     } else {
       console[level === 'debug' ? 'log' : level](prefix, msg);
     }
+    if (this._stream && !this._streamDead) {
+      const dataPart = data !== undefined ? ' ' + JSON.stringify(data) : '';
+      this._stream.write(`${prefix} ${msg}${dataPart}\n`);
+    }
+  }
+
+  close(): void {
+    this._stream?.end();
+    this._stream = null;
   }
 }
 
+function pad(n: number): string { return n.toString().padStart(2, '0'); }
+
 export class ComponentLogger {
   readonly isDebug: boolean;
   constructor(private logger: Logger, private component: string) {

package/tests/optimization.test.ts

@@ -325,7 +325,7 @@ describe('B5: Cursor-Based Stream Materialize', () => {
 });
 
 describe('B6: Batched Replication Events', () => {
-  test('transaction batches repl events', () => {
+  test('transaction batches repl events', async () => {
     const db = new BodDB({ path: ':memory:', replication: { role: 'primary' } });
     db.replication!['_started'] = true;
     db.replication!['startPrimary']();
@@ -336,6 +336,7 @@ describe('B6: Batched Replication Events', () => {
       tx.set('b', 2);
       tx.set('c', 3);
     });
+    await new Promise(r => setTimeout(r, 50));
 
     // All 3 should have been emitted to _repl
     const replEvents = db.storage.query('_repl');
@@ -344,12 +345,13 @@ describe('B6: Batched Replication Events', () => {
     db.close();
   });
 
-  test('non-transactional writes emit immediately', () => {
+  test('non-transactional writes emit immediately', async () => {
     const db = new BodDB({ path: ':memory:', replication: { role: 'primary' } });
     db.replication!['_started'] = true;
     db.replication!['startPrimary']();
 
     db.set('x', 'val');
+    await new Promise(r => setTimeout(r, 50));
     const replEvents = db.storage.query('_repl');
     expect(replEvents.length).toBe(1);
 

package/tests/repl-load.test.ts

@@ -119,12 +119,13 @@ describe('repl load test', () => {
 
   // ─── 3. Auto-compact under sustained write load ───
 
-  it('auto-compact keeps _repl bounded under sustained 10k writes', () => {
+  it('auto-compact keeps _repl bounded under sustained 10k writes', async () => {
     const { db } = primary({ compact: { maxCount: 200, keepKey: 'path' }, autoCompactThreshold: 500 });
 
     for (let i = 0; i < 10_000; i++) {
       db.set(`stream/key${i % 300}`, { round: Math.floor(i / 300), i });
     }
+    await new Promise(r => setTimeout(r, 50));
 
     const repl = db.get('_repl') as Record<string, any>;
     const count = repl ? Object.keys(repl).length : 0;
@@ -215,7 +216,7 @@ describe('repl load test', () => {
 
   // ─── 7. Heavy overwrite scenario: same 50 paths written 1000× each ───
 
-  it('50 paths × 1000 overwrites: compact deduplicates correctly', () => {
+  it('50 paths × 1000 overwrites: compact deduplicates correctly', async () => {
     const { db } = primary({ compact: { maxCount: 100, keepKey: 'path' }, autoCompactThreshold: 1000 });
 
     for (let round = 0; round < 1000; round++) {
@@ -223,6 +224,7 @@ describe('repl load test', () => {
         db.set(`hot/key${i}`, { round, value: round * 50 + i });
       }
     }
+    await new Promise(r => setTimeout(r, 50));
 
     const repl = db.get('_repl') as Record<string, any>;
     const count = repl ? Object.keys(repl).length : 0;

package/tests/repl-stream-bloat.test.ts

@@ -66,9 +66,10 @@ describe('_repl stream bloat', () => {
 
   // --- Accumulation ---
 
-  it('_repl grows unbounded without compaction', () => {
+  it('_repl grows unbounded without compaction', async () => {
     const { db } = createPrimary();
     fillRepl(db, 5000, 300);
+    await new Promise(r => setTimeout(r, 50));
     const repl = db.get('_repl') as Record<string, any>;
     expect(Object.keys(repl).length).toBe(5000);
   });
@@ -135,13 +136,14 @@ describe('_repl stream bloat', () => {
 
   // --- Auto-compact on write threshold ---
 
-  it('auto-compact triggers after N writes', () => {
+  it('auto-compact triggers after N writes', async () => {
     const { db } = createPrimary({ compact: { maxCount: 50, keepKey: 'path' }, autoCompactThreshold: 100 });
 
     // Write 250 entries — compact triggers at 100, 200; maxCount=50 keeps only 50 each time
     for (let i = 0; i < 250; i++) {
       db.set(`data/item${i}`, { value: i });
     }
+    await new Promise(r => setTimeout(r, 50));
 
     const repl = db.get('_repl') as Record<string, any>;
     const count = repl ? Object.keys(repl).length : 0;
@@ -155,6 +157,7 @@ describe('_repl stream bloat', () => {
   it('short requestTimeout causes streamMaterialize to fail on bloated _repl', async () => {
     const { db: primary, port: primaryPort } = createPrimary();
     fillRepl(primary, 10000, 1000); // 10k entries × 1KB = ~10MB response
+    await new Promise(r => setTimeout(r, 50));
 
     const replCount = Object.keys(primary.get('_repl') as Record<string, any>).length;
     expect(replCount).toBe(10000);
@@ -227,6 +230,7 @@ describe('_repl stream bloat', () => {
   it('compaction reduces _repl and speeds up bootstrap', async () => {
     const { db: primary, port: primaryPort } = createPrimary();
     fillRepl(primary, 5000, 500);
+    await new Promise(r => setTimeout(r, 50));
 
     const beforeCount = Object.keys(primary.get('_repl') as Record<string, any>).length;
     expect(beforeCount).toBe(5000);
@@ -252,7 +256,7 @@ describe('_repl stream bloat', () => {
 
   // --- Repeated writes ---
 
-  it('repeated writes to same paths bloat _repl with duplicates', () => {
+  it('repeated writes to same paths bloat _repl with duplicates', async () => {
     const { db } = createPrimary();
 
     for (let round = 0; round < 50; round++) {
@@ -260,6 +264,7 @@ describe('_repl stream bloat', () => {
         db.set(`config/setting${i}`, { value: round, updated: Date.now() });
       }
     }
+    await new Promise(r => setTimeout(r, 50));
 
     const repl = db.get('_repl') as Record<string, any>;
     const totalEntries = Object.keys(repl).length;

package/tests/replication-topology.test.ts

@@ -165,7 +165,7 @@ describe('Per-path replication topology', () => {
 
   // --- Emit filtering ---
 
-  it('primary-mode path emits to _repl', () => {
+  it('primary-mode path emits to _repl', async () => {
     const db = new BodDB({
       path: ':memory:',
       sweepInterval: 0,
@@ -175,16 +175,17 @@ describe('Per-path replication topology', () => {
       },
     });
     instances.push(db);
-    db.replication!.start();
+    await db.replication!.start();
 
     db.set('_vfs/file1', { data: 'hello' });
+    await new Promise(r => setTimeout(r, 50));
     const repl = db.get('_repl');
     expect(repl).toBeTruthy();
     const events = Object.values(repl as Record<string, any>);
     expect(events.some((e: any) => e.path === '_vfs/file1')).toBe(true);
   });
 
-  it('non-emitting modes do NOT emit', () => {
+  it('emitting modes + fallback all emit to _repl', async () => {
     const db = new BodDB({
       path: ':memory:',
       sweepInterval: 0,
@@ -197,12 +198,15 @@ describe('Per-path replication topology', () => {
       },
     });
     instances.push(db);
-    db.replication!.start();
+    await db.replication!.start();
 
     db.set('local/data', { v: 1 });
     db.set('telemetry/t1', { event: 'click' });
     db.set('other/data', { v: 2 });
 
+    // Wait for deferred setTimeout emits
+    await new Promise(r => setTimeout(r, 50));
+
     const repl = db.get('_repl');
     expect(repl).toBeTruthy();
     const events = Object.values(repl as Record<string, any>);
@@ -211,7 +215,43 @@ describe('Per-path replication topology', () => {
     expect(events.some((e: any) => e.path === 'other/data')).toBe(true);
   });
 
-  it('writeonly path emits', () => {
+  it('replica and readonly modes do NOT emit to _repl', async () => {
+    const pPort = getPort();
+    const primary = createNode({ port: pPort, replication: { role: 'primary' } });
+    await primary.replication!.start();
+
+    const db = new BodDB({
+      path: ':memory:',
+      sweepInterval: 0,
+      replication: {
+        role: 'primary',
+        primaryUrl: `ws://localhost:${pPort}`,
+        paths: [
+          { path: 'cached', mode: 'replica' },
+          { path: 'feeds', mode: 'readonly' },
+          { path: 'local', mode: 'primary' },
+        ],
+      },
+    });
+    instances.push(db);
+    await db.replication!.start();
+
+    // Write directly (bypassing transport proxy) to test emit filtering
+    db.set('local/data', { v: 1 });
+    db.set('cached/data', { v: 2 });
+    db.set('feeds/data', { v: 3 });
+
+    // Wait for deferred emits
+    await new Promise(r => setTimeout(r, 50));
+
+    const repl = db.get('_repl');
+    const events = repl ? Object.values(repl as Record<string, any>) : [];
+    expect(events.some((e: any) => e.path === 'local/data')).toBe(true);
+    expect(events.some((e: any) => e.path === 'cached/data')).toBe(false);
+    expect(events.some((e: any) => e.path === 'feeds/data')).toBe(false);
+  });
+
+  it('writeonly path emits', async () => {
     const db = new BodDB({
       path: ':memory:',
       sweepInterval: 0,
@@ -221,9 +261,10 @@ describe('Per-path replication topology', () => {
       },
     });
     instances.push(db);
-    db.replication!.start();
+    await db.replication!.start();
 
     db.set('telemetry/t1', { event: 'click' });
+    await new Promise(r => setTimeout(r, 50));
     const repl = db.get('_repl');
     expect(repl).toBeTruthy();
     const events = Object.values(repl as Record<string, any>);
@@ -796,7 +837,7 @@ describe('Per-path replication topology', () => {
   it('sync path written locally AND emitted (end-to-end)', async () => {
     const pPort = getPort();
     const primary = createNode({ port: pPort, replication: { role: 'primary' } });
-    primary.replication!.start();
+    await primary.replication!.start();
 
     const rPort = getPort();
     const replica = createNode({
@@ -813,6 +854,7 @@ describe('Per-path replication topology', () => {
     replica.set('config/theme', 'dark');
     expect(replica.get('config/theme')).toBe('dark');
 
+    await new Promise(r => setTimeout(r, 50));
     const repl = replica.get('_repl');
     expect(repl).toBeTruthy();
     const events = Object.values(repl as Record<string, any>);

package/tests/replication.test.ts

@@ -8,6 +8,7 @@ let nextPort = 24400 + Math.floor(Math.random() * 1000);
 describe('ReplicationEngine', () => {
   const instances: BodDB[] = [];
   const clients: BodClient[] = [];
+  const tick = () => new Promise(r => setTimeout(r, 50));
 
   afterEach(() => {
     for (const c of clients) c.disconnect();
@@ -48,7 +49,7 @@ describe('ReplicationEngine', () => {
   }
 
   // 1. Primary emits events to _repl on set/delete/update/push
-  it('primary emits replication events on set/delete/update/push', () => {
+  it('primary emits replication events on set/delete/update/push', async () => {
     const db = new BodDB({
       path: ':memory:',
       sweepInterval: 0,
@@ -61,6 +62,7 @@ describe('ReplicationEngine', () => {
     db.update({ 'users/u2/age': 30 });
     db.delete('users/u1');
     db.push('logs', { msg: 'hello' });
+    await tick();
 
     const replData = db.get('_repl');
     expect(replData).toBeTruthy();
@@ -76,7 +78,7 @@ describe('ReplicationEngine', () => {
   });
 
   // 2. Loop prevention — _replaying flag prevents re-emission
-  it('does not emit events when replaying', () => {
+  it('does not emit events when replaying', async () => {
     const db = new BodDB({
       path: ':memory:',
       sweepInterval: 0,
@@ -88,13 +90,14 @@ describe('ReplicationEngine', () => {
     db.setReplaying(true);
     db.set('users/u1', { name: 'Test' });
     db.setReplaying(false);
+    await tick();
 
     const replData = db.get('_repl');
     expect(replData).toBeNull();
   });
 
   // 3. Excluded prefixes are not replicated
-  it('skips excluded prefixes', () => {
+  it('skips excluded prefixes', async () => {
     const db = new BodDB({
       path: ':memory:',
       sweepInterval: 0,
@@ -105,6 +108,7 @@ describe('ReplicationEngine', () => {
 
     db.set('internal/config', { foo: 1 });
     db.set('users/u1', { name: 'Eli' });
+    await tick();
 
     const replData = db.get('_repl') as Record<string, any>;
     const entries = Object.values(replData);
@@ -160,7 +164,7 @@ describe('ReplicationEngine', () => {
   });
 
   // 7. Transaction events are replicated
-  it('transaction write events are replicated', () => {
+  it('transaction write events are replicated', async () => {
     const db = new BodDB({
       path: ':memory:',
       sweepInterval: 0,
@@ -173,6 +177,7 @@ describe('ReplicationEngine', () => {
       tx.set('a/1', 'one');
       tx.set('a/2', 'two');
     });
+    await tick();
 
     const replData = db.get('_repl') as Record<string, any>;
     const entries = Object.values(replData);
@@ -211,7 +216,7 @@ describe('ReplicationEngine', () => {
   });
 
   // 10. Multi-path update exclusion check (all paths checked, not just first)
-  it('update with mixed excluded/non-excluded paths emits only non-excluded', () => {
+  it('update with mixed excluded/non-excluded paths emits only non-excluded', async () => {
     const db = new BodDB({
       path: ':memory:',
       sweepInterval: 0,
@@ -221,6 +226,7 @@ describe('ReplicationEngine', () => {
     db.replication!.start();
 
     db.update({ '_internal/x': 1, 'data/y': 2 });
+    await tick();
 
     const replData = db.get('_repl') as Record<string, any>;
     const entries = Object.values(replData);
@@ -229,7 +235,7 @@ describe('ReplicationEngine', () => {
   });
 
   // 11. Sweep fires delete events for expired paths
-  it('sweep expired paths are replicated as deletes', () => {
+  it('sweep expired paths are replicated as deletes', async () => {
     const db = new BodDB({
       path: ':memory:',
       sweepInterval: 0,
@@ -242,6 +248,7 @@ describe('ReplicationEngine', () => {
     // Force expiry by manipulating the DB directly
     db.storage.db.prepare('UPDATE nodes SET expires_at = 1 WHERE path LIKE ?').run('sessions/s1%');
     db.sweep();
+    await tick();
 
     const replData = db.get('_repl') as Record<string, any>;
     const entries = Object.values(replData);
@@ -251,7 +258,7 @@ describe('ReplicationEngine', () => {
   });
 
   // 12. Recursion guard — emit doesn't infinite loop
-  it('emit does not cause infinite recursion', () => {
+  it('emit does not cause infinite recursion', async () => {
     const db = new BodDB({
       path: ':memory:',
       sweepInterval: 0,
@@ -262,6 +269,7 @@ describe('ReplicationEngine', () => {
 
     db.set('users/u1', 'test');
     expect(db.get('users/u1')).toBe('test');
+    await tick();
     const replData = db.get('_repl') as Record<string, any>;
     expect(Object.values(replData).length).toBe(1);
   });
@@ -333,6 +341,7 @@ describe('ReplicationEngine', () => {
     expect(local.get('ext/users/u1')).toEqual({ name: 'Eli' });
 
     local.set('mydata/x', 'hello');
+    await tick();
     const replData = local.get('_repl') as Record<string, any>;
     const entries = Object.values(replData);
     expect(entries.some((e: any) => e.path === 'mydata/x')).toBe(true);