bff-store 0.1.0

package/docs/REMOTE_STORAGE_CONFIG.md

@@ -0,0 +1,125 @@
+# Remote Storage 客户端配置存储后端
+
+## 背景
+
+用户希望在客户端通过 `remoteStorage()` 配置使用 jsonl 还是 mongodb，以及各自的参数。
+
+## 设计
+
+### 客户端 API
+
+```typescript
+// 使用 MongoDB
+const adapter = remoteStorage({
+  backend: 'mongodb',
+  mongoUrl: 'mongodb://user:pass@host:27017',
+  database: 'myapp'
+})
+
+// 使用 JSONL
+const adapter = remoteStorage({
+  backend: 'jsonl',
+  jsonlDir: '/tmp/my-app-data'
+})
+
+// 默认 (不指定 backend，走 BFF 默认配置)
+const adapter = remoteStorage()
+```
+
+### 请求流程
+
+1. 客户端发起请求，带 `backend` 类型和配置
+2. BFF 根据 `backend` 路由到 mongodb 或 jsonl adapter
+3. 每次请求都传配置参数
+
+### BFF 行为
+
+BFF 根据客户端请求中的 `backend` 类型，动态调用对应 adapter。BFF 不预建连接。
+
+## 实现
+
+### 1. `src/storage/adapters/remoteStorage.ts`
+
+更新 `RemoteStorageOptions` 类型：
+
+```typescript
+export interface RemoteStorageOptions {
+  baseUrl?: string;
+  entityId?: string;
+  // 新增: 存储后端配置
+  backend?: 'mongodb' | 'jsonl';
+  mongoUrl?: string;
+  mongoDb?: string;
+  jsonlDir?: string;
+}
+```
+
+更新 `remoteStorage()` 函数，将 backend 配置放入请求中。
+
+### 2. `src/storage/protocol.ts`
+
+更新 `RestStorageProtocol`，在请求中附带 backend 配置：
+
+```typescript
+buildGetUrl(key: string): string {
+  const url = `${this.baseUrl}/storage/get/${encodeURIComponent(key)}`;
+  // 附带 backend 配置
+  const params = new URLSearchParams();
+  if (this.entityId) params.set('entityId', this.entityId);
+  if (this.backend) params.set('backend', this.backend);
+  if (this.backend === 'mongodb' && this.mongoUrl) {
+    params.set('mongoUrl', this.mongoUrl);
+    params.set('mongoDb', this.mongoDb || 'jotai_state_store');
+  }
+  if (this.backend === 'jsonl' && this.jsonlDir) {
+    params.set('jsonlDir', this.jsonlDir);
+  }
+  // ...
+}
+```
+
+### 3. `src/server/handlers.ts`
+
+更新 handlers，从请求中解析 backend 配置，动态创建对应 adapter：
+
+```typescript
+async handleSet(req, res) {
+  const { key } = req.params;
+  const { value, backend, mongoUrl, mongoDb, jsonlDir } = await parseBody(req);
+
+  let storage;
+  if (backend === 'mongodb') {
+    const adapter = await mongodbStorage({ url: mongoUrl, database: mongoDb });
+    storage = adapter.storage;
+  } else if (backend === 'jsonl') {
+    const adapter = jsonlStorage({ dir: jsonlDir });
+    storage = adapter.storage;
+  } else {
+    // 使用默认 storage
+    storage = getDefaultStorage();
+  }
+
+  await storage.set(key, value);
+  // ...
+}
+```
+
+### 4. `src/server/router.ts`
+
+更新 router 传递 backend 相关参数。
+
+## 文件修改清单
+
+| 文件 | 修改内容 |
+|------|----------|
+| `src/storage/adapters/remoteStorage.ts` | 添加 backend 配置字段 |
+| `src/storage/protocol.ts` | 附带 backend 配置到请求 URL |
+| `src/server/handlers.ts` | 解析 backend 并动态路由 |
+| `src/server/router.ts` | 传递 backend 参数 |
+
+## 验证
+
+1. 构建后检查主 bundle 不含 mongodb/jsonl 代码
+2. 测试 `remoteStorage({ backend: 'mongodb', ... })` 能正常工作
+3. 测试 `remoteStorage({ backend: 'jsonl', ... })` 能正常工作
+4. 测试默认 `remoteStorage()` 仍能正常工作

package/docs/SIDECAR_SERVER.md

@@ -0,0 +1,184 @@
+# Embedded Sidecar API Server (BFF)
+
+## Overview
+
+Provides a lightweight HTTP server that acts as a BFF (Backend for Frontend) proxy for storage backends (JSONL/MongoDB). Designed for use in browser/Next.js environments where direct file system or database access isn't available.
+
+## Architecture
+
+```
+┌─────────────────────────────────────────────────────────┐
+│                    Main Application                      │
+│                   (Next.js/Node.js)                     │
+│                                                          │
+│  useStore() ───► remoteStorageAdapter ──► localhost:3847 │
+│                      (HTTP)              │               │
+└──────────────────────────────────────────┼───────────────┘
+                                           │
+                                           ▼
+                              ┌────────────────────────┐
+                              │   Embedded API Server  │
+                              │       (BFF Layer)      │
+                              │                        │
+                              │  /storage/get/:key     │
+                              │  /storage/set/:key     │
+                              │  /storage/delete/:key  │
+                              │  /storage/batch-get    │
+                              │  /storage/batch-set    │
+                              └────────────────────────┘
+```
+
+## Auto-Start Behavior
+
+The server starts **automatically** when using `remoteStorage()`:
+
+```typescript
+import { createStore, useStore, remoteStorage } from 'bff-store';
+
+// Server auto-starts on first createStore call
+const store = createStore('user-1', config, {
+  storage: remoteStorage()
+});
+```
+
+**Singleton pattern**: Only one server instance runs. Subsequent `createStore` calls reuse the existing server.
+
+## API Endpoints
+
+| Method | Endpoint | Body | Response |
+|--------|----------|------|----------|
+| GET | `/storage/get/:key?entityId=x` | - | `{ value: T }` |
+| POST | `/storage/set/:key?entityId=x` | `{ value: T }` | `{ success: true }` |
+| DELETE | `/storage/delete/:key?entityId=x` | - | `{ success: true }` |
+| POST | `/storage/batch-get?entityId=x` | `{ keys: string[] }` | `{ entries: {...} }` |
+| POST | `/storage/batch-set?entityId=x` | `{ entries: {...} }` | `{ success: true }` |
+| GET | `/health` | - | `{ status: 'ok' }` |
+
+## Server Options
+
+```typescript
+interface ServerOptions {
+  port?: number;        // Default: 3847
+  host?: string;       // Default: localhost
+  backend: 'jsonl' | 'mongodb';
+  jsonlDir?: string;   // For JSONL backend (default: ./data)
+  mongoUrl?: string;   // For MongoDB backend (required)
+  mongoDb?: string;    // For MongoDB backend (default: jotai_state_store)
+}
+```
+
+## Programmatic Usage
+
+### Auto-Start (Recommended)
+
+```typescript
+import { createStore, remoteStorage } from 'bff-store';
+
+// Server auto-starts with defaults (JSONL, port 3847, ./data dir)
+const store = createStore('user-1', config, {
+  storage: remoteStorage()
+});
+```
+
+### Customize Server Before First Use
+
+```typescript
+import { startServer, createStore, remoteStorage } from 'bff-store';
+
+// Start with custom settings
+await startServer({
+  backend: 'mongodb',
+  mongoUrl: 'mongodb://localhost:27017',
+  port: 5000,
+});
+
+// Now all remoteStorage() calls use this server
+const store = createStore('user-1', config, {
+  storage: remoteStorage()
+});
+```
+
+### Using startServer Directly
+
+```typescript
+import { startServer } from 'bff-store';
+
+const server = await startServer({
+  backend: 'jsonl',
+  port: 3847,
+  jsonlDir: './data',
+});
+
+// Server runs until Ctrl+C
+```
+
+## Remote Storage Adapter
+
+```typescript
+import { remoteStorage } from 'bff-store';
+
+// Default: http://localhost:3847
+const adapter = remoteStorage();
+
+// Custom server URL
+const adapter = remoteStorage({ baseUrl: 'http://custom:9999' });
+
+// With default entityId
+const adapter = remoteStorage({ entityId: 'user-123' });
+```
+
+## Graceful Shutdown
+
+The server handles `SIGINT` and `SIGTERM` signals:
+- Stops accepting new connections
+- Closes existing connections
+- Exits cleanly after 5 seconds max
+
+## File Structure
+
+```
+src/server/
+├── index.ts          # Main entry, singleton manager, startServer()
+├── router.ts         # HTTP pattern-based router
+├── handlers.ts        # Storage operation handlers
+└── entityIdCache.ts  # Multi-tenant storage cache
+```
+
+### Router (`router.ts`)
+
+Pattern-based HTTP router with named parameter extraction.
+
+```typescript
+router.get('/storage/get/:key', handler);  // :key is a named param
+router.post('/storage/set/:key', handler);
+router.delete('/storage/delete/:key', handler);
+router.post('/storage/batch-get', handler);
+router.post('/storage/batch-set', handler);
+```
+
+### EntityIdCache (`entityIdCache.ts`)
+
+Caches JSONL storage adapters per `entityId` to avoid recreation:
+
+```typescript
+const cache = new EntityIdCache({ dir: './data' });
+const storage = cache.getStorage('user-123');  // Gets or creates
+```
+
+### Handlers (`handlers.ts`)
+
+Request handlers for storage operations. Each handler reads `entityId` from URL query params:
+
+```typescript
+const handlers = createStorageHandlers({
+  getStorage: (entityId?: string) => entityIdCache.getStorage(entityId),
+});
+```
+
+## Testing
+
+```bash
+npm test -- tests/server.test.ts
+```
+
+Server tests create a temporary directory for JSONL storage, start the server on port 3849, and verify all endpoints work correctly.

package/package.json

@@ -0,0 +1,62 @@
+{
+  "name": "bff-store",
+  "version": "0.1.0",
+  "description": "A jotai-based state management library with pluggable storage adapters",
+  "main": "dist/index.js",
+  "module": "dist/index.mjs",
+  "types": "dist/index.d.ts",
+  "exports": {
+    ".": {
+      "types": "./dist/index.d.ts",
+      "import": "./dist/index.mjs",
+      "require": "./dist/index.js"
+    },
+    "./jsonl": {
+      "types": "./dist/storage/jsonl-entry.d.ts",
+      "import": "./dist/storage/jsonl-entry.mjs",
+      "require": "./dist/storage/jsonl-entry.js"
+    },
+    "./mongodb": {
+      "types": "./dist/storage/mongodb-entry.d.ts",
+      "import": "./dist/storage/mongodb-entry.mjs",
+      "require": "./dist/storage/mongodb-entry.js"
+    },
+    "./server": {
+      "types": "./dist/server/entry.d.ts",
+      "import": "./dist/server/entry.mjs",
+      "require": "./dist/server/entry.js"
+    }
+  },
+  "scripts": {
+    "build": "tsup && cp package.json dist/package.json && node scripts/adapt-dist-package.js && for f in dist/*.d.mts; do cp \"$f\" \"${f%.d.mts}.d.ts\"; done",
+    "dev": "tsup --watch",
+    "test": "vitest",
+    "start": "node dist/server/cli.js"
+  },
+  "keywords": [
+    "jotai",
+    "state-management",
+    "react",
+    "storage"
+  ],
+  "peerDependencies": {
+    "jotai": ">=2.0.0",
+    "react": ">=18.0.0"
+  },
+  "devDependencies": {
+    "@testing-library/dom": "^10.4.1",
+    "@testing-library/react": "^16.3.2",
+    "@types/node": "^20.0.0",
+    "@types/react": "^19.2.16",
+    "jotai": "^2.6.0",
+    "jsdom": "^29.1.1",
+    "react": "^18.2.0",
+    "react-dom": "^18.2.0",
+    "tsup": "^8.0.0",
+    "typescript": "^5.0.0",
+    "vitest": "^1.0.0"
+  },
+  "dependencies": {
+    "mongodb": "^6.21.0"
+  }
+}

package/scripts/adapt-dist-package.js

@@ -0,0 +1,33 @@
+const fs = require('fs');
+const path = require('path');
+
+const distPkgPath = path.resolve(__dirname, '..', 'dist', 'package.json');
+
+if (!fs.existsSync(distPkgPath)) {
+  console.error('dist/package.json not found — skipping path adaptation');
+  process.exit(1);
+}
+
+const pkg = JSON.parse(fs.readFileSync(distPkgPath, 'utf-8'));
+
+// Strip ./dist/ prefix from top-level entry fields
+for (const field of ['main', 'module', 'types']) {
+  if (typeof pkg[field] === 'string') {
+    pkg[field] = pkg[field].replace(/^dist\//, '');
+  }
+}
+
+// Strip ./dist/ prefix from exports map
+if (pkg.exports) {
+  for (const key of Object.keys(pkg.exports)) {
+    const entry = pkg.exports[key];
+    for (const cond of ['types', 'import', 'require', 'default']) {
+      if (typeof entry[cond] === 'string') {
+        entry[cond] = entry[cond].replace(/^\.\/dist\//, './');
+      }
+    }
+  }
+}
+
+fs.writeFileSync(distPkgPath, JSON.stringify(pkg, null, 2) + '\n');
+console.log('dist/package.json paths adapted for local resolution');

package/src/atomCreator.ts

@@ -0,0 +1,76 @@
+import { atom, getDefaultStore } from 'jotai';
+import type { AtomConfig, PersistedAtomWithLoading } from './types';
+import type { Storage } from './storage/base';
+import { DebouncerMap } from './debouncer';
+
+/**
+ * Module-level debouncer map for all persisted atoms.
+ * Each unique key (atom config key) gets its own debouncer.
+ */
+const debouncerMap = new DebouncerMap();
+
+/**
+ * Creates a persisted atom with loading state tracking
+ */
+export function createPersistedAtom<T>(
+  config: AtomConfig<T>,
+  entityId: string,
+  storage: Storage,
+  options?: { immediate?: boolean; debounceMs?: number }
+): PersistedAtomWithLoading<T> {
+  const baseAtom = atom<T>(config.defaultValue);
+  const loadingAtom = atom<boolean>(true);
+
+  // Load initial value on mount
+  baseAtom.onMount = (setValue) => {
+    storage
+      .get<T>(config.key)
+      .then((value) => {
+        if (value !== null && value !== undefined) {
+          setValue(value);
+        }
+      })
+      .catch(console.error)
+      .finally(() => {
+        // Mark as loaded
+        setTimeout(() => {
+          const store = getDefaultStore();
+          store.set(loadingAtom, false);
+        }, 0);
+      });
+  };
+
+  // Create write atom with persistence
+  const writeAtom = atom(
+    (get) => get(baseAtom),
+    (get, set, update: T | ((prev: T) => T)) => {
+      const newValue =
+        typeof update === 'function'
+          ? (update as (prev: T) => T)(get(baseAtom))
+          : update;
+
+      // Update local state first
+      set(baseAtom, newValue);
+
+      // Save to storage
+      if (options?.immediate) {
+        // Immediate save for critical data
+        storage.set(config.key, newValue).catch(console.error);
+      } else {
+        // Debounced save for normal data
+        // Use entityId:key as the debounce key to avoid cross-store interference
+        const debounceKey = `${entityId}:${config.key}`;
+        const debounceMs = options?.debounceMs ?? 800;
+        const saveFn = () => {
+          storage.set(config.key, newValue).catch(console.error);
+        };
+        debouncerMap.debounce(debounceKey, saveFn, debounceMs);
+      }
+    }
+  );
+
+  return {
+    atom: writeAtom,
+    loadingAtom,
+  };
+}

package/src/createStore.ts

@@ -0,0 +1,77 @@
+import type { AtomConfigs, Store, StoreAtoms, StoreLoadingAtoms } from './types';
+import type { StorageAdapter } from './storage/base';
+import { createPersistedAtom } from './atomCreator';
+
+/**
+ * Creates a store with multiple persisted atoms
+ *
+ * @param entityId - Unique identifier for this store instance
+ * @param config - Array of atom configurations
+ * @param options - Store options including storage adapter
+ *
+ * @example
+ * ```typescript
+ * const config = [
+ *   { key: 'theme', defaultValue: '' },
+ *   { key: 'characters', defaultValue: [] },
+ * ] as const;
+ *
+ * const adapter = jsonlStorage({ dir: './sessions' });
+ *
+ * const store = createStore('novel-123', config, {
+ *   storage: adapter,
+ * });
+ * ```
+ */
+export function createStore(
+  entityId: string,
+  config: AtomConfigs,
+  options?: {
+    storage: StorageAdapter;
+    debounceMs?: number;
+  }
+): Store {
+  const adapter = options?.storage;
+  const debounceMs = options?.debounceMs ?? 800;
+
+  // Auto-start embedded server when using remote storage (Node.js only)
+  // In browser/Next.js environments, remoteStorage connects to an already-running BFF server
+  if (adapter?.name === 'remote' && typeof window === 'undefined' && typeof process !== 'undefined') {
+    import('./server').then(({ startServer }) => {
+      startServer().catch((err) => {
+        console.error('[bff-store] Failed to auto-start server:', err);
+      });
+    });
+  }
+
+  if (!adapter) {
+    throw new Error('Storage adapter is required');
+  }
+
+  // Initialize adapter with entityId if it supports it
+  if ('setEntityId' in adapter && typeof adapter.setEntityId === 'function') {
+    adapter.setEntityId(entityId);
+  }
+
+  const storage = adapter.storage;
+
+  const atoms: StoreAtoms = {};
+  const loadingAtoms: StoreLoadingAtoms = {};
+
+  // Create atoms synchronously for immediate availability
+  for (const atomConfig of config) {
+    const result = createPersistedAtom(atomConfig, entityId, storage, {
+      immediate: atomConfig.immediate,
+      debounceMs,
+    });
+    atoms[atomConfig.key] = result.atom;
+    loadingAtoms[atomConfig.key] = result.loadingAtom;
+  }
+
+  return {
+    entityId,
+    config,
+    atoms,
+    loadingAtoms,
+  };
+}

package/src/debouncer.ts

@@ -0,0 +1,84 @@
+/**
+ * Debouncer - manages debounced function execution
+ *
+ * Encapsulates timer state for delayed function calls.
+ * Each Debouncer instance manages one debounce pipeline.
+ */
+export interface Debouncer {
+  readonly ms: number;
+  run(fn: () => void): void;
+  cancel(): void;
+}
+
+export function createDebouncer(ms: number): Debouncer {
+  let timer: ReturnType<typeof setTimeout> | null = null;
+
+  return {
+    get ms() {
+      return ms;
+    },
+    run(fn) {
+      if (timer) clearTimeout(timer);
+      timer = setTimeout(() => {
+        fn();
+        timer = null;
+      }, ms);
+    },
+    cancel() {
+      if (timer) {
+        clearTimeout(timer);
+        timer = null;
+      }
+    },
+  };
+}
+
+/**
+ * Factory for creating debounced save functions per key.
+ * Maintains a map of debouncers, one per unique key.
+ */
+export class DebouncerMap {
+  private debouncers = new Map<string, Debouncer>();
+  private readonly defaultMs: number;
+
+  constructor(defaultMs: number = 800) {
+    this.defaultMs = defaultMs;
+  }
+
+  getDebouncer(key: string, ms?: number): Debouncer {
+    let debouncer = this.debouncers.get(key);
+    if (!debouncer) {
+      debouncer = createDebouncer(ms ?? this.defaultMs);
+      this.debouncers.set(key, debouncer);
+    }
+    return debouncer;
+  }
+
+  /**
+   * Execute a function with debounce for a given key.
+   * Subsequent calls for the same key reset the timer.
+   */
+  debounce(key: string, fn: () => void, ms?: number): void {
+    const debouncer = this.getDebouncer(key, ms);
+    debouncer.run(fn);
+  }
+
+  /**
+   * Cancel pending debounced call for a key
+   */
+  cancel(key: string): void {
+    const debouncer = this.debouncers.get(key);
+    if (debouncer) {
+      debouncer.cancel();
+    }
+  }
+
+  /**
+   * Cancel all pending debounced calls
+   */
+  cancelAll(): void {
+    for (const debouncer of this.debouncers.values()) {
+      debouncer.cancel();
+    }
+  }
+}

package/src/index.ts

@@ -0,0 +1,35 @@
+// ========================================
+// Main Export
+// ========================================
+
+export { createStore } from './createStore';
+export { useStore } from './useStore';
+export { createPersistedAtom } from './atomCreator';
+
+// ========================================
+// Types
+// ========================================
+
+export type {
+  AtomConfig,
+  AtomConfigs,
+  AtomType,
+  Store,
+  StorageOptions,
+  MemoryStorageOptions,
+  UseStoreReturn,
+} from './types';
+
+// ========================================
+// Storage Adapters
+// ========================================
+
+export { memoryStorage, createMemoryStorage } from './storage/memory';
+export { remoteStorage, createRemoteStorage } from './storage/adapters/remoteStorage';
+export type { Storage, StorageAdapter, StorageFactory, AsyncStorageFactory } from './storage/base';
+
+// Transport & Protocol
+export { HttpTransport, createStorageFromTransport } from './storage/transport';
+export { RestStorageProtocol, createStorageWithProtocol } from './storage/protocol';
+export type { TransportAdapter } from './storage/transport';
+export type { StorageHttpProtocol } from './storage/protocol';