zen-fs-config 0.3.30 → 0.4.0

package/DESIGN.md

@@ -25,18 +25,22 @@ ConfigRepo (this library)
         └─ Backend Z (replica)
 ```
 
-### 2.2 No Global Primary
+### 2.2 IndexedDB as Local Primary (Offline-First)
 
-Globally, all backends are equal peers — none is the "source of truth."
+Every program instance uses **IndexedDB** as its local primary backend. All config reads and writes target IndexedDB directly, ensuring offline availability and fast local access.
 
-Locally, each program instance has a "primary backend" — the backend it directly reads/writes through. This is a per-instance choice, not a property of the backend itself.
+User-provided backends (Gitee, S3, RemoteStorage, etc.) are added as **replicas** — they receive bi-directional sync with the local IndexedDB but are never the direct target of config operations.
 
 ```
-Program A → Primary = Backend X, Replicas = [Y, Z]
-Program B → Primary = Backend Y, Replicas = [X, Z]
-Program C → Primary = Backend Z, Replicas = [X, Y]
+Program A → Primary = IndexedDB (local), Replicas = [Gitee, S3]
+Program B → Primary = IndexedDB (local), Replicas = [Gitee, S3]
 ```
 
+This means:
+- Config is always available offline (IndexedDB persists in the browser)
+- Remote backends accelerate multi-device sync, not local access
+- Re-opening the app requires zero backend parameters — IndexedDB + `.meta/backends/` contain everything needed
+
 ### 2.3 Self-Describing Configuration
 
 Backend topology and sync rules are stored **inside** the configuration repository (in `.meta/`), not passed as external parameters. This means any node that can read the config repo can bootstrap the entire sync network.
@@ -47,18 +51,25 @@ External input at startup is limited to: **which backend to connect to** and opt
 
 ```
 /
-├─ .meta/                               [not synced]
-│  ├─ backends.json                     Backend topology (self-describing)
-│  ├─ sync-rules.json                   Sync rules
-│  └─ .conflicts/                       Conflict archives (safekeeping)
-│     └─ {timestamp}_{path}.from-{a}.to-{b}.json
+├─ .meta/                               [synced to replicas]
+│  ├─ backends/                          Backend topology (one file per backend)
+│  │  ├─ local-idb.json                  { id, type, options, description }
+│  │  ├─ gitee-prod.json
+│  │  └─ ...
+│  ├─ .deleted/                          Tombstones for deletion propagation
+│  │  └─ {encoded-path}.json
+│  └─ .conflicts/                        Conflict archives (safekeeping)
+│     └─ {timestamp}_{path}/
+│        ├─ meta.json
+│        ├─ source
+│        └─ target
 │
-├─ {appId}/                             [one-way: owner → replicas, no conflict]
+├─ {appId}/                             [synced: owner → replicas]
 │  ├─ db.json
 │  ├─ cache.json
 │  └─ .db.json.version                  Sidecar version file
 │
-├─ shared/                              [bi-directional, conflict possible]
+├─ shared/                              [synced: bi-directional]
 │  ├─ feature-flags.json
 │  ├─ api-version.json
 │  └─ .feature-flags.json.version
@@ -74,10 +85,10 @@ External input at startup is limited to: **which backend to connect to** and opt
 
 | Directory | Sync Direction | Conflict Risk | Purpose |
 |---|---|---|---|
-| `/{appId}/` | One-way (owner → replicas) | None (single writer) | Per-app private config |
+| `/{appId}/` | Bi-directional (primary ↔ replicas) | Low (single device) | Per-app private config |
 | `/shared/` | Bi-directional | Possible (multiple writers) | Cross-app shared config |
 | `/nodes/` | None (by default) | None | Per-node local config |
-| `/.meta/` | None | None | Topology, rules, conflict archives |
+| `/.meta/` | Bi-directional | None (topology files) | Backend topology, tombstones, conflict archives |
 
 ### 3.2 Config-to-File Mapping
 
@@ -101,29 +112,33 @@ The serializer is determined by file extension:
 
 Users can inject a custom `ConfigSerializer` for other formats.
 
-## 4. Backend Topology (`.meta/backends.json`)
+## 4. Backend Topology (`.meta/backends/*.json`)
+
+Each backend is stored as an individual JSON file in `.meta/backends/`. This allows atomic add/remove operations without rewriting the entire topology.
 
+**`.meta/backends/local-idb.json`** (always present):
 ```json
 {
-  "version": 1,
-  "backends": [
-    {
-      "id": "local-idb",
-      "type": "IndexedDB",
-      "options": { "dbName": "app-config" },
-      "description": "Browser local storage"
-    },
-    {
-      "id": "remote-s3",
-      "type": "S3Bucket",
-      "options": { "bucket": "app-config-bucket", "region": "us-east-1" },
-      "description": "Cloud backup"
-    }
-  ]
+  "id": "local-idb",
+  "type": "IndexedDB",
+  "options": { "storeName": "zen-fs-config-my-app" },
+  "description": "Local IndexedDB primary backend"
 }
 ```
 
-Each program instance selects one backend as its primary via `primaryBackendId`. Others become replicas.
+**`.meta/backends/gitee-prod.json`** (user-added replica):
+```json
+{
+  "id": "gitee-prod",
+  "type": "Gitee",
+  "options": { "token": "...", "owner": "...", "repo": "...", "branch": "main" },
+  "description": "Production Gitee config repo"
+}
+```
+
+The local IndexedDB backend (`local-idb`) is always the primary — all config operations target it directly. All other backends are replicas with bi-directional sync.
+
+**Migration**: If a legacy `.meta/backends.json` file exists (pre-0.4.0), it is automatically migrated to individual files on startup.
 
 ## 5. Sync Rules (`.meta/sync-rules.json`)
 
@@ -343,99 +358,116 @@ interface ConfigRepo {
   /** List conflict archives */
   listConflicts(): Promise<ConflictArchive[]>;
 
-  /** Dispose: stop sync, release cache FS */
+  /** Read backend topology (aggregated from .meta/backends/*.json) */
+  getBackends(): Promise<BackendsMeta | null>;
+
+  /** Write backend topology (writes each backend as individual file) */
+  updateBackends(meta: BackendsMeta): Promise<void>;
+
+  /** Dynamically add a replica backend */
+  addBackend(id: string, type: string, options: Record<string, unknown>, description?: string): Promise<void>;
+
+  /** Dynamically remove a replica backend */
+  removeBackend(id: string): Promise<void>;
+
+  /** Delete a file with tombstone (propagates deletion to all backends) */
+  deleteFile(path: string): Promise<void>;
+
+  /** Sync .meta/ files to all replicas */
+  syncMetaToReplicas(): Promise<void>;
+
+  /** Dispose: stop sync, release resources */
   dispose(): Promise<void>;
 }
 ```
 
 ## 10. Initialization
 
+### Zero-parameter (offline-first)
+
 ```typescript
 import { createConfigRepo } from 'zen-fs-config';
 
-const repo = await createConfigRepo('app-a', {
-  // The only required external input: which backend to connect to
-  primaryBackendId: 'local-idb',
+// No parameters needed — IndexedDB is always created as primary
+const repo = await createConfigRepo('my-app');
 
-  // Backend connection info (only for the primary)
-  backendInfo: {
-    type: 'IndexedDB',
-    options: { dbName: 'app-a-config' }
-  },
+// Config is immediately available from IndexedDB
+repo.setConfig('/db/host', { hostname: 'localhost', port: 3306 });
+```
 
-  // Node ID (optional, see §8.2 for auto-detection)
-  nodeId: 'server-1',
+### With initial replica backend
 
-  // Cache configuration (optional, defaults shown)
-  cache: {
-    storeType: 'MemoryCacheStore',
-    ttlMs: 60000
+```typescript
+const repo = await createConfigRepo('my-app', {
+  // Optional: provide a remote backend as initial replica
+  primaryBackendId: 'gitee-prod',
+  backendInfo: {
+    type: 'Gitee',
+    options: { token: '...', owner: '...', repo: '...', branch: 'main' },
   },
-
-  // Bootstrap data (only used when .meta/backends.json doesn't exist)
-  bootstrap: {
-    backends: [
-      { id: 'local-idb', type: 'IndexedDB', options: { dbName: 'app-config' } },
-      { id: 'remote-s3', type: 'S3Bucket', options: { bucket: 'app-config' } }
-    ],
-    syncRules: [
-      { prefix: '/app-a/', direction: 'one-way', conflictStrategy: 'source-wins', replicas: ['local-idb', 'remote-s3'] },
-      { prefix: '/shared/', direction: 'bi-directional', conflictStrategy: 'merge', replicas: ['local-idb', 'remote-s3'] },
-      { prefix: '/nodes/', direction: 'none' },
-      { prefix: '/.meta/', direction: 'none' }
-    ]
-  }
+  // Optional: customize IndexedDB store name
+  idbStoreName: 'my-app-config',
+  // Optional: node ID (auto-detected if not provided)
+  nodeId: 'server-1',
 });
 
-// Normal config operations
-repo.setConfig('/db/host', { hostname: 'localhost', port: 3306 });
-const dbConfig = repo.getConfig<{ hostname: string; port: number }>('/db/host');
-
-// Node-local config
-repo.setNodeConfig('server-1', '/local.json', { ip: '10.0.0.1' });
+// Later, add more backends dynamically
+await repo.addBackend('s3-backup', 'S3Bucket', {
+  bucket: 'app-config',
+  region: 'us-east-1',
+}, 'S3 backup');
 
-// Publish for debugging (one-time sync)
-await repo.publishNodeConfig('server-1');
+// Remove a backend
+await repo.removeBackend('gitee-prod');
 
 // Cleanup
 await repo.dispose();
 ```
 
+### Re-opening (zero parameters)
+
+```typescript
+// On subsequent opens, just pass appId
+// IndexedDB + .meta/backends/ contain all state
+const repo = await createConfigRepo('my-app');
+
+// All previously added backends are automatically reconnected
+const backends = await repo.getBackends();
+// backends.backends = [{ id: 'local-idb', ... }, { id: 's3-backup', ... }]
+```
+
 ## 11. Initialization Flow
 
 ```
-createConfigRepo('app-a', { primaryBackendId: 'X', backendInfo: {...}, bootstrap: {...} })
+createConfigRepo('my-app', options?)
   │
-  ├─ 1. Connect to primary backend (backendId = 'X')
+  ├─ 1. Create IndexedDB backend (always, ID = 'local-idb')
+  │     storeName = options.idbStoreName || `zen-fs-config-${appId}`
   │
-  ├─ 2. Wrap with zen-fs-cache → CachedFileSystem(primaryBackend, cacheStore, { ttlMs })
+  ├─ 2. Ensure /.meta/ directory exists
   │
-  ├─ 3. Configure ZenFS VFS: { '/': cachedFS }
+  ├─ 3. Migrate legacy .meta/backends.json → .meta/backends/*.json (if exists)
   │
-  ├─ 4. Read .meta/backends.json
-  │     ├─ Exists → parse topology, create replica backend instances
-  │     └─ Not exists → write bootstrap data to .meta/
+  ├─ 4. Ensure local-idb descriptor exists in .meta/backends/
   │
-  ├─ 5. Read .meta/sync-rules.json
-  │     ├─ Exists → parse rules
-  │     └─ Not exists → use bootstrap syncRules
+  ├─ 5. If options.backendInfo provided:
+  │     ├─ Generate replica ID (options.primaryBackendId or auto)
+  │     └─ Write descriptor to .meta/backends/{replicaId}.json (if not exists)
   │
-  ├─ 6. For each rule with direction != 'none':
-  │     ├─ Create SyncPair(
-  │     │     source: cachedFS,
-  │     │     target: replicaBackend,
-  │     │     { direction, conflictStrategy, filter: { includePrefixes: [rule.prefix] } }
-  │     │   )
-  │     └─ syncEngine.watch(pairId)
+  ├─ 6. Read all backend descriptors from .meta/backends/
   │
   ├─ 7. Determine nodeId (explicit > env > auto-generated .node-id file)
   │
-  ├─ 8. Create ZenFS Context:
-  │     ├─ Allowed paths: /{appId}/, /shared/, /nodes/{nodeId}/, /.meta/
-  │     ├─ Root chroot: /{appId}/ (for getConfig/setConfig)
-  │     └─ Full access for zen-fs-sync (unrestricted)
+  ├─ 8. Create ConfigRepo with primary = 'local-idb'
   │
-  └─ 9. Return ConfigRepo { fs, appId, nodeId, ... }
+  ├─ 9. setupSync: for each backend (except local-idb):
+  │     ├─ Create backend instance
+  │     ├─ Create SyncPair(IndexedDB, replica, bi-directional)
+  │     └─ syncEngine.watch(pairId)
+  │
+  ├─ 10. syncMetaToReplicas: push .meta/ changes to all replicas
+  │
+  └─ 11. Load config cache from IndexedDB
 ```
 
 ## 12. Data Flow
@@ -486,11 +518,12 @@ Application
 
 ## 13. Peer Dependencies
 
-| Package | Role | Version |
-|---|---|---|
-| `@zenfs/core` | Virtual file system, backends, VFS, Context | >=2.3.0 |
-| `zen-fs-cache` | Read caching with ETag/304 revalidation | >=1.0.0 |
-| `zen-fs-sync` | Cross-backend sync engine | >=0.1.0 |
+| Package | Role | Version | Required |
+|---|---|---|---|
+| `@zenfs/core` | Virtual file system, backends, VFS, Context | >=2.3.0 | Yes |
+| `@zenfs/dom` | IndexedDB backend (browser) | >=1.0.0 | Yes (browser) |
+| `zen-fs-sync` | Cross-backend sync engine | >=0.1.0 | Yes |
+| `zen-fs-cache` | Read caching with ETag/304 revalidation | >=1.0.0 | No (optional) |
 
 ## 14. Extension Points
 

package/README.md

@@ -1,6 +1,6 @@
 # zen-fs-config
 
-基于 ZenFS 的分布式配置管理库。多个应用实例通过任意 ZenFS 后端共享配置，支持应用隔离、共享空间、节点本地配置和冲突安全。
+基于 ZenFS 的分布式配置管理库。以 IndexedDB 为本地主后端（offline-first），用户提供的远程后端作为副本自动同步。支持应用隔离、共享空间、节点本地配置和冲突安全。
 
 **GitHub**: https://github.com/weijia/zen-fs-config
 **NPM**: `zen-fs-config`
@@ -9,66 +9,109 @@
 ## 安装
 
 ```bash
-npm install zen-fs-config @zenfs/core zen-fs-cache zen-fs-sync
+npm install zen-fs-config @zenfs/core @zenfs/dom zen-fs-sync
 ```
 
+> `@zenfs/dom` 提供 IndexedDB 后端（浏览器环境必需）。`zen-fs-cache` 为可选依赖。
+
 ## 快速开始
 
+### 1. 初始化（零参数）
+
 ```typescript
-import { createConfigRepo, registerBackend } from 'zen-fs-config';
-import { InMemory } from '@zenfs/core';
+import { createConfigRepo } from 'zen-fs-config';
 
-// 1. 注册自定义后端（可选，InMemory 已内置）
-registerBackend('InMemory', async (options) => {
-  return InMemory.create({ maxSize: options.maxSize as number ?? 100 * 1024 * 1024 });
-});
+// 不传任何后端参数，自动创建 IndexedDB 本地主后端
+const repo = await createConfigRepo('my-app');
 
-// 2. 创建配置仓库
-const repo = await createConfigRepo('my-app', {
-  primaryBackendId: 'local-memory',
-  backendInfo: {
-    type: 'InMemory',
-    options: { label: 'my-app-config' },
-  },
-  cache: { storeType: 'MemoryCacheStore', ttlMs: 60_000 },
-  bootstrap: {
-    backends: [
-      { id: 'local-memory', type: 'InMemory', options: { label: 'primary' } },
-    ],
-    syncRules: [
-      { prefix: '/my-app/', direction: 'one-way', conflictStrategy: 'source-wins', replicas: ['local-memory'] },
-      { prefix: '/shared/', direction: 'bi-directional', conflictStrategy: 'merge', replicas: ['local-memory'] },
-      { prefix: '/nodes/', direction: 'none' },
-    ],
-  },
+// 读写配置（同步 API，从 IndexedDB 读取）
+repo.setConfig('/database', { host: 'localhost', port: 5432 });
+const db = repo.getConfig<{ host: string; port: number }>('/database');
+```
+
+### 2. 设置新的配置
+
+```typescript
+repo.setConfig('/cache', { ttl: 3600, maxSize: '100MB' });
+repo.setConfig('/feature-flags', { newUI: true, beta: false });
+```
+
+### 3. 增加数据后端（副本）
+
+```typescript
+import { registerBackend } from 'zen-fs-config';
+import { Gitee } from 'zen-fs-gitee';
+
+// 注册后端类型
+registerBackend('Gitee', async (options) => {
+  return Gitee.create(options);
 });
 
-// 3. 读写配置（同步 API，从内存缓存读取）
-repo.setConfig('/database', { host: 'localhost', port: 5432 });
+// 动态添加副本后端，自动与本地 IndexedDB 双向同步
+await repo.addBackend('gitee-prod', 'Gitee', {
+  token: 'your-token',
+  owner: 'your-name',
+  repo: 'config-repo',
+  branch: 'main',
+}, '生产环境 Gitee 配置仓库');
+```
+
+### 4. 自动同步
+
+```typescript
+// setConfig 写入 IndexedDB 后，自动同步到所有副本后端
+repo.setConfig('/database', { host: 'new-host', port: 5432 });
+
+// 手动触发同步（通常不需要，同步是自动的）
+await repo.flush();
+```
+
+### 5. 再次打开时初始化
+
+```typescript
+// 重新打开页面时，只需传入 appId
+// IndexedDB 中的配置和后端拓扑会自动恢复
+const repo = await createConfigRepo('my-app');
+
+// 配置直接从 IndexedDB 读取（离线可用）
 const db = repo.getConfig<{ host: string; port: number }>('/database');
 
-// 4. 节点本地配置（异步 API，不自动同步）
-await repo.setNodeConfig('node-1', '/debug', { level: 'verbose' });
-const debug = await repo.getNodeConfig('node-1', '/debug');
+// 已注册的副本后端会自动重新连接并同步
+const backends = await repo.getBackends();
+console.log(backends?.backends.map(b => b.id)); // ['local-idb', 'gitee-prod', ...]
+```
 
-// 5. 发布节点配置到同步后端（用于调试）
-await repo.publishNodeConfig('node-1');
+### 带初始后端初始化
 
-// 6. 清理
-await repo.dispose();
+```typescript
+// 首次初始化时可以直接传入远程后端
+const repo = await createConfigRepo('my-app', {
+  primaryBackendId: 'gitee-prod',  // 副本后端 ID
+  backendInfo: {
+    type: 'Gitee',
+    options: { token: 'xxx', owner: 'xxx', repo: 'xxx', branch: 'main' },
+  },
+  idbStoreName: 'my-app-config',  // 自定义 IndexedDB store 名称
+});
+
+// 之后重新打开时不需要再传后端参数
+const repo2 = await createConfigRepo('my-app');
 ```
 
 ## 目录结构
 
 ```
 /
-├── {appId}/           # 应用私有配置（单向同步，每个应用只能读写自己的）
-├── shared/            # 跨应用共享配置（双向同步，merge 冲突策略）
-├── nodes/{nodeId}/    # 节点本地配置（不同步，除非手动 publish）
+├── {appId}/              # 应用私有配置（自动同步到副本）
+├── shared/               # 跨应用共享配置（双向同步）
+├── nodes/{nodeId}/       # 节点本地配置（不同步）
 └── .meta/
-    ├── backends.json  # 后端拓扑（自描述，任意后端可引导）
-    ├── sync-rules.json
-    └── .conflicts/    # 冲突归档（双方内容都保存，永不丢失）
+    ├── backends/          # 后端拓扑（每个后端一个文件）
+    │   ├── local-idb.json
+    │   ├── gitee-prod.json
+    │   └── ...
+    ├── .deleted/          # 删除墓碑（跨后端删除传播）
+    └── .conflicts/        # 冲突归档（双方内容都保存）
 ```
 
 每个配置文件有 sidecar 版本文件：`db.json` → `.db.json.version`（版本号 + SHA-256 哈希）。
@@ -77,8 +120,12 @@ await repo.dispose();
 
 | 方法 | 说明 |
 |---|---|
-| `getConfig<T>(path)` | 同步读取应用配置（从内存缓存） |
+| `createConfigRepo(appId, options?)` | 创建配置仓库。IndexedDB 始终为主后端，`backendInfo` 作为副本 |
+| `getConfig<T>(path)` | 同步读取应用配置（从 IndexedDB） |
 | `setConfig(path, data)` | 同步写入应用配置（异步持久化 + 自动同步） |
+| `addBackend(id, type, options, desc?)` | 动态添加副本后端，自动建立双向同步 |
+| `removeBackend(id)` | 移除副本后端，停止同步 |
+| `getBackends()` | 读取所有后端拓扑（从 `.meta/backends/*.json` 聚合） |
 | `getNodeConfig<T>(nodeId, path)` | 异步读取节点本地配置 |
 | `setNodeConfig(nodeId, path, data)` | 异步写入节点本地配置（不同步） |
 | `publishNodeConfig(nodeId)` | 将节点配置一次性同步到所有后端 |
@@ -86,44 +133,61 @@ await repo.dispose();
 | `flush()` | 手动触发所有同步 |
 | `listConflicts()` | 列出所有冲突归档 |
 | `resolveConflict(id, merged)` | 用合并内容解决冲突 |
-| `fs.promises.*` | 标准 fs API，chroot 隔离到 `/{appId}/` 和 `/shared/` |
+| `fs.promises.*` | 标准 fs API，chroot 隔离到 `/{appId}/` |
 | `dispose()` | 停止同步、释放资源 |
 
 ## 后端注册
 
+zen-fs-config 内置两个后端：
+- **IndexedDB** — 本地主后端（基于 `@zenfs/dom`），无需注册
+- **InMemory** — 内存后端（基于 `@zenfs/core`），用于测试
+
+注册自定义后端：
+
 ```typescript
 import { registerBackend } from 'zen-fs-config';
 
+// 注册 Gitee 后端
+registerBackend('Gitee', async (options) => {
+  const { Gitee } = await import('zen-fs-gitee');
+  return Gitee.create(options);
+});
+
 // 注册 S3 后端
 registerBackend('S3Bucket', async (options) => {
   const { S3Bucket } = await import('@zenfs/core');
   return S3Bucket.create(options);
 });
+```
 
-// 注册自定义后端
-registerBackend('my-custom', async (options) => {
-  return {
-    readFile: (path) => { /* ... */ },
-    writeFile: (path, data) => { /* ... */ },
-    readdir: (path) => { /* ... */ },
-    stat: (path) => { /* ... */ },
-    exists: (path) => { /* ... */ },
-    mkdir: (path) => { /* ... */ },
-    unlink: (path) => { /* ... */ },
-    rmdir: (path) => { /* ... */ },
-    rename: (old, newPath) => { /* ... */ },
-  };
-});
+## 架构概览
+
+```
+Application code
+    ↓ (reads/writes via standard fs API)
+ConfigRepo (this library)
+    ├─ IndexedDB (local primary, always)
+    │   └─ All config operations target IndexedDB first
+    └─ zen-fs-sync → Bi-directional sync
+        ├─ Replica X (e.g., Gitee)
+        ├─ Replica Y (e.g., S3)
+        └─ Replica Z (e.g., RemoteStorage)
 ```
 
+- **IndexedDB 是唯一的主后端**：所有读写操作直接操作 IndexedDB，保证离线可用
+- **远程后端是副本**：通过 `addBackend()` 或 `createConfigRepo({ backendInfo })` 添加
+- **自动同步**：对 IndexedDB 的修改会自动同步到所有副本后端
+- **自描述拓扑**：后端配置存储在 `.meta/backends/` 目录中，每个后端一个 JSON 文件
+
 ## 依赖
 
-| 包 | 说明 |
-|---|---|
-| `@zenfs/core >=2.3.0` | ZenFS 虚拟文件系统 |
-| `zen-fs-cache >=1.0.0` | ETag/TTL 缓存层 |
-| `zen-fs-sync >=0.1.0` | 跨后端同步引擎 |
+| 包 | 说明 | 必需 |
+|---|---|---|
+| `@zenfs/core >=2.3.0` | ZenFS 虚拟文件系统 | 是 |
+| `@zenfs/dom >=1.0.0` | IndexedDB 后端（浏览器） | 是（浏览器） |
+| `zen-fs-sync >=0.1.0` | 跨后端同步引擎 | 是 |
+| `zen-fs-cache >=1.0.0` | ETag/TTL 缓存层 | 否（可选） |
 
 ## License
 
-MIT
+MIT