@rljson/fs-agent 0.0.3 → 0.0.4

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -447,3 +447,50 @@ This peer-to-peer pattern with server coordination enables:
447
447
  ✅ **Real-world deployment**: Works across networks, not just in-memory mocks
448
448
 
449
449
  **This is why clients must NEVER access server Io/Bs directly** - it would bypass the entire peer-to-peer mechanism and make the system only work in single-process scenarios.
450
+
451
+ ## Bounce-Back Prevention
452
+
453
+ Bidirectional sync creates a potential infinite loop:
454
+
455
+ ```
456
+ Client A writes file → syncToDb stores tree → broadcasts ref →
457
+ Client B receives ref → syncFromDb restores file → fs watcher fires →
458
+ syncToDb stores tree → broadcasts ref → Client A receives ref → ...
459
+ ```
460
+
461
+ FsAgent uses three layers of deduplication to break the loop:
462
+
463
+ 1. **Ref-level dedup**: After `syncToDb` stores a tree, it compares the
464
+ resulting ref against `_lastSentRef`. If identical, no broadcast occurs.
465
+
466
+ 2. **Content-key dedup**: Even when refs differ (e.g. different mtimes produce
467
+ different hashes), `syncToDb` computes a content key from file paths +
468
+ blobIds. If the content key matches `_lastSentContentKey`, the broadcast
469
+ is skipped.
470
+
471
+ 3. **Content comparison before restore**: In `syncFromDb`, before restoring
472
+ an incoming tree, FsAgent compares its file content map against the current
473
+ filesystem. If they are equivalent, the restore is skipped entirely,
474
+ preventing the filesystem watcher from firing.
475
+
476
+ Additionally, all sync callbacks are **debounced** (default 300ms) to coalesce
477
+ rapid filesystem events (e.g. multi-file saves, editor autosave) into a single
478
+ sync operation.
479
+
480
+ ## Known Constraints
481
+
482
+ ### macOS Finder Paste and Rename
483
+
484
+ Bidirectional sync relies on Node.js `fs.watch` (FSEvents on macOS). This works
485
+ reliably for programmatic file operations (Node.js `writeFile`/`copyFile`,
486
+ terminal commands, editor saves) but **not** for macOS Finder's paste-and-rename
487
+ workflow.
488
+
489
+ Finder generates rapid, non-atomic, multi-step event sequences (create temp →
490
+ write → rename → delete temp) that FSEvents may coalesce, reorder, or split
491
+ unpredictably. This causes intermediate filesystem states to be scanned and
492
+ broadcast before the operation completes, leading to sync conflicts.
493
+
494
+ **This is a known limitation and is not supported.** Use programmatic operations
495
+ or terminal commands instead of Finder drag-and-drop or paste-and-rename for
496
+ files in synced folders.
package/README.public.md CHANGED
@@ -21,9 +21,11 @@ found in the LICENSE file in the root of this package.
21
21
  - 💾 **Smart Storage**: Content-addressed blob storage eliminates duplicate file content
22
22
  - 📜 **Version History**: Every change is tracked with complete insert history
23
23
  - 🔁 **Bidirectional Sync**: Changes in either direction are automatically propagated
24
- - 🛡️ **Loop Prevention**: Intelligent pause/resume mechanism prevents infinite sync loops
24
+ - 🛡️ **Loop Prevention**: Content-based dedup, ref tracking, and debounce prevent infinite sync loops
25
+ - ⏱️ **Timeout Guards**: Every async operation is bounded to prevent silent hangs
26
+ - 🎯 **Debounce**: Rapid filesystem events are coalesced into a single sync cycle
25
27
  - ✅ **Type-Safe**: Full TypeScript support with comprehensive type definitions
26
- - 🧪 **Battle-Tested**: 100% test coverage with 139 tests
28
+ - 🧪 **Battle-Tested**: 100% test coverage with 204 tests (SocketMock + Socket.IO)
27
29
  - 🧹 **Target Cleanup**: Optional `cleanTarget` restore prunes stale files and directories
28
30
 
29
31
  ## Installation
@@ -700,6 +702,36 @@ try {
700
702
  }
701
703
  ```
702
704
 
705
+ ## Timeout & Debounce Configuration
706
+
707
+ Every async operation in FsAgent is guarded by a timeout to prevent silent hangs.
708
+ Rapid filesystem events are debounced to coalesce into a single sync cycle.
709
+
710
+ ```typescript
711
+ const agent = new FsAgent('./my-project', bs, {
712
+ timeouts: {
713
+ dbQuery: 10_000, // Single db.get() query (default: 10s)
714
+ fetchTree: 20_000, // Fetching an entire tree from the DB (default: 20s)
715
+ extract: 15_000, // Filesystem extract/scan (default: 15s)
716
+ restore: 15_000, // Filesystem restore (default: 15s)
717
+ syncCallback: 25_000, // Overall syncFromDb callback (default: 25s)
718
+ debounceMs: 300, // Coalesce rapid FS events (default: 300ms)
719
+ },
720
+ });
721
+ ```
722
+
723
+ ## Bounce-Back Prevention
724
+
725
+ Bidirectional sync can cause infinite loops when both clients detect each
726
+ other's restores as changes. FsAgent prevents this with three layers:
727
+
728
+ 1. **Ref tracking** (`_lastSentRef`): Skips re-broadcast if the stored tree
729
+ produces the same ref as the last one we sent.
730
+ 2. **Content-key dedup** (`_lastSentContentKey`): Compares file paths + blobIds
731
+ (ignoring mtimes) to detect identical content with different tree hashes.
732
+ 3. **Content comparison before restore**: `syncFromDb` compares the incoming
733
+ tree's content with the current filesystem and skips restore if identical.
734
+
703
735
  ## Performance
704
736
 
705
737
  - **Efficient Scanning**: Only scans changed directories
@@ -707,6 +739,8 @@ try {
707
739
  - **Content-Addressed**: Fast lookups via hashes
708
740
  - **Optimized Watching**: Uses native filesystem events
709
741
  - **Smart Sync**: Only syncs when changes detected
742
+ - **Debounced Events**: Rapid changes coalesced into single operations
743
+ - **Bounded Operations**: All async operations time out to prevent hangs
710
744
 
711
745
  ## Troubleshooting
712
746
 
@@ -763,6 +797,29 @@ ignore: ['*'];
763
797
  ignore: ['node_modules', '*.log'];
764
798
  ```
765
799
 
800
+ ## Known Constraints
801
+
802
+ ### macOS Finder paste and rename
803
+
804
+ Bidirectional sync relies on Node.js `fs.watch` (FSEvents on macOS) to detect
805
+ filesystem changes. This works reliably for **programmatic** file operations:
806
+
807
+ - ✅ `writeFile`, `copyFile`, `rename` from Node.js or shell commands
808
+ - ✅ Editor saves (VS Code, Vim, etc.)
809
+ - ✅ Terminal commands (`echo`, `cp`, `mv`, `touch`, etc.)
810
+ - ✅ Application-generated file changes
811
+
812
+ However, **macOS Finder's paste-and-rename workflow** generates rapid,
813
+ non-atomic, multi-step event sequences (create temporary file → write → rename →
814
+ delete temporary) that FSEvents may coalesce, reorder, or split unpredictably.
815
+ This can cause intermediate states to be scanned and broadcast before the
816
+ operation completes, leading to sync conflicts.
817
+
818
+ **This is a known limitation of filesystem watching on macOS and is not
819
+ supported.** If you need to add or rename files in synced folders, use
820
+ programmatic operations or terminal commands instead of Finder drag-and-drop or
821
+ paste-and-rename.
822
+
766
823
  ## Related Packages
767
824
 
768
825
  - `@rljson/db` - RLJSON database
@@ -447,3 +447,50 @@ This peer-to-peer pattern with server coordination enables:
447
447
  ✅ **Real-world deployment**: Works across networks, not just in-memory mocks
448
448
 
449
449
  **This is why clients must NEVER access server Io/Bs directly** - it would bypass the entire peer-to-peer mechanism and make the system only work in single-process scenarios.
450
+
451
+ ## Bounce-Back Prevention
452
+
453
+ Bidirectional sync creates a potential infinite loop:
454
+
455
+ ```
456
+ Client A writes file → syncToDb stores tree → broadcasts ref →
457
+ Client B receives ref → syncFromDb restores file → fs watcher fires →
458
+ syncToDb stores tree → broadcasts ref → Client A receives ref → ...
459
+ ```
460
+
461
+ FsAgent uses three layers of deduplication to break the loop:
462
+
463
+ 1. **Ref-level dedup**: After `syncToDb` stores a tree, it compares the
464
+ resulting ref against `_lastSentRef`. If identical, no broadcast occurs.
465
+
466
+ 2. **Content-key dedup**: Even when refs differ (e.g. different mtimes produce
467
+ different hashes), `syncToDb` computes a content key from file paths +
468
+ blobIds. If the content key matches `_lastSentContentKey`, the broadcast
469
+ is skipped.
470
+
471
+ 3. **Content comparison before restore**: In `syncFromDb`, before restoring
472
+ an incoming tree, FsAgent compares its file content map against the current
473
+ filesystem. If they are equivalent, the restore is skipped entirely,
474
+ preventing the filesystem watcher from firing.
475
+
476
+ Additionally, all sync callbacks are **debounced** (default 300ms) to coalesce
477
+ rapid filesystem events (e.g. multi-file saves, editor autosave) into a single
478
+ sync operation.
479
+
480
+ ## Known Constraints
481
+
482
+ ### macOS Finder Paste and Rename
483
+
484
+ Bidirectional sync relies on Node.js `fs.watch` (FSEvents on macOS). This works
485
+ reliably for programmatic file operations (Node.js `writeFile`/`copyFile`,
486
+ terminal commands, editor saves) but **not** for macOS Finder's paste-and-rename
487
+ workflow.
488
+
489
+ Finder generates rapid, non-atomic, multi-step event sequences (create temp →
490
+ write → rename → delete temp) that FSEvents may coalesce, reorder, or split
491
+ unpredictably. This causes intermediate filesystem states to be scanned and
492
+ broadcast before the operation completes, leading to sync conflicts.
493
+
494
+ **This is a known limitation and is not supported.** Use programmatic operations
495
+ or terminal commands instead of Finder drag-and-drop or paste-and-rename for
496
+ files in synced folders.
@@ -21,9 +21,11 @@ found in the LICENSE file in the root of this package.
21
21
  - 💾 **Smart Storage**: Content-addressed blob storage eliminates duplicate file content
22
22
  - 📜 **Version History**: Every change is tracked with complete insert history
23
23
  - 🔁 **Bidirectional Sync**: Changes in either direction are automatically propagated
24
- - 🛡️ **Loop Prevention**: Intelligent pause/resume mechanism prevents infinite sync loops
24
+ - 🛡️ **Loop Prevention**: Content-based dedup, ref tracking, and debounce prevent infinite sync loops
25
+ - ⏱️ **Timeout Guards**: Every async operation is bounded to prevent silent hangs
26
+ - 🎯 **Debounce**: Rapid filesystem events are coalesced into a single sync cycle
25
27
  - ✅ **Type-Safe**: Full TypeScript support with comprehensive type definitions
26
- - 🧪 **Battle-Tested**: 100% test coverage with 139 tests
28
+ - 🧪 **Battle-Tested**: 100% test coverage with 204 tests (SocketMock + Socket.IO)
27
29
  - 🧹 **Target Cleanup**: Optional `cleanTarget` restore prunes stale files and directories
28
30
 
29
31
  ## Installation
@@ -700,6 +702,36 @@ try {
700
702
  }
701
703
  ```
702
704
 
705
+ ## Timeout & Debounce Configuration
706
+
707
+ Every async operation in FsAgent is guarded by a timeout to prevent silent hangs.
708
+ Rapid filesystem events are debounced to coalesce into a single sync cycle.
709
+
710
+ ```typescript
711
+ const agent = new FsAgent('./my-project', bs, {
712
+ timeouts: {
713
+ dbQuery: 10_000, // Single db.get() query (default: 10s)
714
+ fetchTree: 20_000, // Fetching an entire tree from the DB (default: 20s)
715
+ extract: 15_000, // Filesystem extract/scan (default: 15s)
716
+ restore: 15_000, // Filesystem restore (default: 15s)
717
+ syncCallback: 25_000, // Overall syncFromDb callback (default: 25s)
718
+ debounceMs: 300, // Coalesce rapid FS events (default: 300ms)
719
+ },
720
+ });
721
+ ```
722
+
723
+ ## Bounce-Back Prevention
724
+
725
+ Bidirectional sync can cause infinite loops when both clients detect each
726
+ other's restores as changes. FsAgent prevents this with three layers:
727
+
728
+ 1. **Ref tracking** (`_lastSentRef`): Skips re-broadcast if the stored tree
729
+ produces the same ref as the last one we sent.
730
+ 2. **Content-key dedup** (`_lastSentContentKey`): Compares file paths + blobIds
731
+ (ignoring mtimes) to detect identical content with different tree hashes.
732
+ 3. **Content comparison before restore**: `syncFromDb` compares the incoming
733
+ tree's content with the current filesystem and skips restore if identical.
734
+
703
735
  ## Performance
704
736
 
705
737
  - **Efficient Scanning**: Only scans changed directories
@@ -707,6 +739,8 @@ try {
707
739
  - **Content-Addressed**: Fast lookups via hashes
708
740
  - **Optimized Watching**: Uses native filesystem events
709
741
  - **Smart Sync**: Only syncs when changes detected
742
+ - **Debounced Events**: Rapid changes coalesced into single operations
743
+ - **Bounded Operations**: All async operations time out to prevent hangs
710
744
 
711
745
  ## Troubleshooting
712
746
 
@@ -763,6 +797,29 @@ ignore: ['*'];
763
797
  ignore: ['node_modules', '*.log'];
764
798
  ```
765
799
 
800
+ ## Known Constraints
801
+
802
+ ### macOS Finder paste and rename
803
+
804
+ Bidirectional sync relies on Node.js `fs.watch` (FSEvents on macOS) to detect
805
+ filesystem changes. This works reliably for **programmatic** file operations:
806
+
807
+ - ✅ `writeFile`, `copyFile`, `rename` from Node.js or shell commands
808
+ - ✅ Editor saves (VS Code, Vim, etc.)
809
+ - ✅ Terminal commands (`echo`, `cp`, `mv`, `touch`, etc.)
810
+ - ✅ Application-generated file changes
811
+
812
+ However, **macOS Finder's paste-and-rename workflow** generates rapid,
813
+ non-atomic, multi-step event sequences (create temporary file → write → rename →
814
+ delete temporary) that FSEvents may coalesce, reorder, or split unpredictably.
815
+ This can cause intermediate states to be scanned and broadcast before the
816
+ operation completes, leading to sync conflicts.
817
+
818
+ **This is a known limitation of filesystem watching on macOS and is not
819
+ supported.** If you need to add or rename files in synced folders, use
820
+ programmatic operations or terminal commands instead of Finder drag-and-drop or
821
+ paste-and-rename.
822
+
766
823
  ## Related Packages
767
824
 
768
825
  - `@rljson/db` - RLJSON database
@@ -1,4 +1,5 @@
1
1
  import { Bs } from '@rljson/bs';
2
+ import { ClientId, SyncConfig } from '@rljson/rljson';
2
3
  import { FsBlobAdapter } from './fs-blob-adapter.ts';
3
4
  import { StoreFsTreeOptions } from './fs-db-adapter.ts';
4
5
  import { FsScanner, FsTree } from './fs-scanner.ts';
@@ -23,12 +24,56 @@ export interface FsAgentOptions {
23
24
  bidirectional?: boolean;
24
25
  /** Restore options applied when syncing from DB */
25
26
  restoreOptions?: RestoreOptions;
27
+ /** Timeout configuration for async operations */
28
+ timeouts?: TimeoutConfig;
29
+ /**
30
+ * Centralized sync protocol configuration.
31
+ * When provided, this SyncConfig is forwarded to every Connector
32
+ * created by {@link FsAgent.fromClient}, and governs whether
33
+ * `sendWithAck()` (when `requireAck` is true) or `send()` is used
34
+ * in {@link FsAgent.syncToDb}.
35
+ *
36
+ * The same SyncConfig should also be passed to the Server and Client
37
+ * constructors so that every layer uses the same protocol settings.
38
+ */
39
+ syncConfig?: SyncConfig;
40
+ /**
41
+ * Stable client identity. When provided, it is forwarded to every
42
+ * Connector created by {@link FsAgent.fromClient}. When omitted but
43
+ * `syncConfig.includeClientIdentity` is true, each Connector
44
+ * auto-generates its own identity.
45
+ */
46
+ clientIdentity?: ClientId;
26
47
  }
27
48
  /** Restore options */
28
49
  export interface RestoreOptions {
29
50
  /** Remove files/dirs on target that are not present in the tree */
30
51
  cleanTarget?: boolean;
31
52
  }
53
+ /**
54
+ * Timeout configuration for async operations (milliseconds).
55
+ * Every async operation in FsAgent is guarded by a timeout to prevent
56
+ * silent hangs in socket communication, filesystem I/O, or database queries.
57
+ */
58
+ export interface TimeoutConfig {
59
+ /** Timeout for a single db.get() query. Default: 10 000 ms */
60
+ dbQuery?: number;
61
+ /** Timeout for fetching an entire tree from the DB. Default: 20 000 ms */
62
+ fetchTree?: number;
63
+ /** Timeout for a filesystem extract / scan. Default: 15 000 ms */
64
+ extract?: number;
65
+ /** Timeout for a filesystem restore. Default: 15 000 ms */
66
+ restore?: number;
67
+ /** Timeout for the overall syncFromDb callback. Default: 25 000 ms */
68
+ syncCallback?: number;
69
+ /**
70
+ * Debounce delay for sync callbacks (milliseconds). Default: 300 ms.
71
+ * Rapid filesystem events (e.g. macOS Finder "Keep Both" copy+rename)
72
+ * are coalesced into a single sync operation after this quiet period.
73
+ * Also applies to incoming database refs in syncFromDb.
74
+ */
75
+ debounceMs?: number;
76
+ }
32
77
  /**
33
78
  * Orchestrates filesystem operations with tree structures and blob storage
34
79
  */
@@ -42,6 +87,9 @@ export declare class FsAgent {
42
87
  private _stopSync?;
43
88
  private _stopSyncFromDb?;
44
89
  private _lastSentRef?;
90
+ /** Content fingerprint of the last tree we broadcasted (paths+blobIds) */
91
+ private _lastSentContentKey?;
92
+ private _timeouts;
45
93
  constructor(rootPath: string, bs?: Bs, options?: FsAgentOptions);
46
94
  /**
47
95
  * Gets the root path
@@ -59,6 +107,27 @@ export declare class FsAgent {
59
107
  * Gets the adapter instance
60
108
  */
61
109
  get adapter(): FsBlobAdapter;
110
+ /**
111
+ * Gets the current timeout configuration
112
+ */
113
+ get timeouts(): Required<TimeoutConfig>;
114
+ /**
115
+ * Wraps a promise with a timeout.
116
+ * Rejects with a descriptive error if the promise does not settle
117
+ * within the given number of milliseconds.
118
+ * @param promise - The promise to guard
119
+ * @param ms - Maximum allowed time in milliseconds
120
+ * @param label - Human-readable label included in the error message
121
+ */
122
+ private static _withTimeout;
123
+ /**
124
+ * Sends a ref through the connector.
125
+ * Uses `sendWithAck()` when the connector has `requireAck` enabled,
126
+ * otherwise falls back to fire-and-forget `send()`.
127
+ * @param connector - The Connector to send through
128
+ * @param ref - The ref to broadcast
129
+ */
130
+ private _sendRef;
62
131
  /**
63
132
  * Starts automatic syncing to database
64
133
  * Note: Auto-sync requires Connector which is not available in constructor.
@@ -130,6 +199,15 @@ export declare class FsAgent {
130
199
  * @returns Array of all tree nodes in the tree
131
200
  */
132
201
  private _fetchTreeRecursively;
202
+ /**
203
+ * Fetches tree from database without restoring to filesystem.
204
+ * Separated from loadFromDb to allow content comparison before restore.
205
+ * @param db - Database instance
206
+ * @param treeKey - Tree table key
207
+ * @param rootRef - Root tree reference (hash)
208
+ * @returns FsTree structure ready for restore
209
+ */
210
+ private _fetchTreeFromDb;
133
211
  /**
134
212
  * Loads tree from database and restores to filesystem
135
213
  * Writes to filesystem from DB trees and Bs blobs
@@ -159,10 +237,37 @@ export declare class FsAgent {
159
237
  * @param db - Database instance
160
238
  * @param connector - Connector instance for socket-based sync
161
239
  * @param treeKey - Tree table key
162
- * @param options - Storage options (e.g., notify)
240
+ * @param options - Storage options (e.g., skipNotification)
163
241
  * @returns Function to stop watching
164
242
  */
165
243
  syncToDb(db: Db, connector: Connector, treeKey: string, options?: StoreFsTreeOptions): Promise<() => void>;
244
+ /**
245
+ * Builds a map of relativePath → blobId for all files in a tree.
246
+ * Used to compare trees by content rather than by hash (which includes mtime).
247
+ * @param tree - Tree structure to extract file content map from
248
+ */
249
+ private _getFileContentMap;
250
+ /**
251
+ * Derives a deterministic string key from a content map so that two trees
252
+ * with identical file paths + blobIds produce the same key regardless of
253
+ * mtime differences.
254
+ * @param map - Content map (relativePath → blobId)
255
+ */
256
+ private _contentKeyFromMap;
257
+ /**
258
+ * Derives a deterministic content key from an FsTree.
259
+ * @param tree - Tree structure to derive content key from
260
+ */
261
+ private _contentKeyFromTree;
262
+ /**
263
+ * Compares two trees by file content (relativePath + blobId).
264
+ * Ignores mtime differences — trees are equivalent if they have the same
265
+ * files with the same content. This prevents bounce-back restores from
266
+ * destroying locally-created files during bidirectional sync.
267
+ * @param a - First tree to compare
268
+ * @param b - Second tree to compare
269
+ */
270
+ private _treesHaveEquivalentContent;
166
271
  /**
167
272
  * Watches database for tree changes and syncs to filesystem
168
273
  * Uses Connector for socket-based notifications
@@ -181,16 +286,21 @@ export declare class FsAgent {
181
286
  * @param treeKey - Tree table key (route will be `/${treeKey}`)
182
287
  * @param client - Client instance with io and bs properties
183
288
  * @param socket - Socket instance for connector communication
184
- * @param options - Optional FsAgent options (db and treeKey are set automatically)
289
+ * @param options - Optional FsAgent options (db and treeKey are set automatically).
290
+ * `syncConfig` and `clientIdentity` from these options are forwarded to
291
+ * the Connector so that a single config origin governs all layers.
185
292
  * @returns Configured FsAgent instance with simplified sync API
186
293
  * @example
187
294
  * ```typescript
188
- * const agent = await FsAgent.fromClient('./my-folder', 'sharedTree', client, socket);
295
+ * const syncConfig: SyncConfig = { requireAck: true, maxDedupSetSize: 5000 };
296
+ * const agent = await FsAgent.fromClient(
297
+ * './my-folder', 'sharedTree', client, socket, { syncConfig },
298
+ * );
189
299
  * // Simplified sync methods - no db/connector/treeKey needed
190
- * await agent.syncToDbSimple({ notify: true });
300
+ * await agent.syncToDbSimple();
191
301
  * await agent.syncFromDbSimple({ cleanTarget: true });
192
302
  * // Original methods still work
193
- * await agent.syncToDb(db, connector, treeKey, { notify: true });
303
+ * await agent.syncToDb(db, connector, treeKey);
194
304
  * ```
195
305
  */
196
306
  static fromClient(filePath: string, treeKey: string, client: any, // Client type from \@rljson/server
package/dist/fs-agent.js CHANGED
@@ -1,5 +1,5 @@
1
1
  import { BsMem } from "@rljson/bs";
2
- import { timeId, Route, createTreesTableCfg } from "@rljson/rljson";
2
+ import { Route, createTreesTableCfg } from "@rljson/rljson";
3
3
  import { stat, readFile, mkdir, writeFile, readdir, utimes, rm } from "fs/promises";
4
4
  import { dirname, join, sep } from "path";
5
5
  import { hip } from "@rljson/hash";
@@ -124,7 +124,6 @@ class FsDbAdapter {
124
124
  * @returns The root tree reference
125
125
  */
126
126
  async storeFsTree(fsTree, options = {}) {
127
- const { notify = false } = options;
128
127
  if (!fsTree) {
129
128
  throw new Error("fsTree cannot be null or undefined");
130
129
  }
@@ -153,37 +152,10 @@ class FsDbAdapter {
153
152
  (tree) => tree._hash !== fsTree.rootHash
154
153
  );
155
154
  trees.push(rootTree);
156
- const treeTable = {
157
- _type: "trees",
158
- _data: trees
159
- };
160
- try {
161
- await this.db.core.import({
162
- [this.treeKey]: treeTable
163
- });
164
- } catch (error) {
165
- throw new Error(
166
- `Failed to import tree data into database: ${error instanceof Error ? error.message : String(error)}`
167
- );
168
- }
169
- const treeRootRef = trees[trees.length - 1]._hash;
170
- const historyRow = {
171
- timeId: timeId(),
172
- route: `/${this.treeKey}/${treeRootRef}`,
173
- [`${this.treeKey}Ref`]: treeRootRef
174
- };
175
- const historyTable = {
176
- _type: "insertHistory",
177
- _data: [historyRow]
178
- };
179
- await this.db.core.import({
180
- [`${this.treeKey}InsertHistory`]: historyTable
155
+ const results = await this.db.insertTrees(this.treeKey, trees, {
156
+ skipNotification: options.skipNotification
181
157
  });
182
- if (notify) {
183
- const treeKeyRoute = Route.fromFlat(`/${this.treeKey}`);
184
- this.db.notify.notify(treeKeyRoute, historyRow);
185
- }
186
- return treeRootRef;
158
+ return results[0][`${this.treeKey}Ref`];
187
159
  }
188
160
  /**
189
161
  * Get the tree table key
@@ -499,6 +471,14 @@ class FsScanner {
499
471
  });
500
472
  }
501
473
  }
474
+ const DEFAULT_TIMEOUTS = {
475
+ dbQuery: 1e4,
476
+ fetchTree: 2e4,
477
+ extract: 15e3,
478
+ restore: 15e3,
479
+ syncCallback: 25e3,
480
+ debounceMs: 300
481
+ };
502
482
  class FsAgent {
503
483
  _scanner;
504
484
  _adapter;
@@ -509,11 +489,15 @@ class FsAgent {
509
489
  _stopSync;
510
490
  _stopSyncFromDb;
511
491
  _lastSentRef;
492
+ /** Content fingerprint of the last tree we broadcasted (paths+blobIds) */
493
+ _lastSentContentKey;
494
+ _timeouts;
512
495
  constructor(rootPath, bs, options = {}) {
513
496
  this._rootPath = rootPath;
514
497
  this._bs = bs || new BsMem();
515
498
  this._db = options.db;
516
499
  this._treeKey = options.treeKey;
500
+ this._timeouts = { ...DEFAULT_TIMEOUTS, ...options.timeouts };
517
501
  this._scanner = new FsScanner(rootPath, { ...options, bs: this._bs });
518
502
  this._adapter = new FsBlobAdapter(this._bs);
519
503
  if (this._db && this._treeKey) {
@@ -547,6 +531,51 @@ class FsAgent {
547
531
  get adapter() {
548
532
  return this._adapter;
549
533
  }
534
+ /**
535
+ * Gets the current timeout configuration
536
+ */
537
+ get timeouts() {
538
+ return this._timeouts;
539
+ }
540
+ /**
541
+ * Wraps a promise with a timeout.
542
+ * Rejects with a descriptive error if the promise does not settle
543
+ * within the given number of milliseconds.
544
+ * @param promise - The promise to guard
545
+ * @param ms - Maximum allowed time in milliseconds
546
+ * @param label - Human-readable label included in the error message
547
+ */
548
+ static _withTimeout(promise, ms, label) {
549
+ return new Promise((resolve, reject) => {
550
+ const timer = setTimeout(() => {
551
+ reject(new Error(`Timeout after ${ms}ms: ${label}`));
552
+ }, ms);
553
+ promise.then(
554
+ (value) => {
555
+ clearTimeout(timer);
556
+ resolve(value);
557
+ },
558
+ (err) => {
559
+ clearTimeout(timer);
560
+ reject(err);
561
+ }
562
+ );
563
+ });
564
+ }
565
+ /**
566
+ * Sends a ref through the connector.
567
+ * Uses `sendWithAck()` when the connector has `requireAck` enabled,
568
+ * otherwise falls back to fire-and-forget `send()`.
569
+ * @param connector - The Connector to send through
570
+ * @param ref - The ref to broadcast
571
+ */
572
+ async _sendRef(connector, ref) {
573
+ if (connector.syncConfig?.requireAck) {
574
+ await connector.sendWithAck(ref);
575
+ } else {
576
+ connector.send(ref);
577
+ }
578
+ }
550
579
  /**
551
580
  * Starts automatic syncing to database
552
581
  * Note: Auto-sync requires Connector which is not available in constructor.
@@ -727,8 +756,15 @@ class FsAgent {
727
756
  processed.add(currentHash);
728
757
  let result;
729
758
  try {
730
- result = await db.get(route, { _hash: currentHash });
731
- } catch {
759
+ result = await FsAgent._withTimeout(
760
+ db.get(route, { _hash: currentHash }),
761
+ this._timeouts.dbQuery,
762
+ `db.get(${treeKey}, _hash=${currentHash.slice(0, 8)}…)`
763
+ );
764
+ } catch (error) {
765
+ if (error instanceof Error && error.message.startsWith("Timeout")) {
766
+ throw error;
767
+ }
732
768
  continue;
733
769
  }
734
770
  const treeData = result?.rljson?.[treeKey];
@@ -751,24 +787,22 @@ class FsAgent {
751
787
  return Array.from(fetchedNodes.values());
752
788
  }
753
789
  /**
754
- * Loads tree from database and restores to filesystem
755
- * Writes to filesystem from DB trees and Bs blobs
790
+ * Fetches tree from database without restoring to filesystem.
791
+ * Separated from loadFromDb to allow content comparison before restore.
756
792
  * @param db - Database instance
757
793
  * @param treeKey - Tree table key
758
794
  * @param rootRef - Root tree reference (hash)
759
- * @param targetPath - Optional target path (defaults to rootPath)
760
- * @param options - Restore options
795
+ * @returns FsTree structure ready for restore
761
796
  */
762
- async loadFromDb(db, treeKey, rootRef, targetPath, options) {
797
+ async _fetchTreeFromDb(db, treeKey, rootRef) {
763
798
  if (!rootRef || rootRef.trim() === "") {
764
799
  throw new Error("rootRef cannot be empty");
765
800
  }
766
801
  const route = Route.fromFlat(treeKey);
767
- const allNodes = await this._fetchTreeRecursively(
768
- db,
769
- route,
770
- treeKey,
771
- rootRef
802
+ const allNodes = await FsAgent._withTimeout(
803
+ this._fetchTreeRecursively(db, route, treeKey, rootRef),
804
+ this._timeouts.fetchTree,
805
+ `fetchTree(${treeKey}@${rootRef.slice(0, 8)}…)`
772
806
  );
773
807
  if (allNodes.length === 0) {
774
808
  throw new Error(
@@ -786,10 +820,19 @@ class FsAgent {
786
820
  `Root tree node "${rootRef}" not found in tree data. Available hashes: ${Array.from(trees.keys()).slice(0, 5).join(", ")}${trees.size > 5 ? "..." : ""}`
787
821
  );
788
822
  }
789
- const fsTree = {
790
- rootHash: rootRef,
791
- trees
792
- };
823
+ return { rootHash: rootRef, trees };
824
+ }
825
+ /**
826
+ * Loads tree from database and restores to filesystem
827
+ * Writes to filesystem from DB trees and Bs blobs
828
+ * @param db - Database instance
829
+ * @param treeKey - Tree table key
830
+ * @param rootRef - Root tree reference (hash)
831
+ * @param targetPath - Optional target path (defaults to rootPath)
832
+ * @param options - Restore options
833
+ */
834
+ async loadFromDb(db, treeKey, rootRef, targetPath, options) {
835
+ const fsTree = await this._fetchTreeFromDb(db, treeKey, rootRef);
793
836
  await this.restore(fsTree, targetPath, options);
794
837
  }
795
838
  /**
@@ -845,33 +888,115 @@ class FsAgent {
845
888
  * @param db - Database instance
846
889
  * @param connector - Connector instance for socket-based sync
847
890
  * @param treeKey - Tree table key
848
- * @param options - Storage options (e.g., notify)
891
+ * @param options - Storage options (e.g., skipNotification)
849
892
  * @returns Function to stop watching
850
893
  */
851
894
  async syncToDb(db, connector, treeKey, options) {
852
- const initialRef = await this.storeInDb(db, treeKey, options);
895
+ const initialRef = await FsAgent._withTimeout(
896
+ this.storeInDb(db, treeKey, options),
897
+ this._timeouts.fetchTree,
898
+ `syncToDb → initial storeInDb(${treeKey})`
899
+ );
853
900
  if (initialRef) {
854
901
  this._lastSentRef = initialRef;
855
- connector.send(initialRef);
902
+ const currentTree = this._scanner.tree;
903
+ if (currentTree) {
904
+ this._lastSentContentKey = this._contentKeyFromTree(currentTree);
905
+ }
906
+ await this._sendRef(connector, initialRef);
856
907
  }
857
- const syncCallback = async () => {
858
- const tree = this._scanner.tree;
859
- if (tree) {
860
- const dbAdapter = new FsDbAdapter(db, treeKey);
861
- const ref = await dbAdapter.storeFsTree(tree, options);
862
- this._lastSentRef = ref;
863
- if (ref) {
864
- connector.send(ref);
908
+ let debounceTimer = null;
909
+ const debouncedSync = () => {
910
+ if (debounceTimer) clearTimeout(debounceTimer);
911
+ debounceTimer = setTimeout(async () => {
912
+ debounceTimer = null;
913
+ const tree = this._scanner.tree;
914
+ if (tree) {
915
+ try {
916
+ const contentKey = this._contentKeyFromTree(tree);
917
+ if (contentKey === this._lastSentContentKey) {
918
+ return;
919
+ }
920
+ const dbAdapter = new FsDbAdapter(db, treeKey);
921
+ const ref = await FsAgent._withTimeout(
922
+ dbAdapter.storeFsTree(tree, options),
923
+ this._timeouts.fetchTree,
924
+ `syncToDb → storeFsTree(${treeKey})`
925
+ );
926
+ if (ref === this._lastSentRef) {
927
+ return;
928
+ }
929
+ this._lastSentRef = ref;
930
+ this._lastSentContentKey = contentKey;
931
+ if (ref) {
932
+ await this._sendRef(connector, ref);
933
+ }
934
+ } catch {
935
+ }
865
936
  }
866
- }
937
+ }, this._timeouts.debounceMs);
867
938
  };
868
- this._scanner.onChange(syncCallback);
939
+ this._scanner.onChange(debouncedSync);
869
940
  await this._scanner.watch();
870
941
  return () => {
871
- this._scanner.offChange(syncCallback);
942
+ if (debounceTimer) clearTimeout(debounceTimer);
943
+ this._scanner.offChange(debouncedSync);
872
944
  this._scanner.stopWatch();
873
945
  };
874
946
  }
947
+ /**
948
+ * Builds a map of relativePath → blobId for all files in a tree.
949
+ * Used to compare trees by content rather than by hash (which includes mtime).
950
+ * @param tree - Tree structure to extract file content map from
951
+ */
952
+ _getFileContentMap(tree) {
953
+ const map = /* @__PURE__ */ new Map();
954
+ for (const [, node] of tree.trees) {
955
+ const meta = node?.meta;
956
+ if (meta?.type === "file") {
957
+ map.set(meta.relativePath, meta.blobId ?? "");
958
+ } else if (meta?.type === "directory" && meta.relativePath !== ".") {
959
+ map.set(meta.relativePath, "<dir>");
960
+ }
961
+ }
962
+ return map;
963
+ }
964
+ /**
965
+ * Derives a deterministic string key from a content map so that two trees
966
+ * with identical file paths + blobIds produce the same key regardless of
967
+ * mtime differences.
968
+ * @param map - Content map (relativePath → blobId)
969
+ */
970
+ _contentKeyFromMap(map) {
971
+ const sorted = Array.from(map.entries()).sort(
972
+ (a, b) => a[0].localeCompare(b[0])
973
+ );
974
+ return sorted.map(([p, b]) => `${p}:${b}`).join("\n");
975
+ }
976
+ /**
977
+ * Derives a deterministic content key from an FsTree.
978
+ * @param tree - Tree structure to derive content key from
979
+ */
980
+ _contentKeyFromTree(tree) {
981
+ return this._contentKeyFromMap(this._getFileContentMap(tree));
982
+ }
983
+ /**
984
+ * Compares two trees by file content (relativePath + blobId).
985
+ * Ignores mtime differences — trees are equivalent if they have the same
986
+ * files with the same content. This prevents bounce-back restores from
987
+ * destroying locally-created files during bidirectional sync.
988
+ * @param a - First tree to compare
989
+ * @param b - Second tree to compare
990
+ */
991
+ _treesHaveEquivalentContent(a, b) {
992
+ const aFiles = this._getFileContentMap(a);
993
+ const bFiles = this._getFileContentMap(b);
994
+ if (aFiles.size !== bFiles.size) return false;
995
+ for (const [path, blobId] of aFiles) {
996
+ if (bFiles.get(path) !== blobId) return false;
997
+ }
998
+ return true;
999
+ }
875
1000
  /**
876
1001
  * Watches database for tree changes and syncs to filesystem
877
1002
  * Uses Connector for socket-based notifications
@@ -885,24 +1010,60 @@ class FsAgent {
885
1010
  if (!this._scanner["_watcher"]) {
886
1011
  await this._scanner.watch();
887
1012
  }
888
- const syncCallback = async (treeRef) => {
889
- if (!treeRef || typeof treeRef !== "string") {
890
- return;
891
- }
892
- if (treeRef === this._lastSentRef) {
893
- return;
894
- }
1013
+ let pendingRef = null;
1014
+ let fromDbTimer = null;
1015
+ const processRef = async (treeRef) => {
895
1016
  this._scanner.pauseWatch();
896
1017
  try {
897
- await this.loadFromDb(db, treeKey, treeRef, void 0, restoreOptions);
1018
+ const incomingTree = await FsAgent._withTimeout(
1019
+ this._fetchTreeFromDb(db, treeKey, treeRef),
1020
+ this._timeouts.fetchTree,
1021
+ `syncFromDb → fetchTree(${treeKey}@${treeRef.slice(0, 8)}…)`
1022
+ );
1023
+ const currentTree = await FsAgent._withTimeout(
1024
+ this.extract(),
1025
+ this._timeouts.extract,
1026
+ `syncFromDb → extract(${this._rootPath})`
1027
+ );
1028
+ if (this._treesHaveEquivalentContent(currentTree, incomingTree)) {
1029
+ return;
1030
+ }
1031
+ await FsAgent._withTimeout(
1032
+ this.restore(incomingTree, void 0, restoreOptions),
1033
+ this._timeouts.restore,
1034
+ `syncFromDb → restore(${treeKey})`
1035
+ );
1036
+ const postRestoreTree = await this._scanner.scan();
1037
+ const dbAdapter = new FsDbAdapter(db, treeKey);
1038
+ const postRestoreRef = await dbAdapter.storeFsTree(postRestoreTree, {
1039
+ skipNotification: true
1040
+ });
1041
+ this._lastSentRef = postRestoreRef;
1042
+ this._lastSentContentKey = this._contentKeyFromTree(postRestoreTree);
898
1043
  } catch {
899
1044
  } finally {
900
1045
  this._scanner.resumeWatch();
901
1046
  }
902
1047
  };
1048
+ const syncCallback = async (treeRef) => {
1049
+ if (!treeRef || typeof treeRef !== "string") {
1050
+ return;
1051
+ }
1052
+ pendingRef = treeRef;
1053
+ if (fromDbTimer) clearTimeout(fromDbTimer);
1054
+ fromDbTimer = setTimeout(async () => {
1055
+ fromDbTimer = null;
1056
+ const ref = pendingRef;
1057
+ pendingRef = null;
1058
+ if (ref) {
1059
+ await processRef(ref);
1060
+ }
1061
+ }, this._timeouts.debounceMs);
1062
+ };
903
1063
  connector.listen(syncCallback);
904
1064
  return () => {
905
- connector.teardown();
1065
+ if (fromDbTimer) clearTimeout(fromDbTimer);
1066
+ connector.tearDown();
906
1067
  };
907
1068
  }
908
1069
  /**
@@ -913,16 +1074,21 @@ class FsAgent {
913
1074
  * @param treeKey - Tree table key (route will be `/${treeKey}`)
914
1075
  * @param client - Client instance with io and bs properties
915
1076
  * @param socket - Socket instance for connector communication
916
- * @param options - Optional FsAgent options (db and treeKey are set automatically)
1077
+ * @param options - Optional FsAgent options (db and treeKey are set automatically).
1078
+ * `syncConfig` and `clientIdentity` from these options are forwarded to
1079
+ * the Connector so that a single config origin governs all layers.
917
1080
  * @returns Configured FsAgent instance with simplified sync API
918
1081
  * @example
919
1082
  * ```typescript
920
- * const agent = await FsAgent.fromClient('./my-folder', 'sharedTree', client, socket);
1083
+ * const syncConfig: SyncConfig = { requireAck: true, maxDedupSetSize: 5000 };
1084
+ * const agent = await FsAgent.fromClient(
1085
+ * './my-folder', 'sharedTree', client, socket, { syncConfig },
1086
+ * );
921
1087
  * // Simplified sync methods - no db/connector/treeKey needed
922
- * await agent.syncToDbSimple({ notify: true });
1088
+ * await agent.syncToDbSimple();
923
1089
  * await agent.syncFromDbSimple({ cleanTarget: true });
924
1090
  * // Original methods still work
925
- * await agent.syncToDb(db, connector, treeKey, { notify: true });
1091
+ * await agent.syncToDb(db, connector, treeKey);
926
1092
  * ```
927
1093
  */
928
1094
  static async fromClient(filePath, treeKey, client, socket, options) {
@@ -935,7 +1101,13 @@ class FsAgent {
935
1101
  const { Db: Db2, Connector } = await import("@rljson/db");
936
1102
  const db = new Db2(client.io);
937
1103
  const route = Route.fromFlat(`/${treeKey}`);
938
- const connector = new Connector(db, route, socket);
1104
+ const connector = new Connector(
1105
+ db,
1106
+ route,
1107
+ socket,
1108
+ options?.syncConfig,
1109
+ options?.clientIdentity
1110
+ );
939
1111
  const agent = new FsAgent(filePath, client.bs, options);
940
1112
  const enhancedAgent = agent;
941
1113
  enhancedAgent.syncToDbSimple = async (syncOptions) => {
@@ -999,7 +1171,7 @@ async function runClientServerSetup(opts = {}) {
999
1171
  const agentB = new FsAgent(folderB, clientB.bs);
1000
1172
  const helloPathA = join(folderA, "hello.txt");
1001
1173
  await writeFile(helloPathA, "Hello from Client A");
1002
- const rootRef = await agentA.storeInDb(clientDbA, treeKey, { notify: false });
1174
+ const rootRef = await agentA.storeInDb(clientDbA, treeKey, { skipNotification: true });
1003
1175
  await agentB.loadFromDb(clientDbB, treeKey, rootRef);
1004
1176
  const contentB = await readFile(join(folderB, "hello.txt"), "utf8");
1005
1177
  const cleanup = async () => {
@@ -5,12 +5,19 @@ import { FsTree } from './fs-scanner.js';
5
5
  */
6
6
  export interface StoreFsTreeOptions {
7
7
  /**
8
- * Whether to trigger notifications after storing (defaults to false)
8
+ * Whether to skip notifications after storing (defaults to false).
9
+ * When false, observers (e.g. Connector) are notified automatically
10
+ * via the standard db.insertTrees() pipeline.
9
11
  */
10
- notify?: boolean;
12
+ skipNotification?: boolean;
11
13
  }
12
14
  /**
13
- * Adapter for storing filesystem trees in a database
15
+ * Adapter for storing filesystem trees in a database.
16
+ *
17
+ * Uses `db.insertTrees()` to go through the standard insert pipeline:
18
+ * - TreeController writes each node
19
+ * - InsertHistoryRow is created automatically
20
+ * - `notify.notify()` fires so Connector observers broadcast the ref
14
21
  */
15
22
  export declare class FsDbAdapter {
16
23
  private db;
package/dist/index.d.ts CHANGED
@@ -1,4 +1,4 @@
1
- export { FsAgent, type FsAgentOptions } from './fs-agent.ts';
1
+ export { FsAgent, type FsAgentOptions, type RestoreOptions, type TimeoutConfig, } from './fs-agent.ts';
2
2
  export { FsBlobAdapter, type BlobToFileOptions, type FileBlobMeta, type FileToBlobOptions, } from './fs-blob-adapter.ts';
3
3
  export { FsDbAdapter, type StoreFsTreeOptions } from './fs-db-adapter.ts';
4
4
  export { FsScanner, type FsChange, type FsChangeCallback, type FsChangeType, type FsNodeMeta, type FsScanOptions, type FsTree, } from './fs-scanner.ts';
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@rljson/fs-agent",
3
- "version": "0.0.3",
3
+ "version": "0.0.4",
4
4
  "description": "Rljson fs-agent description",
5
5
  "homepage": "https://github.com/rljson/fs-agent",
6
6
  "bugs": "https://github.com/rljson/fs-agent/issues",
@@ -20,40 +20,40 @@
20
20
  ],
21
21
  "type": "module",
22
22
  "devDependencies": {
23
- "@rljson/server": "^0.0.5",
23
+ "@rljson/server": "^0.0.6",
24
24
  "@types/node": "^25.2.3",
25
- "@typescript-eslint/eslint-plugin": "^8.55.0",
26
- "@typescript-eslint/parser": "^8.55.0",
25
+ "@typescript-eslint/eslint-plugin": "^8.56.0",
26
+ "@typescript-eslint/parser": "^8.56.0",
27
27
  "@vitest/coverage-v8": "^4.0.18",
28
28
  "cross-env": "^10.1.0",
29
29
  "eslint": "~9.39.2",
30
- "eslint-plugin-jsdoc": "^62.5.4",
30
+ "eslint-plugin-jsdoc": "^62.5.5",
31
31
  "eslint-plugin-tsdoc": "^0.5.0",
32
32
  "globals": "^17.3.0",
33
33
  "jsdoc": "^4.0.5",
34
34
  "read-pkg": "^10.1.0",
35
35
  "typescript": "~5.9.3",
36
- "typescript-eslint": "^8.55.0",
36
+ "typescript-eslint": "^8.56.0",
37
37
  "vite": "^7.3.1",
38
38
  "vite-node": "^5.3.0",
39
39
  "vite-plugin-dts": "^4.5.4",
40
- "vite-tsconfig-paths": "^6.1.0",
40
+ "vite-tsconfig-paths": "^6.1.1",
41
41
  "vitest": "^4.0.18",
42
42
  "vitest-dom": "^0.1.1"
43
43
  },
44
44
  "dependencies": {
45
- "@rljson/bs": "^0.0.20",
46
- "@rljson/db": "^0.0.13",
45
+ "@rljson/bs": "^0.0.21",
46
+ "@rljson/db": "^0.0.14",
47
47
  "@rljson/hash": "^0.0.18",
48
- "@rljson/io": "^0.0.65",
48
+ "@rljson/io": "^0.0.66",
49
49
  "@rljson/json": "^0.0.23",
50
- "@rljson/rljson": "^0.0.75",
50
+ "@rljson/rljson": "^0.0.76",
51
51
  "socket.io": "^4.8.3",
52
52
  "socket.io-client": "^4.8.3"
53
53
  },
54
54
  "scripts": {
55
- "build": "pnpx vite build && tsc && node scripts/copy-readme-to-dist.js",
56
- "test": "cross-env NODE_OPTIONS=--max-old-space-size=8192 pnpx vitest run --coverage && pnpm run lint",
55
+ "build": "pnpm exec vite build && tsc && node scripts/copy-readme-to-dist.js",
56
+ "test": "cross-env NODE_OPTIONS=--max-old-space-size=8192 pnpm exec vitest run --coverage && pnpm run lint",
57
57
  "prebuild": "npm run test",
58
58
  "lint": "pnpm exec eslint .",
59
59
  "updateGoldens": "cross-env UPDATE_GOLDENS=true pnpm test"