@interocitor/core 0.0.0-beta.10

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (96) hide show
  1. package/LICENSE +21 -0
  2. package/README.md +706 -0
  3. package/dist/adapters/cloudflare.d.ts +78 -0
  4. package/dist/adapters/cloudflare.d.ts.map +1 -0
  5. package/dist/adapters/cloudflare.js +325 -0
  6. package/dist/adapters/google-drive.d.ts +64 -0
  7. package/dist/adapters/google-drive.d.ts.map +1 -0
  8. package/dist/adapters/google-drive.js +339 -0
  9. package/dist/adapters/memory.d.ts +53 -0
  10. package/dist/adapters/memory.d.ts.map +1 -0
  11. package/dist/adapters/memory.js +182 -0
  12. package/dist/adapters/webdav.d.ts +70 -0
  13. package/dist/adapters/webdav.d.ts.map +1 -0
  14. package/dist/adapters/webdav.js +323 -0
  15. package/dist/core/codec.d.ts +20 -0
  16. package/dist/core/codec.d.ts.map +1 -0
  17. package/dist/core/codec.js +102 -0
  18. package/dist/core/compaction.d.ts +45 -0
  19. package/dist/core/compaction.d.ts.map +1 -0
  20. package/dist/core/compaction.js +190 -0
  21. package/dist/core/connected-stores.d.ts +77 -0
  22. package/dist/core/connected-stores.d.ts.map +1 -0
  23. package/dist/core/connected-stores.js +76 -0
  24. package/dist/core/crdt.d.ts +36 -0
  25. package/dist/core/crdt.d.ts.map +1 -0
  26. package/dist/core/crdt.js +174 -0
  27. package/dist/core/errors.d.ts +47 -0
  28. package/dist/core/errors.d.ts.map +1 -0
  29. package/dist/core/errors.js +61 -0
  30. package/dist/core/flush.d.ts +9 -0
  31. package/dist/core/flush.d.ts.map +1 -0
  32. package/dist/core/flush.js +98 -0
  33. package/dist/core/hlc.d.ts +25 -0
  34. package/dist/core/hlc.d.ts.map +1 -0
  35. package/dist/core/hlc.js +75 -0
  36. package/dist/core/ids.d.ts +49 -0
  37. package/dist/core/ids.d.ts.map +1 -0
  38. package/dist/core/ids.js +132 -0
  39. package/dist/core/internals.d.ts +33 -0
  40. package/dist/core/internals.d.ts.map +1 -0
  41. package/dist/core/internals.js +72 -0
  42. package/dist/core/manifest.d.ts +56 -0
  43. package/dist/core/manifest.d.ts.map +1 -0
  44. package/dist/core/manifest.js +203 -0
  45. package/dist/core/pull.d.ts +26 -0
  46. package/dist/core/pull.d.ts.map +1 -0
  47. package/dist/core/pull.js +113 -0
  48. package/dist/core/row-id.d.ts +12 -0
  49. package/dist/core/row-id.d.ts.map +1 -0
  50. package/dist/core/row-id.js +11 -0
  51. package/dist/core/schema-types.d.ts +26 -0
  52. package/dist/core/schema-types.d.ts.map +1 -0
  53. package/dist/core/schema-types.js +31 -0
  54. package/dist/core/schema-types.type-test.d.ts +2 -0
  55. package/dist/core/schema-types.type-test.d.ts.map +1 -0
  56. package/dist/core/schema-types.type-test.js +224 -0
  57. package/dist/core/sync-engine.d.ts +364 -0
  58. package/dist/core/sync-engine.d.ts.map +1 -0
  59. package/dist/core/sync-engine.js +2475 -0
  60. package/dist/core/table.d.ts +260 -0
  61. package/dist/core/table.d.ts.map +1 -0
  62. package/dist/core/table.js +461 -0
  63. package/dist/core/types.d.ts +952 -0
  64. package/dist/core/types.d.ts.map +1 -0
  65. package/dist/core/types.js +6 -0
  66. package/dist/crypto/encryption.d.ts +61 -0
  67. package/dist/crypto/encryption.d.ts.map +1 -0
  68. package/dist/crypto/encryption.js +216 -0
  69. package/dist/crypto/keys.d.ts +48 -0
  70. package/dist/crypto/keys.d.ts.map +1 -0
  71. package/dist/crypto/keys.js +54 -0
  72. package/dist/handshake/channel.d.ts +117 -0
  73. package/dist/handshake/channel.d.ts.map +1 -0
  74. package/dist/handshake/channel.js +245 -0
  75. package/dist/handshake/index.d.ts +216 -0
  76. package/dist/handshake/index.d.ts.map +1 -0
  77. package/dist/handshake/index.js +199 -0
  78. package/dist/handshake/qr-public.d.ts +3 -0
  79. package/dist/handshake/qr-public.d.ts.map +1 -0
  80. package/dist/handshake/qr-public.js +1 -0
  81. package/dist/handshake/qr.d.ts +100 -0
  82. package/dist/handshake/qr.d.ts.map +1 -0
  83. package/dist/handshake/qr.js +102 -0
  84. package/dist/index.d.ts +50 -0
  85. package/dist/index.d.ts.map +1 -0
  86. package/dist/index.js +50 -0
  87. package/dist/storage/credential-store.d.ts +122 -0
  88. package/dist/storage/credential-store.d.ts.map +1 -0
  89. package/dist/storage/credential-store.js +356 -0
  90. package/dist/storage/local-store.d.ts +64 -0
  91. package/dist/storage/local-store.d.ts.map +1 -0
  92. package/dist/storage/local-store.js +490 -0
  93. package/dist/storage/reset.d.ts +10 -0
  94. package/dist/storage/reset.d.ts.map +1 -0
  95. package/dist/storage/reset.js +18 -0
  96. package/package.json +76 -0
package/README.md ADDED
@@ -0,0 +1,706 @@
1
+ <p align="center">
2
+ <a href="https://github.com/TheUiTeam/interocitor">
3
+ <img src="../../docs/assets/hero.svg" alt="Interocitor" width="640" />
4
+ </a>
5
+ </p>
6
+
7
+ <p align="center">
8
+ <em>Encrypted local-first CRDT database and durable file store for browser apps.</em>
9
+ </p>
10
+
11
+ # @interocitor/core
12
+
13
+ End‑to‑end encrypted, local‑first app data over storage you already own
14
+ (Google Drive, WebDAV, Cloudflare R2, your own server). Interocitor gives
15
+ your app both structured CRDT rows and path-addressed files/images. The
16
+ cloud is a mailbox; merge and decryption happen on the device.
17
+
18
+ ## What it is
19
+
20
+ - A client-side CRDT database for structured state. Local reads/writes go
21
+ through an embedded store (IndexedDB in the browser). The engine ships
22
+ diffs, not queries.
23
+ - A durable file store for blobs/images that belong to the same mesh.
24
+ Files are encrypted, path-addressed, and overwritten/deleted explicitly;
25
+ they do not merge or compact.
26
+ - A CRDT over per‑column HLC values. Every device converges to the same
27
+ row state without a central merge authority.
28
+ - An end‑to‑end encryption layer. The remote sees ciphertext blobs and
29
+ enough metadata to route them; nothing else.
30
+ - A pluggable transport. Backends provide object storage operations for
31
+ sync objects and, optionally, first-class durable file operations. See
32
+ `docs/adapter-contract.md`.
33
+
34
+ ## Why
35
+
36
+ - **Offline‑first by construction.** Every read and write hits a local
37
+ store. Network is needed only to share state with other devices.
38
+ - **No server you have to operate.** The remote is dumb storage. You can
39
+ run the engine against a user's own Google Drive and ship zero
40
+ backend.
41
+ - **End‑to‑end encryption by default.** The default config encrypts
42
+ every change file and snapshot before it leaves the device.
43
+ - **Small, embeddable runtime.** No workers, no background services
44
+ required.
45
+
46
+ ## Quick start
47
+
48
+ ```ts
49
+ import { Interocitor } from '@interocitor/core';
50
+ import { GoogleDriveAdapter } from '@interocitor/core/adapters/google-drive';
51
+
52
+ const adapter = new GoogleDriveAdapter({ clientId: 'YOUR_GOOGLE_CLIENT_ID' });
53
+
54
+ const db = new Interocitor(adapter, {
55
+ dbName: 'my-app', // local DB name; keep stable across reloads
56
+ appName: 'My App', // shown in biometric prompts / OS keychain
57
+ remotePath: '/MyApp', // mesh-scoped folder on the remote
58
+ encrypted: true, // E2E encryption (default)
59
+ });
60
+
61
+ await db.init();
62
+ await db.connect();
63
+
64
+ // Write — local-first, no network required
65
+ await db.table('todos').add(
66
+ { text: 'Ship privacy-first sync', done: false },
67
+ { prefix: 'todo' },
68
+ );
69
+
70
+ // Share with another device — see "New device / restore" below
71
+ console.log('Passphrase:', db.getPassphrase());
72
+ ```
73
+
74
+ > The constructor accepts either `(adapter, config)` or `(config)` alone.
75
+ > Use the `(config)` form for local‑only mode (no remote). When you
76
+ > attach a remote later, call `setRemoteStorage(adapter)` and then
77
+ > `connect()`.
78
+
79
+ ## Mental model
80
+
81
+ ```mermaid
82
+ flowchart LR
83
+ A[Browser app] --> B[Interocitor engine]
84
+ B --> C[Local store]
85
+ B --> D[Encrypt locally]
86
+ D --> E[Adapter]
87
+ E --> F[Remote mailbox<br/>ciphertext only]
88
+ F --> E
89
+ E --> G[Decrypt locally]
90
+ G --> B
91
+ ```
92
+
93
+ The remote is a mailbox. For rows, the engine puts encrypted change
94
+ files into it and pulls down change files from peers. For files/images,
95
+ the engine puts encrypted durable objects under `files/` and reads or
96
+ deletes them by app path. Row merging happens on the device. There is no
97
+ server-side compute required for correctness.
98
+
99
+ Four artifact families live on the remote:
100
+
101
+ - **change files** — one per write batch, named `<HLC>-chg_<id>.json`;
102
+ - **snapshots** — periodic full row state, written by compaction;
103
+ - **durable files** — app blobs/images under `files/`, addressed by app path;
104
+ - **a manifest** — pointer to the current generation, plus mesh metadata.
105
+
106
+ See `docs/adapter-contract.md` for the full layout.
107
+
108
+ ## Security model
109
+
110
+ Encryption is on by default (`encrypted: true`). The engine uses
111
+ **AES‑GCM 256** with a key derived from a base58 passphrase.
112
+
113
+ What it protects:
114
+
115
+ - Row contents (field names, field values, table names) inside change
116
+ files and snapshots.
117
+ - Integrity of each entry (AES‑GCM authentication tag).
118
+
119
+ What it does **not** protect:
120
+
121
+ - File names (they encode the HLC and a device id).
122
+ - Manifest contents (mesh id, schema version, epoch, encrypted flag,
123
+ serverId).
124
+ - Sizes, write timing, device count.
125
+ - The local store on the device (IndexedDB / SQLite stores plaintext).
126
+
127
+ What a malicious remote can still do:
128
+
129
+ - Drop, withhold, replay, or roll back files.
130
+ - Observe activity timing and device identities.
131
+
132
+ For the threat model, the metadata table, and full mitigations see
133
+ `docs/security-model.md`.
134
+
135
+ ## Offline guarantees
136
+
137
+ | Operation | Network? |
138
+ | --- | --- |
139
+ | `new Interocitor()` | No |
140
+ | `init()` | No |
141
+ | `table.add()` / `patch()` / `replace()` / `delete()` | No |
142
+ | `table.row()` / `table.query()` / `table.where()` | No |
143
+ | `connect()` | Yes (if adapter configured) |
144
+ | `flush()` / `pull()` / `compact()` | Yes |
145
+
146
+ Writes are persisted locally and queued in an outbox. They survive
147
+ reload, crash, and offline periods. On reconnect the engine drains the
148
+ outbox to the remote.
149
+
150
+ ## Sync guarantees
151
+
152
+ - **Eventual convergence.** Two devices that have seen the same set of
153
+ change files (in any order) reach byte‑identical local state.
154
+ - **Per‑column merge.** Conflicts are resolved field‑by‑field, not at
155
+ the row level. Default strategy is `'remote-wins'` — see
156
+ "Conflict resolution".
157
+ - **Idempotent merge.** Replaying an already‑applied change is a no‑op.
158
+ Safe to re‑pull, safe to re‑process the same change file twice.
159
+ - **Per‑device HLC monotonicity.** A single device's HLCs strictly
160
+ increase. Cross‑device order is total but only as wall‑clocks allow.
161
+ - **Restore via snapshot.** A device that joins late (or rehydrates after
162
+ a long offline) loads the latest snapshot, then applies any change
163
+ files newer than the snapshot watermark.
164
+
165
+ What we do **not** guarantee:
166
+
167
+ - Cross‑device atomicity. A `db.batch()` is one atomic remote file, but
168
+ two unrelated batches from different devices are independent.
169
+ - Real‑time delivery. The default poll interval is 30 s; some adapters
170
+ (Cloudflare) layer push notifications on top.
171
+ - Recovery if the remote silently lies (drops writes, rolls back the
172
+ manifest). See `docs/security-model.md`.
173
+ - Recovery if the passphrase is lost — see "New device / restore".
174
+
175
+ ### Sync cadence
176
+
177
+ The engine adapts how often it polls the remote so it does useful work
178
+ when there is data to sync, and stays out of the way otherwise. This is
179
+ all automatic — there is no configuration knob.
180
+
181
+ - **Adaptive backoff.** Polling starts at `pollInterval` (30 s by
182
+ default). After every poll that merges zero entries the interval
183
+ doubles, capped at 60 s. Any poll that merges ≥ 1 entry resets the
184
+ interval back to base. The intent is to absorb idle bursts of clients
185
+ without hammering the remote, while still recovering immediately when
186
+ data starts flowing.
187
+ - **Tab visibility.** When the host page is hidden
188
+ (`document.visibilityState === 'hidden'`) the current poll interval is
189
+ multiplied by 10 — backgrounded tabs poll lazily. When the tab becomes
190
+ visible again the interval is reset to base **and** an immediate
191
+ `pull()` is fired so foregrounded data jumps in without waiting for
192
+ the next tick. This is a standard SWR‑style refresh‑on‑focus pattern.
193
+ The listener is wired during `connect()` and torn down by
194
+ `disconnect()`; environments without `document` (e.g. Node) skip it.
195
+ - **Push fallback.** Adapters that push invalidations (Cloudflare relay)
196
+ trigger an immediate pull and bypass the polling cadence entirely.
197
+ Polling is the safety net when the push channel is unavailable; the
198
+ WS connection itself uses bounded retries with a long cooldown after
199
+ repeated failed upgrades, so a misbehaving relay cannot DDoS itself.
200
+
201
+ ## Setup lifecycle
202
+
203
+ ```
204
+ new Interocitor(adapter?, config)
205
+
206
+
207
+ configureMesh({...}) ← optional; pin meshId / passphrase upfront
208
+
209
+
210
+ init() ← opens local store, restores credentials
211
+
212
+
213
+ setRemoteStorage(adapter) ← only if no adapter passed to ctor
214
+
215
+
216
+ connect() ← loads/creates manifest, pulls, flushes, polls
217
+ ```
218
+
219
+ `connect()` will auto‑`init()` if needed. App code should still treat
220
+ init as explicit so credential restore happens before any writes.
221
+
222
+ The `encrypted` flag is **pinned at mesh bootstrap** and cannot change
223
+ between sessions for the same `dbName`. Reconnecting with a different
224
+ mode throws a typed `MeshEncryptionMismatchError`.
225
+
226
+ ```ts
227
+ import { MeshEncryptionMismatchError } from '@interocitor/core';
228
+
229
+ try {
230
+ await db.connect();
231
+ } catch (err) {
232
+ if (err instanceof MeshEncryptionMismatchError) {
233
+ // err.expectedMode is the mode the remote was bootstrapped with
234
+ }
235
+ throw err;
236
+ }
237
+ ```
238
+
239
+ ## New device / restore
240
+
241
+ Joining a new device to an existing mesh requires three things:
242
+
243
+ 1. The **`meshId`** of the existing mesh.
244
+ 2. The **passphrase** that decrypts the mesh.
245
+ 3. Access to the same **remote mailbox** (the same `remotePath` on a
246
+ storage backend the new device can reach).
247
+
248
+ A production app should keep CRUD, sync lifecycle, and pairing separate:
249
+
250
+ ```text
251
+ lib/interocitor-db.ts engine, schema, local repository, credential primitives
252
+ lib/interocitor-sync.ts mesh id lifecycle, adapter, connect/disconnect, recovery
253
+ lib/interocitor-pairing.ts QR handshake only
254
+ ```
255
+
256
+ The pairing flow ships credentials over an ECDH relay handshake. The handshake relay base and sync adapter base are different concepts: a relay base is the temporary handshake-file path, such as `/Taska`; the Cloudflare adapter base URL is the concrete Worker route, such as `/sync/io/{meshId}`. The engine `remotePath` is still the mesh folder path used inside that adapter, such as `/Taska`.
257
+
258
+ ### Pairing intents
259
+
260
+ **Join QR** is for a device that does not have credentials yet:
261
+
262
+ 1. Joiner mints a fresh mesh id.
263
+ 2. Joiner creates a Cloudflare adapter with base URL `/sync/io/{meshId}`.
264
+ 3. Joiner calls `generateJoinQR()`.
265
+ 4. Existing paired device scans and pushes credentials.
266
+ 5. Joiner receives credentials from `credentials` on the result.
267
+ 6. Joiner applies the passphrase and connects to the minted mesh.
268
+
269
+ **Share QR** is for an existing mesh member inviting a new device:
270
+
271
+ 1. Existing device connects to the active mesh.
272
+ 2. Existing device calls `generateShareQR()` with `remotePath` and `passphrase`.
273
+ 3. New device scans and receives credentials from `handleScannedQR()`.
274
+ 4. New device applies credentials and connects using the adapter config from the payload.
275
+
276
+ `handleScannedQR()` returns `null` for join intent because the scanner pushed its own credentials. It returns credentials for share intent because the scanner received credentials. Always handle the return value:
277
+
278
+ ```ts
279
+ import { decodeQRPayload, handleScannedQR, parseQRFromUrl } from '@interocitor/core';
280
+ // Raw QR payload decoder is also available from '@interocitor/core/handshake/qr'.
281
+
282
+ const payload = parseQRFromUrl(location.hash) ?? decodeQRPayload(rawPastedPayload);
283
+ const received = await handleScannedQR({ adapter, relayBase: '/Taska', payload });
284
+
285
+ if (received) {
286
+ if (received.passphrase) db.setPassphrase(received.passphrase);
287
+ await connectFromPayload(received.remotePath);
288
+ }
289
+ ```
290
+
291
+ After the handshake the new device:
292
+
293
+ ```ts
294
+ const db = new Interocitor(adapter, {
295
+ dbName: 'my-app',
296
+ appName: 'My App',
297
+ remotePath: '/Taska',
298
+ passphrase: 'base58-from-handshake',
299
+ encrypted: true,
300
+ });
301
+
302
+ await db.init();
303
+ await db.connect(); // pulls the manifest, rehydrates from snapshot, then catches up
304
+ ```
305
+
306
+ On `connect()` the engine:
307
+
308
+ 1. Reads `manifest.json` from the remote.
309
+ 2. Compares stored `meshId` (from the credential store) against the live
310
+ one. Mismatch → `MeshCredentialMismatchError`.
311
+ 3. If local epoch < remote epoch, calls `rehydrate()` to load the
312
+ latest snapshot.
313
+ 4. Pulls any change files newer than the snapshot watermark.
314
+ 5. Starts polling.
315
+
316
+ > **If the passphrase is lost and no other device holds it, the mesh is
317
+ > unreadable.** The engine has no recovery path — the data is end‑to‑end
318
+ > encrypted and the key is the passphrase. Treat the passphrase as the
319
+ > only thing that matters; back it up out of band (1Password, paper,
320
+ > another device's `WebAuthnCredentialStore`).
321
+
322
+ For the credential store details (records, anchors, biometric paths)
323
+ see `docs/credential-store.md`.
324
+
325
+ ## Core API
326
+
327
+ ```ts
328
+ const db = new Interocitor(adapter, {
329
+ dbName: 'my-app',
330
+ appName: 'My App',
331
+ remotePath: '/MyApp',
332
+ schema,
333
+ });
334
+ await db.init();
335
+
336
+ await db.table('tasks').add({ title: 'Ship it', done: false }, { prefix: 'task' });
337
+ await db.table('tasks').patch(taskId, { done: true });
338
+ await db.table('tasks').replace(taskId, fullTask);
339
+ await db.table('tasks').delete(taskId);
340
+
341
+ await db.table('tasks').row(taskId); // single row
342
+ await db.table('tasks').query(); // all rows
343
+ await db.table('tasks').where('done').equals(false).orderBy('title');
344
+
345
+ await db.connect(); // attach + sync
346
+ await db.flush(); // force outbox drain
347
+ await db.pull(); // force pull
348
+ await db.compact(); // see docs/compaction.md
349
+ await db.disconnect(); // tear down
350
+
351
+ // Durable files — encrypted, not compacted, not merged
352
+ await db.putFile('receipts/may.pdf', pdfBytes, 'application/pdf');
353
+ const pdf = await db.getFile('receipts/may.pdf');
354
+ const fileMeta = await db.getFileMetadata('receipts/may.pdf');
355
+ await db.deleteFile('receipts/may.pdf');
356
+
357
+ // Images — first-class file helpers
358
+ await db.putImage('avatars/me.png', fileOrBlob);
359
+ const image = await db.getImage('avatars/me.png'); // { data, blob, metadata }
360
+ const blobUrl = await db.getImageBlobUrl('avatars/me.png'); // { url, revoke }
361
+ blobUrl.revoke();
362
+
363
+ // Credentials
364
+ db.getPassphrase();
365
+ db.setPassphrase(passphrase);
366
+ await db.secureWithBiometrics();
367
+ await db.restoreWithBiometrics();
368
+ await db.clearCredentials();
369
+
370
+ // Batched writes — one ChangeEntry per batch
371
+ await db.batch(async () => {
372
+ await db.table('todos').add({ title: 'a' });
373
+ await db.table('todos').patch(otherId, { done: true });
374
+ });
375
+ ```
376
+
377
+ ### Files and images
378
+
379
+ Durable files are application objects stored under the mesh `files/` namespace. They are encrypted with the same mesh key when encryption is enabled, but they are not CRDT rows: no merge, no replay, no compaction, no snapshot membership. A file at a path simply exists until overwritten or deleted.
380
+
381
+ ```ts
382
+ await db.putFile('attachments/report.pdf', bytes, 'application/pdf');
383
+
384
+ const bytes = await db.getFile('attachments/report.pdf');
385
+ const meta = await db.getFileMetadata('attachments/report.pdf');
386
+
387
+ await db.deleteFile('attachments/report.pdf');
388
+ ```
389
+
390
+ `StoredFileMetadata` includes path, size, content type, uploader device, upload time, stored/plaintext byte sizes, and backend-tracked access counters when available.
391
+
392
+ Images are convenience wrappers over files:
393
+
394
+ ```ts
395
+ await db.putImage('avatars/me.png', file); // File, Blob, ArrayBuffer, Uint8Array, data URL, or SVG string
396
+
397
+ const image = await db.getImage('avatars/me.png');
398
+ // image.data: Uint8Array
399
+ // image.blob: Blob
400
+ // image.contentType: image/png, image/jpeg, ...
401
+
402
+ const view = await db.getImageBlobUrl('avatars/me.png');
403
+ img.src = view.url;
404
+ view.revoke();
405
+ ```
406
+
407
+ ### Schema typing
408
+
409
+ ```ts
410
+ import { types } from '@interocitor/core';
411
+ import type { DatabaseSchemaDefinition } from '@interocitor/core';
412
+
413
+ const schema = {
414
+ tables: {
415
+ todos: {
416
+ fields: {
417
+ text: types.string,
418
+ done: types.boolean,
419
+ createdAt: types.index(types.date),
420
+ note: types.string.optional,
421
+ items: types.typed<TodoItem[]>('json'),
422
+ },
423
+ },
424
+ },
425
+ } satisfies DatabaseSchemaDefinition;
426
+
427
+ // Inferred row type:
428
+ // { text: string; done: boolean; createdAt: Date; note?: string; items: TodoItem[] }
429
+ ```
430
+
431
+ `.optional` makes the field optional in the inferred type. Indexed and
432
+ unique fields cannot be optional. `types.index(...)` marks a field for
433
+ efficient `where`/`orderBy`.
434
+
435
+ ### Conflict resolution
436
+
437
+ Per‑column CRDT with HLC. Default merge strategy: **`'remote-wins'`**.
438
+
439
+ > "Remote‑wins" is per‑column, not per‑row. When two devices write the
440
+ > same column on the same row, the merge keeps the value with the higher
441
+ > HLC. Because HLCs are timestamp‑first, this is "later wall‑clock
442
+ > wins, ties broken by device id". Calling it "remote‑wins" is a
443
+ > historical accident of where the merge runs (during pull); both sides
444
+ > apply the same rule and reach the same answer. Override per database,
445
+ > table, or field if you need `'lww'`, `'local-wins'`, or a custom
446
+ > `MergeFunction`.
447
+
448
+ Available strategies:
449
+
450
+ - `'lww'` — last‑write‑wins by HLC. Functionally identical to
451
+ `'remote-wins'` for this CRDT but keeps semantics explicit.
452
+ - `'remote-wins'` (default).
453
+ - `'local-wins'` — keep local on tie; remote still wins on a strictly
454
+ greater HLC.
455
+ - `MergeFunction` — `(local, remote, ctx) => result` for custom logic.
456
+
457
+ ### Deletion semantics
458
+
459
+ `delete()` writes a **tombstone**, not an immediate unlink. Tombstones:
460
+
461
+ - carry `deletedHlc` like any other write;
462
+ - hide the row from public reads and queries;
463
+ - keep an empty payload, so deleted user data does not linger inside the
464
+ tombstone;
465
+ - are needed so that slower devices cannot replay an older `add()` and
466
+ resurrect the row.
467
+
468
+ Re-inserting the same row id after a delete starts a fresh row
469
+ incarnation. Columns from the old incarnation are not carried forward;
470
+ a partial insert only contains the new columns.
471
+
472
+ Compaction can later hard-delete tombstones from snapshots. The engine
473
+ tracks per-device acknowledgements in `devices/<deviceId>.json`; once
474
+ all active devices have observed a manifest watermark, compaction
475
+ publishes `manifest.gcFloorHlc` as a point of no return. Tombstones with
476
+ `deletedHlc <= gcFloorHlc` are omitted from the next snapshot.
477
+
478
+ Devices not seen within `offlineGraceMs` (default seven days) are
479
+ excluded from GC consensus. If one wakes up with local outbox entries at
480
+ or before `gcFloorHlc`, the engine refuses to flush those entries and
481
+ rehydrates from the canonical snapshot instead.
482
+
483
+ ### Local store
484
+
485
+ The local store is a pluggable `LocalStoreAdapter`. Browser default is
486
+ IndexedDB. The Swift package ships SQLite. Tests use an in‑memory
487
+ implementation. Reads, writes, queries, and the outbox all go through
488
+ this interface.
489
+
490
+ ## Adapters
491
+
492
+ | Adapter | Use when | Notes |
493
+ | --- | --- | --- |
494
+ | `GoogleDriveAdapter` | You want zero infrastructure | OAuth in the browser; the user owns the data |
495
+ | `WebDAVAdapter` | Self‑hosted (Nextcloud, OwnCloud, custom WebDAV) | Easy to inspect remotely |
496
+ | `CloudflareAdapter` | You operate a worker; want push invalidations | Experimental |
497
+ | `MemoryAdapter` | Tests and demos | No persistence |
498
+
499
+ Implementing your own adapter: see `docs/adapter-contract.md` for
500
+ required semantics, consistency assumptions, and the contract test
501
+ suite.
502
+
503
+ ## Remote mailbox layout
504
+
505
+ For `remotePath: '/MyApp'`:
506
+
507
+ ```
508
+ /MyApp/
509
+ ├── manifest.json # pointer { currentGeneration, file }
510
+ ├── manifest-1.json # generation 1
511
+ ├── manifest-2.json # ...
512
+ ├── changes/
513
+ │ ├── head.json # { latestHlc } — pull fast path
514
+ │ └── <HLC>-chg_<id>.json # encrypted change entries
515
+ ├── devices/
516
+ │ └── <deviceId>.json # device metadata
517
+ ├── files/
518
+ │ └── <app path> # durable encrypted app files
519
+ └── mainline/
520
+ └── snapshot-<epoch>-<serverId>.json # encrypted snapshot
521
+ ```
522
+
523
+ `remotePath` is **mesh‑scoped**. One folder = one mesh = one logical
524
+ database. Two meshes sharing the same folder will fight over the
525
+ manifest and poison the remote.
526
+
527
+ ## Maintenance / compaction
528
+
529
+ Compaction collapses the change log into a snapshot, bumps the manifest,
530
+ prunes old change files, and advances the tombstone GC floor when active
531
+ devices have acknowledged the prior watermark. Sync works without it;
532
+ the remote folder just grows until *some* device compacts.
533
+
534
+ Two paths run automatically:
535
+
536
+ - **Immediate sampled** — after a flush of ≥ `compactAutoThreshold`
537
+ (default 50) ops, with probability ≈
538
+ `compactAutoSampleNumerator / compactAutoDeviceCount`.
539
+ - **Delayed two‑phase** — per‑write timer (10 ± 5 min) → check that
540
+ remote change files exceed `compactRemoteChangeThreshold` (default 2)
541
+ → second timer (15 ± 5 min) → run.
542
+
543
+ Both paths are deduped by a single in‑flight guard. You can also call
544
+ `db.compact()` manually.
545
+
546
+ > **The auto defaults and the recommended manual policy are different
547
+ > things.** The manual policy ("idle > 1 min, churn > 20") is what to
548
+ > gate a "Sync now" button on. The auto defaults are what runs without
549
+ > any button. See `docs/compaction.md`.
550
+
551
+ > **Compaction is not race‑safe across devices.** The adapter contract
552
+ > has no CAS/ETag write, so two simultaneous compactors can both
553
+ > overwrite the manifest pointer. Mitigations: rely on probabilistic
554
+ > avoidance for small meshes, or run with `serverManaged: true` and a
555
+ > single authorized writer.
556
+
557
+ Full protocol, events, lock story, device acknowledgement / GC-floor
558
+ rules, prune invariants, and tuning checklist: **`docs/compaction.md`**.
559
+
560
+ ## Connected stores
561
+
562
+ `engine.connectedStores` is a small **credential vault** for sub-stores
563
+ that conceptually belong to this Interocitor (for example a "reviews"
564
+ store derived from a "family planner" store). It does **not** construct
565
+ or run child engines — it only stores credentials so apps don't have to
566
+ reinvent that storage themselves.
567
+
568
+ Direction is one-way by construction: the parent stores credentials for
569
+ sub-stores, and sub-stores have no awareness of the parent. Anyone with
570
+ read access to the parent inherits read access to the sub-store
571
+ credentials persisted here.
572
+
573
+ ```ts
574
+ import type { ConnectedStoreCredentials } from '@interocitor/core';
575
+
576
+ await db.connectedStores.put({
577
+ id: 'reviews',
578
+ alias: 'family-reviews',
579
+ remotePath: '/family/reviews',
580
+ passphrase: 'review-pass',
581
+ encrypted: true,
582
+ dbName: 'reviews-db',
583
+ adapter: { kind: 'memory' },
584
+ metadata: { icon: 'star' },
585
+ });
586
+
587
+ const all: ConnectedStoreCredentials[] = await db.connectedStores.list();
588
+ const reviews = await db.connectedStores.get('reviews');
589
+ await db.connectedStores.remove('reviews');
590
+ ```
591
+
592
+ Notes:
593
+
594
+ - Credentials are persisted as a single JSON list under a dedicated meta
595
+ key inside the parent's local store. Sub-stores never appear as parent
596
+ tables or rows.
597
+ - `put(creds)` upserts by `id`, preserves `createdAt`, and refreshes
598
+ `updatedAt` automatically.
599
+ - The engine never reads `passphrase`/`adapter`/`remotePath` from these
600
+ records — apps construct their own `Interocitor` instances from them.
601
+
602
+ ## Schema migration
603
+
604
+ Interocitor now manages **local cache/index upgrades automatically**.
605
+ Adding or removing `types.index(...)` fields no longer requires bumping a
606
+ public schema version just to keep IndexedDB in sync. The local store
607
+ computes its own cache fingerprint, repairs missing indexes on open, and
608
+ falls back to scans if a stale cache slips through.
609
+
610
+ `schema.version` is now **optional** and only matters if you want an
611
+ explicit logical compatibility gate in the remote manifest. If you set
612
+ it, the engine writes it to `manifest.schema` and will reject manifests
613
+ written under a different logical version.
614
+
615
+ Use `schema.version` only for app-level data meaning changes such as:
616
+
617
+ 1. row shapes that old clients cannot safely read,
618
+ 2. `onInit` migrations that rewrite logical data,
619
+ 3. staged rollouts where you want explicit manifest compatibility checks.
620
+
621
+ Recommended pattern for logical migrations:
622
+
623
+ 1. Bump `schema.version` in your code.
624
+ 2. Implement a one-shot `onInit` migration that reads old rows from the
625
+ local store and writes the new shape back. Use `db.batch(...)` to
626
+ keep it atomic per row group.
627
+ 3. Trigger compaction after the migration so the snapshot is written
628
+ under the new logical version and old change files are pruned.
629
+ 4. During rollout, make sure old clients either tolerate both shapes or
630
+ are blocked by the manifest version mismatch on connect.
631
+
632
+ For breaking changes that cannot be rolled out gradually, the
633
+ heavier path is to bootstrap a fresh mesh, replicate data over, and
634
+ retire the old mesh. The engine does not automate this.
635
+
636
+ ## Events
637
+
638
+ ```ts
639
+ db.on(event => {
640
+ switch (event.type) {
641
+ case 'sync:start': /* pull began */ break;
642
+ case 'sync:complete': /* event.entriesMerged */ break;
643
+ case 'change': /* event.table, event.rowId, event.row */ break;
644
+ case 'delete': /* event.table, event.rowId */ break;
645
+ case 'flush:start': /* event.entryCount */ break;
646
+ case 'flush:complete': break;
647
+ case 'flush:error': /* event.error */ break;
648
+ case 'remote:poisoned': /* unrecoverable; see security-model.md */ break;
649
+ case 'credentials:meshMismatch': /* stored meshId != live; offer clearCredentials() */ break;
650
+ case 'compact:warning': /* outbox is large */ break;
651
+ // compact:auto:start / complete / skip / error / delayed:* — see docs/compaction.md
652
+ }
653
+ });
654
+ ```
655
+
656
+ ## What this is not
657
+
658
+ - not Firebase
659
+ - not Fireproof
660
+ - not PowerSync
661
+ - not Replicache
662
+ - not a hosted backend
663
+ - not a query engine over the cloud
664
+ - not a server‑trusted merge layer
665
+
666
+ ## What can go wrong
667
+
668
+ A short field guide. Detailed mitigations in the linked docs.
669
+
670
+ | Symptom | Likely cause | Where to look |
671
+ | --- | --- | --- |
672
+ | `MeshCredentialMismatchError` on connect | Same `dbName`, new mesh; stale credential record | `engine.clearCredentials()` then reconnect; `docs/credential-store.md` |
673
+ | `MeshEncryptionMismatchError` on connect | App flipped `encrypted` between sessions | Pin `encrypted` per `dbName`, never change |
674
+ | `remote:poisoned` event | Decode failure on a manifest, change file, or snapshot | `docs/security-model.md` — usually wrong key, schema drift, or remote tampering |
675
+ | Writes never appear on peer | Peer never compacted, or peer's poll interval is long, or remote dropped writes | Check `flush:complete` events; check remote folder by hand |
676
+ | Local store has rows that are "old" after re‑pair | Engine kept local data when you re‑paired with a fresh mesh | Either delete local DB on re‑pair, or accept the merge |
677
+ | Lost passphrase | No recovery | Passphrase is the key. Back it up out of band |
678
+ | Long‑offline device "lost" recent edits | Rehydrate replaced local state with the snapshot | Local writes already in the outbox survive; in‑flight uncommitted UI state does not |
679
+ | Compaction never runs | `autoCompact: false`, or no remote, or `compactAutoThreshold` never reached | `docs/compaction.md` — subscribe to `compact:auto:skip` |
680
+ | Two compactors race | No CAS in the adapter; small probability in small meshes | Use `serverManaged: true` for large meshes |
681
+
682
+ ## Tests
683
+
684
+ ```bash
685
+ yarn workspace @interocitor/core test
686
+ ```
687
+
688
+ Adapter contract tests (run for every adapter):
689
+
690
+ ```bash
691
+ yarn workspace @interocitor/core test webdav.adapter.contract
692
+ ```
693
+
694
+ ## Package context
695
+
696
+ Part of the Interocitor monorepo. See:
697
+
698
+ - root `README.md` — monorepo overview
699
+ - `docs/security-model.md` — threat model
700
+ - `docs/adapter-contract.md` — adapter requirements
701
+ - `docs/compaction.md` — compaction protocol & tuning
702
+ - `docs/credential-store.md` — credential persistence
703
+
704
+ ## License
705
+
706
+ MIT