event-sourced-collection 0.0.7 → 0.0.8
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/ARCHITECTURE.md +42 -42
- package/CHANGELOG.md +65 -0
- package/LICENSE +1 -1
- package/README.md +480 -145
- package/dist/browser-event-sourced-db-CCOAgE31.d.mts +121 -0
- package/dist/browser-event-sourced-db-CCOAgE31.d.mts.map +1 -0
- package/dist/browser.d.mts +30 -75
- package/dist/browser.d.mts.map +1 -1
- package/dist/browser.mjs +106 -58
- package/dist/browser.mjs.map +1 -1
- package/dist/create-event-sourced-db-handle-BAos-9pS.d.mts +105 -0
- package/dist/create-event-sourced-db-handle-BAos-9pS.d.mts.map +1 -0
- package/dist/{lazy-singleton-BuBmSQb3.mjs → create-event-sourced-db-handle-BPsVV1x6.mjs} +427 -25
- package/dist/create-event-sourced-db-handle-BPsVV1x6.mjs.map +1 -0
- package/dist/index.d.mts +244 -18
- package/dist/index.d.mts.map +1 -1
- package/dist/index.mjs +21 -4
- package/dist/index.mjs.map +1 -1
- package/dist/node.d.mts +101 -0
- package/dist/node.d.mts.map +1 -0
- package/dist/node.mjs +101 -0
- package/dist/node.mjs.map +1 -0
- package/dist/react-native.d.mts +82 -39
- package/dist/react-native.d.mts.map +1 -1
- package/dist/react-native.mjs +64 -27
- package/dist/react-native.mjs.map +1 -1
- package/dist/react.d.mts +1 -1
- package/dist/react.mjs.map +1 -1
- package/dist/types-BSdudVLh.d.mts +832 -0
- package/dist/types-BSdudVLh.d.mts.map +1 -0
- package/dist/{web-locks-C4279VL2.mjs → web-locks-Bi2f732X.mjs} +39 -2
- package/dist/web-locks-Bi2f732X.mjs.map +1 -0
- package/docs/SETUP.md +261 -0
- package/docs/usage.md +399 -0
- package/examples/postgres-sync-server/README.md +9 -9
- package/examples/postgres-sync-server/handlers.ts +2 -6
- package/examples/web-worker-sync/README.md +5 -5
- package/examples/web-worker-sync/collections.snippet.ts +1 -1
- package/package.json +36 -28
- package/dist/lazy-singleton-BuBmSQb3.mjs.map +0 -1
- package/dist/types-CIzhepuW.d.mts +0 -436
- package/dist/types-CIzhepuW.d.mts.map +0 -1
- package/dist/web-locks-C4279VL2.mjs.map +0 -1
package/ARCHITECTURE.md
CHANGED
|
@@ -98,13 +98,13 @@ bundles free of browser APIs.
|
|
|
98
98
|
|
|
99
99
|
### Reserved collections
|
|
100
100
|
|
|
101
|
-
| Id
|
|
102
|
-
|
|
|
103
|
-
| `outbox`
|
|
104
|
-
| `inbox`
|
|
105
|
-
| `deadletter`
|
|
106
|
-
| `syncmeta`
|
|
107
|
-
| `rowversions` | Last applied event id per row (conflict detection only)
|
|
101
|
+
| Id | Role |
|
|
102
|
+
| ------------- | ------------------------------------------------------------- |
|
|
103
|
+
| `outbox` | Local mutations; push due set |
|
|
104
|
+
| `inbox` | Pulled events; replay / skip markers |
|
|
105
|
+
| `deadletter` | Terminal push failures and unapplicable pulled events |
|
|
106
|
+
| `syncmeta` | Singleton: cursor, backendId, clientId, lastSyncAt, lastError |
|
|
107
|
+
| `rowversions` | Last applied event id per row (conflict detection only) |
|
|
108
108
|
|
|
109
109
|
User collection ids must not collide with these.
|
|
110
110
|
|
|
@@ -158,13 +158,13 @@ path). Deferred lock acquisition returns `{ deferred: true }` without firing
|
|
|
158
158
|
|
|
159
159
|
**Why:**
|
|
160
160
|
|
|
161
|
-
| Factor
|
|
162
|
-
|
|
|
163
|
-
| Mental model
|
|
164
|
-
| Per-row `sync` flag
|
|
165
|
-
| Visualization
|
|
166
|
-
| Cursor
|
|
167
|
-
| Storage layer
|
|
161
|
+
| Factor | One Mixed Table | Outbox + Inbox |
|
|
162
|
+
| ------------------- | --------------------------------------------------------- | ------------------------------------------------------------- |
|
|
163
|
+
| Mental model | A single row could be "ours, pending" or "theirs, synced" | Direction is explicit: outbox = ours, inbox = theirs |
|
|
164
|
+
| Per-row `sync` flag | Overloaded — means "pushed" or "applied" depending on row | Unambiguous — outbox `sync` = pushed, inbox `sync` = replayed |
|
|
165
|
+
| Visualization | Needs raw SQL access to inspect | Just `useLiveQuery(db.collections.outbox)` |
|
|
166
|
+
| Cursor | Dedicated `esdb_cursor` row to keep in step | A single `syncmeta` row, seeded from inbox state on upgrade |
|
|
167
|
+
| Storage layer | Hand-rolled SQLite table + indexes | Reuses TanStack DB persistence (reactive, multi-tab aware) |
|
|
168
168
|
|
|
169
169
|
Three more reserved collections support the sync loop: `deadletter` (events that
|
|
170
170
|
will never be retried, in either direction), `syncmeta` (one row holding the pull
|
|
@@ -174,7 +174,7 @@ cursor, the backend identity, and this device's own identity), and `rowversions`
|
|
|
174
174
|
**Per-row sync semantics:**
|
|
175
175
|
|
|
176
176
|
- **outbox** — `sync: false` means the mutation has not been pushed yet. `sync: true` means the server accepted it (no errors) and assigned a `globalSeq`.
|
|
177
|
-
- **outbox `syncStatus`** — `pending` means eligible to send, `synced` means confirmed, and `failed` means the server rejected it but marked it retryable; `nextAttemptAt` records when the backoff window opens. A rejection that is
|
|
177
|
+
- **outbox `syncStatus`** — `pending` means eligible to send, `synced` means confirmed, and `failed` means the server rejected it but marked it retryable; `nextAttemptAt` records when the backoff window opens. A rejection that is _not_ retryable never lingers as `failed` — it moves straight to `deadletter`.
|
|
178
178
|
- **inbox** — `sync: false` means the event was pulled but not yet resolved. `sync: true` means resolved: applied on top of the local data, originated locally and already applied before push, or deliberately skipped. Skipped rows carry `skipped: true` and a `skipReason`, and still allow the cursor to advance.
|
|
179
179
|
- **deadletter** — terminal. Rows carry a `direction` (`outbound` / `inbound`) and a `reason` (`rejected`, `maxAttemptsExceeded`, `conflict`, `replayFailed`, `manual`), and only move via `retryDeadLetter()` or `discardDeadLetter()`. `retryDeadLetter()` returns each event to the queue it came from: the outbox for outbound, the inbox for inbound.
|
|
180
180
|
|
|
@@ -332,12 +332,12 @@ const { ensureDb, db } = createBrowserEventSourcedDB({
|
|
|
332
332
|
|
|
333
333
|
**Why:**
|
|
334
334
|
|
|
335
|
-
| Factor
|
|
336
|
-
|
|
|
337
|
-
| Storage per event
|
|
338
|
-
| Index / cursor
|
|
339
|
-
| Who assigns order
|
|
340
|
-
| Multi-device → one API → one DB
|
|
335
|
+
| Factor | BIGSERIAL (`global_seq`) | UUIDv7 |
|
|
336
|
+
| ------------------------------------ | ----------------------------- | ----------------------------------------------------------- |
|
|
337
|
+
| Storage per event | 8 bytes | 16 bytes |
|
|
338
|
+
| Index / cursor | Dense integer compare | Lexicographic string |
|
|
339
|
+
| Who assigns order | One server (authority) | Whoever mints the UUID |
|
|
340
|
+
| Multi-device → one API → one DB | Correct | Unnecessary for order |
|
|
341
341
|
| Multiple independent write primaries | Needs a distributed sequencer | Helps generate unique sortable IDs without a central SERIAL |
|
|
342
342
|
|
|
343
343
|
**“Distributed writers” means multiple databases minting order, not multiple
|
|
@@ -351,11 +351,11 @@ require switching to UUIDv7. Server-minted UUIDv7 before commit has the same gap
|
|
|
351
351
|
|
|
352
352
|
How to handle it (pick at least one server-side approach):
|
|
353
353
|
|
|
354
|
-
| Approach
|
|
355
|
-
|
|
|
356
|
-
| `pg_advisory_xact_lock` around insert
|
|
357
|
-
| Serve only `global_seq < pg_snapshot_xmin(...)` | Server pull | Never returns in-flight sequences
|
|
358
|
-
| `pullOverlap`
|
|
354
|
+
| Approach | Where | Notes |
|
|
355
|
+
| ----------------------------------------------- | ----------- | ---------------------------------------------------------- |
|
|
356
|
+
| `pg_advisory_xact_lock` around insert | Server push | One writer assigns+commits at a time; simplest correct fix |
|
|
357
|
+
| Serve only `global_seq < pg_snapshot_xmin(...)` | Server pull | Never returns in-flight sequences |
|
|
358
|
+
| `pullOverlap` | Client | Fallback; narrows the window, does not close it |
|
|
359
359
|
|
|
360
360
|
Annotated handlers (plain comments, not a checklist):
|
|
361
361
|
[`examples/postgres-sync-server/`](./examples/postgres-sync-server/).
|
|
@@ -439,7 +439,7 @@ event applied to that row) and sends it with the event. The server rejects the
|
|
|
439
439
|
push with code `CONFLICT` when the row has moved on. Off by default.
|
|
440
440
|
|
|
441
441
|
**Why not always on:** it costs a write to the `rowversions` index on every
|
|
442
|
-
mutation
|
|
442
|
+
mutation _and_ every replay. Single-device apps get nothing for that.
|
|
443
443
|
|
|
444
444
|
**What it does not do:** there is no rebase. A losing event goes to the
|
|
445
445
|
dead-letter queue with `reason: "conflict"` and stops there. Resolution is your
|
|
@@ -447,7 +447,7 @@ application's job — show the user both versions, re-apply on top of the new
|
|
|
447
447
|
state, or discard.
|
|
448
448
|
|
|
449
449
|
**Without it,** the semantics are not "last writer wins" but **"stale writer
|
|
450
|
-
wins"**: a pending local write pushed
|
|
450
|
+
wins"**: a pending local write pushed _after_ a conflicting server event was
|
|
451
451
|
pulled lands at a higher `global_seq` and overwrites it, with neither side able
|
|
452
452
|
to detect that it happened. For single-user single-device this never comes up;
|
|
453
453
|
for multi-device it is real, silent data loss on concurrently edited rows.
|
|
@@ -499,7 +499,7 @@ row is resolved as skipped, and the cursor advances past it.
|
|
|
499
499
|
**Why:** replay used to rethrow. Because the cursor only advances past resolved
|
|
500
500
|
events, one event the local schema could not accept — a constraint violation, a
|
|
501
501
|
row referencing something this build does not have — stopped that client pulling
|
|
502
|
-
|
|
502
|
+
_anything_ ever again. Every later event queued behind it silently, and the only
|
|
503
503
|
symptom was a repeating error in `lastError`. Outbound events had a full
|
|
504
504
|
retry-then-park pipeline; inbound events had none, despite being the direction
|
|
505
505
|
the client has no control over.
|
|
@@ -529,7 +529,7 @@ by checking `outbox.has(eventId) || event.clientId === clientId`. The `clientId`
|
|
|
529
529
|
half exists precisely so that origin detection keeps working after the outbox has
|
|
530
530
|
been pruned — but a per-process id makes that half useless the moment the page
|
|
531
531
|
reloads. A client that had pruned its outbox and then rewound its cursor (via
|
|
532
|
-
`pullOverlap`, or via a `resetCursor` backend mismatch) would re-apply its
|
|
532
|
+
`pullOverlap`, or via a `resetCursor` backend mismatch) would re-apply its _own_
|
|
533
533
|
history as if it were remote. Replaying an old insert is harmless; replaying an
|
|
534
534
|
old delete silently destroys whatever now lives at that key.
|
|
535
535
|
|
|
@@ -547,7 +547,7 @@ each batch's confirmations are persisted before the next request goes out.
|
|
|
547
547
|
|
|
548
548
|
**Why:** a device offline for a week comes back with thousands of pending
|
|
549
549
|
events. One unbounded POST either times out or exceeds a body limit, and on
|
|
550
|
-
failure
|
|
550
|
+
failure _nothing_ is durable — the next attempt does exactly the same thing and
|
|
551
551
|
fails the same way. Batching makes the work resumable.
|
|
552
552
|
|
|
553
553
|
A transport-level failure stops the loop and is reported alongside the counts
|
|
@@ -566,7 +566,7 @@ and lets a batch exceed `pushBatchSize` rather than split one.
|
|
|
566
566
|
|
|
567
567
|
**Decision:** The pull response may carry a `backendId`. The client stores the
|
|
568
568
|
first one it sees and compares on every subsequent sync. On mismatch the default
|
|
569
|
-
policy `resetCursor` clears the inbox, re-pulls from zero,
|
|
569
|
+
policy `resetCursor` clears the inbox, re-pulls from zero, _and_ marks every
|
|
570
570
|
retained outbox event pending again so local history is re-uploaded.
|
|
571
571
|
|
|
572
572
|
**Why:** the cursor is just an integer. Wiping or swapping the server restarts
|
|
@@ -581,7 +581,7 @@ client already flipped to `sync: true` describe data the replacement backend has
|
|
|
581
581
|
never seen; without the requeue the client keeps them locally, believes they are
|
|
582
582
|
safely synced, and never uploads them again — so a restored-from-backup server
|
|
583
583
|
quietly loses every write made since the backup. Push is idempotent by `eventId`,
|
|
584
|
-
so re-uploading to a
|
|
584
|
+
so re-uploading to a _restored_ backend is a no-op rather than a duplicate.
|
|
585
585
|
|
|
586
586
|
The reset is only discovered during pull, after this sync's push has run, so the
|
|
587
587
|
requeued events would otherwise sit unsent until the next sync. `pushPull` runs a
|
|
@@ -607,8 +607,8 @@ replays events written by the old shape into code expecting the new one. Without
|
|
|
607
607
|
a version there is nothing to branch on and the mismatch surfaces as corrupt
|
|
608
608
|
data rather than an error.
|
|
609
609
|
|
|
610
|
-
**Asymmetric default:** with no upcaster,
|
|
611
|
-
a warning (usually additive changes, usually fine), but
|
|
610
|
+
**Asymmetric default:** with no upcaster, _older_ events are applied as-is with
|
|
611
|
+
a warning (usually additive changes, usually fine), but _newer_ events are
|
|
612
612
|
refused via `unknownEventHandling`. This build cannot know what a future field
|
|
613
613
|
means, and guessing is worse than skipping.
|
|
614
614
|
|
|
@@ -802,10 +802,10 @@ const offlineSync = createOfflineTransaction({
|
|
|
802
802
|
|
|
803
803
|
**How they help us:**
|
|
804
804
|
|
|
805
|
-
| Problem | Solution with Virtual Props
|
|
806
|
-
| --------------------------------------- |
|
|
807
|
-
| "Which todos haven't synced yet?" | `where(({ todo }) => eq(todo.$synced, false))`
|
|
808
|
-
| "Show a spinner on unsynced items" | Render based on `todo.$synced` in the UI
|
|
805
|
+
| Problem | Solution with Virtual Props |
|
|
806
|
+
| --------------------------------------- | ------------------------------------------------------------ |
|
|
807
|
+
| "Which todos haven't synced yet?" | `where(({ todo }) => eq(todo.$synced, false))` |
|
|
808
|
+
| "Show a spinner on unsynced items" | Render based on `todo.$synced` in the UI |
|
|
809
809
|
| "Skip re-logging server-applied events" | Already handled — `acceptMutations` bypasses the write hooks |
|
|
810
810
|
|
|
811
811
|
**Note:** This library no longer needs a re-logging guard. Server replay goes through `acceptMutations`, which never invokes the outbox-logging hooks (see Design Decision #4). Virtual props remain useful for surfacing optimistic vs confirmed state in the UI.
|
|
@@ -888,7 +888,7 @@ Two devices that edit the same row between syncs end up disagreeing, and neither
|
|
|
888
888
|
notices. Say `a` and `b` both edit row `t1`; `a`'s event lands at `globalSeq` 2
|
|
889
889
|
and `b`'s at 3, so the server's answer is unambiguously `b`. `a` pulls both,
|
|
890
890
|
skips its own event 2, applies 3, and is correct. `b` pulls both, applies `a`'s
|
|
891
|
-
event 2, then reaches its
|
|
891
|
+
event 2, then reaches its _own_ event 3 and skips it as local origin — so `b`
|
|
892
892
|
finishes holding `a`'s value while everyone else holds `b`'s.
|
|
893
893
|
|
|
894
894
|
The origin skip is not incidental: it is what stops a client re-applying its own
|
|
@@ -920,7 +920,7 @@ increment `attemptCount` and schedule backoff. A long offline window can exhaust
|
|
|
920
920
|
### 2. Outbox durability vs state durability
|
|
921
921
|
|
|
922
922
|
`persistedCollectionOptions` wraps our mutation hook: it awaits the hook (the
|
|
923
|
-
outbox append) and
|
|
923
|
+
outbox append) and _then_ calls `persistAndConfirmCollectionMutations` to write
|
|
924
924
|
collection state. So the previously documented hazard — state persisting with no
|
|
925
925
|
event — does not occur: if the outbox append rejects, state is never written and
|
|
926
926
|
TanStack rolls the optimistic mutation back. That direction is pinned by the
|
|
@@ -929,7 +929,7 @@ TanStack rolls the optimistic mutation back. That direction is pinned by the
|
|
|
929
929
|
The remaining exposure is the mirror image. The outbox append has already
|
|
930
930
|
committed by the time the state write runs, so a state write that fails leaves an
|
|
931
931
|
event with no local row. The mutation rejects and the optimistic value is rolled
|
|
932
|
-
back, but the event survives, gets pushed, and materialises on every
|
|
932
|
+
back, but the event survives, gets pushed, and materialises on every _other_
|
|
933
933
|
device. The authoring client then skips it on pull as its own echo and never
|
|
934
934
|
materialises it — the one device that originated the row is the only one without
|
|
935
935
|
it.
|
package/CHANGELOG.md
ADDED
|
@@ -0,0 +1,65 @@
|
|
|
1
|
+
# Changelog
|
|
2
|
+
|
|
3
|
+
All notable changes to `event-sourced-collection` are documented in this file.
|
|
4
|
+
|
|
5
|
+
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
|
|
6
|
+
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
|
7
|
+
|
|
8
|
+
## [0.0.7] - 2026-08-09
|
|
9
|
+
|
|
10
|
+
Reliability-focused release: dead-lettering, backend identity, retries, conflict detection, multi-tab locking, React helpers, and published examples.
|
|
11
|
+
|
|
12
|
+
### Added
|
|
13
|
+
|
|
14
|
+
- Dead-letter collection for permanently rejected or exhausted outbound/inbound events
|
|
15
|
+
- `backendId` / `backendMismatch` handling (`resetCursor` | `fail` | `ignore`) and `BackendMismatchError`
|
|
16
|
+
- Retry config with exponential backoff; batched push that preserves earlier batch progress on failure
|
|
17
|
+
- `getSyncStatus()` / `subscribeSyncStatus()` for pending/failed/synced observability
|
|
18
|
+
- `pruneSyncedEvents()` to compact confirmed outbox and resolved inbox rows
|
|
19
|
+
- Optional conflict detection via `rowversions` + `baseVersion`
|
|
20
|
+
- Lifecycle `hooks` (`onMutation`, `onSyncStart`, `onEventPushed`, `onBackendMismatch`, `onDeadLetter`, …)
|
|
21
|
+
- Event schema evolution: `eventSchemaVersion`, `upcastEvent`, `unknownEventHandling`
|
|
22
|
+
- Multi-tab sync lock via Web Locks (`createWebLocksSyncLock`, `supportsWebLocks`); `lock: null` to opt out
|
|
23
|
+
- `createMockSyncBackend()` for in-memory push/pull tests (pagination, rejections, backend reset)
|
|
24
|
+
- `event-sourced-collection/react` entry: `useManualSync`, `useSyncEnabled`, `formatManualSyncMessage`
|
|
25
|
+
- Published examples: `examples/web-worker-sync/` (Dedicated Worker transport) and `examples/postgres-sync-server/`
|
|
26
|
+
- Repository and bugs metadata on the package
|
|
27
|
+
|
|
28
|
+
### Changed
|
|
29
|
+
|
|
30
|
+
- Sync transport surface expanded (`SyncLock`, `SyncStatus`, `RetryConfig`, prune/hook types, …)
|
|
31
|
+
- Package `files` now includes `examples/`
|
|
32
|
+
|
|
33
|
+
## [0.0.6] - 2026-06-24
|
|
34
|
+
|
|
35
|
+
### Changed
|
|
36
|
+
|
|
37
|
+
- Documentation and project-structure guidance for browser apps (`collections.ts`, settings, sync wrappers)
|
|
38
|
+
|
|
39
|
+
## [0.0.5] - 2026-06-24
|
|
40
|
+
|
|
41
|
+
### Added
|
|
42
|
+
|
|
43
|
+
- `CollectionIndexDef` and typed collection index registration on defs
|
|
44
|
+
|
|
45
|
+
## [0.0.2] - 2026-06-24
|
|
46
|
+
|
|
47
|
+
### Added
|
|
48
|
+
|
|
49
|
+
- `createLazySingleton()` for deferred DB init (`ensure` / `proxy` / `reset`)
|
|
50
|
+
- `ARCHITECTURE.md` shipped with the package
|
|
51
|
+
|
|
52
|
+
## [0.0.1] - 2026-06-23
|
|
53
|
+
|
|
54
|
+
### Added
|
|
55
|
+
|
|
56
|
+
- Initial release: `createEventSourcedDB`, browser and React Native helpers
|
|
57
|
+
- Outbox / inbox sync over `SyncTransport` or `createHttpTransport`
|
|
58
|
+
- `createEventSourcedLogger`, `generateEventId` (UUIDv7)
|
|
59
|
+
- Platform entry points: `event-sourced-collection/browser`, `event-sourced-collection/react-native`
|
|
60
|
+
|
|
61
|
+
[0.0.7]: https://www.npmjs.com/package/event-sourced-collection/v/0.0.7
|
|
62
|
+
[0.0.6]: https://www.npmjs.com/package/event-sourced-collection/v/0.0.6
|
|
63
|
+
[0.0.5]: https://www.npmjs.com/package/event-sourced-collection/v/0.0.5
|
|
64
|
+
[0.0.2]: https://www.npmjs.com/package/event-sourced-collection/v/0.0.2
|
|
65
|
+
[0.0.1]: https://www.npmjs.com/package/event-sourced-collection/v/0.0.1
|
package/LICENSE
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
MIT License
|
|
2
2
|
|
|
3
|
-
Copyright (c) 2026
|
|
3
|
+
Copyright (c) 2026 locally-first contributors
|
|
4
4
|
|
|
5
5
|
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
6
|
of this software and associated documentation files (the "Software"), to deal
|