event-sourced-collection 0.0.5 → 0.0.7

package/ARCHITECTURE.md

@@ -1,6 +1,7 @@
 # Architecture & Design Decisions
 
 Technical reference for contributors and anyone evaluating this library's tradeoffs.
+Usage and API: [README.md](./README.md).
 
 ---
 
@@ -29,32 +30,126 @@ This is event sourcing applied to client-side state, scoped to a single user's d
 │  │   e1     | ins  | t1  | false │  │   e9     |   50      | true   │ │
 │  │   e2     | upd  | t1  | true  │  │   e10    |   51      | false  │ │
 │  └───────────────────────────────┘  └───────────────────────────────┘ │
-│           ▲                                            │               │
-│    onInsert/onUpdate/onDelete                   acceptMutations        │
-│    hooks append to outbox                       replays inbox into     │
-│           │                                     state collections      │
+│           ▲            │                               │               │
+│           │     exhausted retries                acceptMutations       │
+│           │     or hard rejection                replays inbox into    │
+│           │            ▼                         state collections     │
+│    onInsert/    ┌──────────────┐                       │               │
+│    onUpdate/    │  deadletter  │                       │               │
+│    onDelete     │  terminal    │                       │               │
+│    hooks        └──────────────┘                       │               │
+│           │                                            ▼               │
 │  ┌──────────────┐  ┌──────────────┐  ┌──────────────┐                │
 │  │  users       │  │  todos       │  │  settings    │                │
 │  │  (persisted) │  │  (persisted) │  │  (persisted) │                │
 │  │  collection  │  │  collection  │  │  collection  │                │
 │  └──────────────┘  └──────────────┘  └──────────────┘                │
 │                                                                        │
-│  Pull cursor = max(synced globalSeq) across inbox (derived)           │
+│  syncmeta: { pullCursor, backendId, lastSyncAt, lastError }           │
+│  rowversions: last event per row (only when conflictDetection is on)  │
 └──────────────────────────────────────────────────────────────────────┘
-         │  push (outbox.sync = false)    │  pull (since derived cursor)
-         ▼                                ▼
+      │  push (batched, backoff)      │  pull (since syncmeta.pullCursor)
+      ▼                               ▼
 ┌──────────────────────────────────────────────────────────────────────┐
 │  Server                                                                │
 │                                                                        │
 │  events table (BIGSERIAL global_seq, event_id UNIQUE, payload JSONB)  │
+│  sync_backend table (backend_id — changes when the store is recreated)│
 │                                                                        │
-│  POST /api/events → assigns global_seq, deduplicates by event_id     │
-│  GET  /api/events?since=N → returns events after N, sorted ASC       │
+│  POST /api/events → assigns global_seq, deduplicates by event_id,     │
+│                     rejects stale writes when base_version is sent    │
+│  GET  /api/events?since=N → events after N sorted ASC, plus backendId │
 └──────────────────────────────────────────────────────────────────────┘
 ```
 
 ---
 
+## Source map
+
+```
+src/
+├── create-event-sourced-db.ts   # orchestration: wire collections, sync, status, APIs
+├── types.ts                     # public wire + config types
+├── sync.ts                      # transport normalization (handlers / URLs / SyncTransport)
+├── mock-sync-backend.ts         # in-memory server for tests
+├── internal/
+│   ├── constants.ts             # reserved ids, defaults, CONFLICT code
+│   ├── types.ts                 # replay / meta types shared by internals
+│   ├── hooks.ts                 # fire-and-forget lifecycle emitter
+│   ├── serial-queue.ts          # in-process sync serialization
+│   ├── sync-meta.ts             # syncmeta row: cursor, backendId, lastSync*
+│   ├── push.ts                  # due set, backoff, batching, dead-letter
+│   ├── pull.ts                  # pagination, origin skip, backend identity
+│   └── replay.ts                # upcast, acceptMutations, rowversions
+└── platforms/
+    ├── web-locks.ts             # browser SyncLock (ifAvailable)
+    ├── browser-event-sourced-db.ts
+    └── react-native-event-sourced-db.ts
+```
+
+`create-event-sourced-db.ts` should stay thin. Push, pull, and replay each have
+enough edge cases (backoff, tx-aware batching, backend reset, upcast asymmetry)
+that they were split out so a change in one path does not force a re-read of the
+whole engine.
+
+Platform packages only inject persistence + an optional lock. The core never
+imports Web Locks or SQLite drivers directly — that keeps Node tests and RN
+bundles free of browser APIs.
+
+### Reserved collections
+
+| Id | Role |
+| -- | ---- |
+| `outbox` | Local mutations; push due set |
+| `inbox` | Pulled events; replay / skip markers |
+| `deadletter` | Terminal push failures and unapplicable pulled events |
+| `syncmeta` | Singleton: cursor, backendId, clientId, lastSyncAt, lastError |
+| `rowversions` | Last applied event id per row (conflict detection only) |
+
+User collection ids must not collide with these.
+
+---
+
+## Sync pipeline (one `sync()` call)
+
+```
+sync() / manualSync()
+  └─ serial queue (this context)
+       └─ lock.tryRun (cross-tab, optional)
+            └─ onSyncStart
+                 ├─ pushOutbox
+                 │    due = !sync && (not failed OR retryable & nextAttemptAt ≤ now)
+                 │    batch by txId (never split a transaction)
+                 │    for each batch:
+                 │      stamp attemptCount / lastAttemptAt
+                 │      transport.push
+                 │      confirmed → sync=true, onEventPushed
+                 │      failed retryable → nextAttemptAt = now + backoff
+                 │      failed hard / maxAttempts → deadletter, onDeadLetter
+                 │      transport throw → backoff whole batch, stop with counts kept
+                 └─ pullInbox
+                      since = max(0, syncmeta.pullCursor - pullOverlap)
+                      for each page:
+                        reconcile backendId
+                          → resetCursor also requeues the synced outbox
+                        skip local origin (outbox.has || clientId match)
+                        insert inbox row → replayEvent → resolve
+                        replay threw → bump inbox attemptCount
+                          → under budget: halt, retry next sync
+                          → budget spent: inbound deadletter, cursor advances
+                        halt leaves cursor unmoved
+                      writePullCursor after each successful page
+                 ├─ pushOutbox again, only if a backend reset requeued events
+                 └─ writeSyncOutcome(lastSyncAt, lastError)
+            └─ onSyncComplete
+```
+
+`manualSync()` then additionally drains unresolved inbox rows (same replay
+path). Deferred lock acquisition returns `{ deferred: true }` without firing
+`onSyncStart` — another tab is already doing the work.
+
+---
+
 ## Design Decisions
 
 ### 1. Separate Outbox and Inbox (Not One Mixed Table)
@@ -68,20 +163,26 @@ This is event sourcing applied to client-side state, scoped to a single user's d
 | Mental model         | A single row could be "ours, pending" or "theirs, synced" | Direction is explicit: outbox = ours, inbox = theirs        |
 | Per-row `sync` flag  | Overloaded — means "pushed" or "applied" depending on row | Unambiguous — outbox `sync` = pushed, inbox `sync` = replayed |
 | Visualization        | Needs raw SQL access to inspect                          | Just `useLiveQuery(db.collections.outbox)`                  |
-| Cursor               | Dedicated `esdb_cursor` row to keep in step              | Derived from `max(synced globalSeq)` across inbox           |
+| Cursor               | Dedicated `esdb_cursor` row to keep in step              | A single `syncmeta` row, seeded from inbox state on upgrade |
 | Storage layer        | Hand-rolled SQLite table + indexes                       | Reuses TanStack DB persistence (reactive, multi-tab aware)  |
 
+Three more reserved collections support the sync loop: `deadletter` (events that
+will never be retried, in either direction), `syncmeta` (one row holding the pull
+cursor, the backend identity, and this device's own identity), and `rowversions`
+(per-row version index, only written when `conflictDetection` is enabled).
+
 **Per-row sync semantics:**
 
 - **outbox** — `sync: false` means the mutation has not been pushed yet. `sync: true` means the server accepted it (no errors) and assigned a `globalSeq`.
-- **outbox `syncStatus`** — `pending` means retryable, `synced` means confirmed, and `failed` means the server or custom push handler explicitly rejected that event. Failed rows also store `attemptCount`, `lastAttemptAt`, `lastError`, `lastErrorCode`, and `retryable`.
-- **inbox** — `sync: false` means the event was pulled but not yet replayed. `sync: true` means it has been applied on top of the local data, or it originated locally and was already applied before push.
+- **outbox `syncStatus`** — `pending` means eligible to send, `synced` means confirmed, and `failed` means the server rejected it but marked it retryable; `nextAttemptAt` records when the backoff window opens. A rejection that is *not* retryable never lingers as `failed` — it moves straight to `deadletter`.
+- **inbox** — `sync: false` means the event was pulled but not yet resolved. `sync: true` means resolved: applied on top of the local data, originated locally and already applied before push, or deliberately skipped. Skipped rows carry `skipped: true` and a `skipReason`, and still allow the cursor to advance.
+- **deadletter** — terminal. Rows carry a `direction` (`outbound` / `inbound`) and a `reason` (`rejected`, `maxAttemptsExceeded`, `conflict`, `replayFailed`, `manual`), and only move via `retryDeadLetter()` or `discardDeadLetter()`. `retryDeadLetter()` returns each event to the queue it came from: the outbox for outbound, the inbox for inbound.
 
 **FK ordering** is still preserved: the inbox replays server events sorted by `globalSeq`, so parents always precede children.
 
-**Tradeoff:** The pull cursor is derived by scanning synced inbox state rather than read from a dedicated row. For the target scope (single-user, modest event counts) this is negligible.
+**Why the cursor moved out of the inbox:** deriving it from `max(synced globalSeq)` meant pruning the inbox silently rewound the client. Storing it in `syncmeta` decouples log retention from sync position; the derived value is still read as a floor so existing databases upgrade without re-pulling.
 
-**Constraint:** Because both are persisted collections, durability and multi-tab coordination are inherited from TanStack DB's persistence layer rather than implemented here.
+**Constraint:** Because these are persisted collections, durability and multi-tab coordination are inherited from TanStack DB's persistence layer rather than implemented here.
 
 ---
 
@@ -226,21 +327,61 @@ const { ensureDb, db } = createBrowserEventSourcedDB({
 
 ### 9. `BIGSERIAL` for Server Ordering (Not UUIDv7)
 
-**Decision:** Server uses PostgreSQL's `BIGSERIAL` for `global_seq`.
+**Decision:** Server uses PostgreSQL's `BIGSERIAL` for `global_seq`. Client
+`eventId` remains UUIDv7 for uniqueness and dedup — those are different jobs.
 
 **Why:**
 
-| Factor              | BIGSERIAL                 | UUIDv7                            |
-| ------------------- | ------------------------- | --------------------------------- |
-| Storage per event   | 8 bytes                   | 16 bytes                          |
-| Index performance   | Faster integer comparison | Slower string comparison          |
-| Ordering guarantee  | Database-level monotonic  | Timestamp-based (clock skew risk) |
-| Readability         | `seq: 42`                 | `019abc12-...`                    |
-| Distributed writers | No (single server)        | Yes                               |
-
-Since there's one server (not distributed), `BIGSERIAL` is strictly superior.
+| Factor | BIGSERIAL (`global_seq`) | UUIDv7 |
+| ------ | ------------------------ | ------ |
+| Storage per event | 8 bytes | 16 bytes |
+| Index / cursor | Dense integer compare | Lexicographic string |
+| Who assigns order | One server (authority) | Whoever mints the UUID |
+| Multi-device → one API → one DB | Correct | Unnecessary for order |
+| Multiple independent write primaries | Needs a distributed sequencer | Helps generate unique sortable IDs without a central SERIAL |
+
+**“Distributed writers” means multiple databases minting order, not multiple
+phones.** Several devices pushing into one Postgres is the intended design.
+
+**Critical caveat — sequence values are allocated before commit.** Two concurrent
+inserts can take `global_seq` 50 and 51, and 51 can commit first. A client that
+pulls during that window sees 51, advances its cursor to 51, and **never sees
+50**. Multi-device traffic increases concurrency and thus this race; it does not
+require switching to UUIDv7. Server-minted UUIDv7 before commit has the same gap.
+
+How to handle it (pick at least one server-side approach):
+
+| Approach | Where | Notes |
+| -------- | ----- | ----- |
+| `pg_advisory_xact_lock` around insert | Server push | One writer assigns+commits at a time; simplest correct fix |
+| Serve only `global_seq < pg_snapshot_xmin(...)` | Server pull | Never returns in-flight sequences |
+| `pullOverlap` | Client | Fallback; narrows the window, does not close it |
+
+Annotated handlers (plain comments, not a checklist):
+[`examples/postgres-sync-server/`](./examples/postgres-sync-server/).
+
+Sketch:
+
+```ts
+// PUSH — one DB transaction per client txId group
+await client.query("BEGIN");
+// Keep assign+commit ordered so pullers don't skip a late-committing row
+await client.query("SELECT pg_advisory_xact_lock($1)", [EVENTS_SEQ_LOCK]);
+// Same eventId again → return the existing global_seq
+// baseVersion mismatch → CONFLICT, retryable: false
+// Sibling failure → ROLLBACK the whole group
+await client.query("COMMIT");
+
+// PULL
+// backendId change → client resets cursor after a DB wipe
+// xmin ceiling → don't serve sequences still in flight
+WHERE global_seq > $since
+  AND global_seq < pg_snapshot_xmin(pg_current_snapshot())
+```
 
-**Constraint:** If you ever need multiple independent write servers, you'd need to switch to UUIDv7 or a distributed sequence generator. For single-server (which covers 99% of apps), this is irrelevant.
+**Constraint:** Only if you run multiple independent write primaries would you
+replace central `BIGSERIAL` with a distributed id scheme — and you would still
+need a total-order story (or CRDTs), not “UUIDv7 as pull cursor” alone.
 
 ---
 
@@ -257,6 +398,272 @@ Since there's one server (not distributed), `BIGSERIAL` is strictly superior.
 
 **Tradeoff:** Slightly larger event payloads for deletes.
 
+Update and delete events additionally carry `previous`, the row state before the
+mutation. That makes every event invertible, which is the prerequisite for
+rollback, undo, and any future rebase support. Inserts have `previous: null`.
+
+---
+
+### 11. Unknown Events Are Recorded, Not Retried Forever
+
+**Decision:** A pulled event targeting a collection this client does not know
+about is written to the inbox, marked resolved with a `skipReason`, and the
+cursor moves past it. Configurable via `unknownEventHandling: "skip" | "fail"`,
+defaulting to `"skip"`.
+
+**Why:**
+
+Halting on an unrecognized event means the cursor never advances, so every
+subsequent sync re-fetches the same blocked event and **no later event ever
+lands**. That turns a single forward-compatibility mismatch into total sync
+failure — which is the normal state of affairs during any staged rollout where
+an old client meets events from a new one.
+
+Skipping keeps the pipeline draining. The event is still durably recorded, so
+nothing is lost and it stays visible for debugging.
+
+**Tradeoff:** A skipped event is not automatically re-applied after the client
+learns the collection. Query the inbox for `skipped: true` rows and replay them
+if you need that.
+
+**Constraint:** Use `"fail"` when you would rather stall than diverge — the
+event stays unresolved and is retried on every sync.
+
+---
+
+### 12. Conflicts Are Detected, Not Resolved (Opt-In)
+
+**Decision:** With `conflictDetection: true`, each mutation records the version
+of the row it was authored against (`baseVersion`, the `eventId` of the last
+event applied to that row) and sends it with the event. The server rejects the
+push with code `CONFLICT` when the row has moved on. Off by default.
+
+**Why not always on:** it costs a write to the `rowversions` index on every
+mutation *and* every replay. Single-device apps get nothing for that.
+
+**What it does not do:** there is no rebase. A losing event goes to the
+dead-letter queue with `reason: "conflict"` and stops there. Resolution is your
+application's job — show the user both versions, re-apply on top of the new
+state, or discard.
+
+**Without it,** the semantics are not "last writer wins" but **"stale writer
+wins"**: a pending local write pushed *after* a conflicting server event was
+pulled lands at a higher `global_seq` and overwrites it, with neither side able
+to detect that it happened. For single-user single-device this never comes up;
+for multi-device it is real, silent data loss on concurrently edited rows.
+
+Full parent links plus automatic rebase, as LiveStore does, remains out of
+scope.
+
+---
+
+### 13. Failed Pushes Retry With Backoff, Then Dead-Letter
+
+**Decision:** A rejection the server marks `retryable: true` is rescheduled with
+exponential backoff (`retry.baseDelayMs` doubling to `retry.maxDelayMs`).
+Anything else — or an event that exhausts `retry.maxAttempts` — is moved out of
+the outbox into `deadletter`.
+
+**Why the queue:** the outbox is ordered by `localSeq` and processed in order. A
+permanently rejected event left in place is re-examined on every sync forever
+and, worse, invites the reader to treat the head of the queue as "stuck". Moving
+it out keeps the outbox meaning exactly one thing: work that will still be
+attempted.
+
+**Why not retry forever:** an event the server will never accept (schema
+mismatch, revoked permission, validation failure) is not a transient fault.
+Retrying it indefinitely burns battery and hides the failure from the user.
+
+**Tradeoff:** dead-lettered events are user work that will be lost unless
+something surfaces them. `retryDeadLetter()` and `discardDeadLetter()` exist, but
+a UI that never reads `db.collections.deadletter` is silently dropping data.
+
+**History:** earlier, `syncStatus === "failed"` permanently filtered events out
+of the due set, so `retryable` / `attemptCount` metadata was write-only. The
+dead-letter path is what makes that metadata meaningful.
+
+**Backoff is deliberately jitter-free** so retry scheduling is reproducible in
+tests. Add jitter in your push handler if many clients share one server.
+
+---
+
+### 13b. Inbound Events Dead-Letter Too
+
+**Decision:** A pulled event whose `acceptMutations` call throws is not
+propagated as a pull error. The failure is recorded on the inbox row
+(`attemptCount`, `lastError`) and the page halts so the event is retried on the
+next sync. Once it has failed `retry.maxAttempts` times it is copied to
+`deadletter` with `direction: "inbound"` and `reason: "replayFailed"`, its inbox
+row is resolved as skipped, and the cursor advances past it.
+
+**Why:** replay used to rethrow. Because the cursor only advances past resolved
+events, one event the local schema could not accept — a constraint violation, a
+row referencing something this build does not have — stopped that client pulling
+*anything* ever again. Every later event queued behind it silently, and the only
+symptom was a repeating error in `lastError`. Outbound events had a full
+retry-then-park pipeline; inbound events had none, despite being the direction
+the client has no control over.
+
+**Why retry before parking:** replay failures are often ordering artefacts that
+resolve once an earlier event lands, so giving up on the first throw would park
+recoverable events. Halting the page preserves `globalSeq` ordering while the
+budget lasts.
+
+**Tradeoff:** advancing past a parked event means the client is knowingly
+inconsistent with the server for that row. That is strictly better than being
+consistent-but-frozen for every row, but it does mean `deadletter` must be
+surfaced for inbound rows too, not just outbound ones. `retryDeadLetter()` sends
+them back to the inbox and replays immediately rather than pushing them, which
+would otherwise re-send another device's event as if this one had authored it.
+
+---
+
+### 13c. Client Identity Is Persisted
+
+**Decision:** `clientId` defaults to a generated id written into the `syncmeta`
+row on first run and reused on every subsequent one, rather than a fresh value
+per process.
+
+**Why:** `isLocalOrigin` decides whether a pulled event is this device's own echo
+by checking `outbox.has(eventId) || event.clientId === clientId`. The `clientId`
+half exists precisely so that origin detection keeps working after the outbox has
+been pruned — but a per-process id makes that half useless the moment the page
+reloads. A client that had pruned its outbox and then rewound its cursor (via
+`pullOverlap`, or via a `resetCursor` backend mismatch) would re-apply its *own*
+history as if it were remote. Replaying an old insert is harmless; replaying an
+old delete silently destroys whatever now lives at that key.
+
+**Tradeoff:** the identity is scoped to the database file, so copying a SQLite
+file to a second device gives both the same identity and each will treat the
+other's events as its own echoes. Pass an explicit `clientId` if you clone
+databases.
+
+---
+
+### 14. Pushes Are Batched, With Progress Persisted Per Batch
+
+**Decision:** The outbox is sent in chunks of `pushBatchSize` (default 100), and
+each batch's confirmations are persisted before the next request goes out.
+
+**Why:** a device offline for a week comes back with thousands of pending
+events. One unbounded POST either times out or exceeds a body limit, and on
+failure *nothing* is durable — the next attempt does exactly the same thing and
+fails the same way. Batching makes the work resumable.
+
+A transport-level failure stops the loop and is reported alongside the counts
+from batches that already succeeded, rather than thrown, so partial progress is
+visible to the caller.
+
+**Batches never split a `txId`.** The server contract asks for each transaction
+to commit atomically, which is impossible if half its events arrive in a later
+request — and if that later request never succeeds, the server has permanently
+applied a partial transaction. Batching therefore packs whole transaction groups
+and lets a batch exceed `pushBatchSize` rather than split one.
+
+---
+
+### 15. Backend Identity Guards Against a Reset Server
+
+**Decision:** The pull response may carry a `backendId`. The client stores the
+first one it sees and compares on every subsequent sync. On mismatch the default
+policy `resetCursor` clears the inbox, re-pulls from zero, *and* marks every
+retained outbox event pending again so local history is re-uploaded.
+
+**Why:** the cursor is just an integer. Wiping or swapping the server restarts
+`global_seq` at 1 while the client keeps asking for events after 500. The server
+truthfully returns nothing, forever, with no error anywhere. Before this, that
+failure mode was undetectable from the client.
+
+Re-pulling from zero is safe because replay is an idempotent upsert.
+
+Requeuing the outbox matters just as much as resetting the cursor. Rows the
+client already flipped to `sync: true` describe data the replacement backend has
+never seen; without the requeue the client keeps them locally, believes they are
+safely synced, and never uploads them again — so a restored-from-backup server
+quietly loses every write made since the backup. Push is idempotent by `eventId`,
+so re-uploading to a *restored* backend is a no-op rather than a duplicate.
+
+The reset is only discovered during pull, after this sync's push has run, so the
+requeued events would otherwise sit unsent until the next sync. `pushPull` runs a
+second push pass when a reset requeued anything, making one `sync()` enough to
+recover. `baseVersion` is cleared on requeue because it names event ids from the
+old backend's history.
+
+**Tradeoff:** events already removed by `pruneSyncedEvents` cannot be re-uploaded
+— pruning trades recoverability for space, and this is where that bill arrives.
+Servers that do not send `backendId` keep the old behaviour: the check is skipped
+entirely rather than guessed at.
+
+---
+
+### 16. Events Carry a Schema Version
+
+**Decision:** Every authored event is stamped with `eventSchemaVersion`. On
+replay, an event whose version differs is passed through the optional
+`upcastEvent` hook.
+
+**Why:** payload shapes change. A device that was offline across a deploy
+replays events written by the old shape into code expecting the new one. Without
+a version there is nothing to branch on and the mismatch surfaces as corrupt
+data rather than an error.
+
+**Asymmetric default:** with no upcaster, *older* events are applied as-is with
+a warning (usually additive changes, usually fine), but *newer* events are
+refused via `unknownEventHandling`. This build cannot know what a future field
+means, and guessing is worse than skipping.
+
+**Upgrade note:** outbox rows written before `schemaVersion` / `baseVersion`
+existed can still sit in SQLite. `toOutboundEvent` and dead-letter construction
+default `schemaVersion` to `1` and `baseVersion` to `null` so a push after
+upgrade does not send `undefined` on the wire.
+
+---
+
+### 17. Leader Election Is Injected, Not Imported
+
+**Decision:** `createEventSourcedDB` accepts an optional `lock` implementing
+`tryRun(name, fn)`. The browser helper defaults to a Web Locks implementation;
+the core imports nothing platform-specific.
+
+**Why `tryRun` and not a queue:** if five tabs each queue a sync behind the
+lock, you get five sequential syncs where one would do. `tryRun` returns
+`{ acquired: false }` immediately when another context holds the lock, and the
+caller gets `{ deferred: true }` back from `sync()`.
+
+**Tradeoff:** this only elects a leader per sync call, not for the process
+lifetime as LiveStore's shared worker does. Writes still happen in every tab —
+it is TanStack DB's persistence layer, not this library, that coordinates those.
+
+---
+
+### 18. Lifecycle Hooks Observe, They Do Not Intercept
+
+**Decision:** `hooks` exposes fire-and-forget callbacks (`onMutation`,
+`onSyncStart`, `onDeadLetter`, and so on). A hook that throws is logged and
+swallowed. Hooks cannot cancel, delay, or rewrite an event.
+
+**Why:** the alternative — awaited, failable, mutating middleware — makes every
+hook part of the sync loop's correctness. One slow analytics call would stall
+sync; one thrown error would abort a push mid-batch. Observation is the common
+need; the rare cases that genuinely want to transform data already have
+`upcastEvent` (replay) and a custom transport (push/pull).
+
+**Tradeoff:** you cannot veto a mutation from a hook. Validate before calling
+`insert`/`update`/`delete` instead.
+
+---
+
+## Status and pruning
+
+`getSyncStatus()` scans the outbox for pending/failed counts and reads
+`syncmeta`. It is O(n) in outbox size and runs on every outbox / deadletter /
+syncmeta change when subscribers exist — fine for modest backlogs, worth
+caching if you keep tens of thousands of pending rows.
+
+`pruneSyncedEvents` only deletes `sync: true` rows. It writes the cursor first
+so pruning cannot rewind pull position. It does **not** prune `rowversions` or
+`deadletter` (see open issues).
+
 ---
 
 ## Pros
@@ -276,11 +683,16 @@ Since there's one server (not distributed), `BIGSERIAL` is strictly superior.
 
 - Requires a server (not P2P)
 - No real-time push (polling or explicit sync)
-- Event log grows unbounded (need pruning strategy for very long-lived apps)
-- No automatic conflict resolution beyond "server decides order"
+- No rollback or rebase — conflicts are detected and parked, never merged (decision #12)
+- Conflict detection is opt-in and costs an index write per mutation and per replay
+- Dead-lettered events are lost unless your UI surfaces them, in both directions (decisions #13, #13b)
+- Two devices editing the same row between syncs do not converge (see Open issues)
+- Leader election is per sync call, not a long-lived leader owning all writes (decision #17)
+- Log pruning is manual — call `pruneSyncedEvents()` yourself on a schedule
+- Offline transport failures still consume retry budget (see Open issues)
 - Cannot replay from event 0 to rebuild state (state is materialized separately)
 - Update events store full modified object (not a minimal diff)
-- Pull cursor is derived by scanning synced inbox state rather than stored as a single authoritative value
+- Materialization is a plain upsert, so there are no custom materializers or derived read models
 
 ---
 
@@ -465,15 +877,137 @@ We build 300 lines. We get 17,000+ lines of battle-tested infrastructure for fre
 
 ---
 
+## Open issues (ordered by production risk)
+
+These are known gaps, not forgotten TODOs. They are listed so contributors do
+not "fix" them without the surrounding context.
+
+### 0. Concurrent edits to one row do not converge
+
+Two devices that edit the same row between syncs end up disagreeing, and neither
+notices. Say `a` and `b` both edit row `t1`; `a`'s event lands at `globalSeq` 2
+and `b`'s at 3, so the server's answer is unambiguously `b`. `a` pulls both,
+skips its own event 2, applies 3, and is correct. `b` pulls both, applies `a`'s
+event 2, then reaches its *own* event 3 and skips it as local origin — so `b`
+finishes holding `a`'s value while everyone else holds `b`'s.
+
+The origin skip is not incidental: it is what stops a client re-applying its own
+pruned history (decision #13c). Dropping it would trade this bug for a worse one.
+
+**Direction:** replace the origin check with a per-row applied-sequence
+watermark. Record the highest `globalSeq` applied to each `(collectionId, key)`,
+updating it both on replay and on push confirmation, and apply an event only when
+its `globalSeq` exceeds it. That makes replay order-independent and idempotent,
+fixes this case, and subsumes the origin check. The cost is a watermark write per
+mutation and per replay — which is what `rowversions` already does under
+`conflictDetection`, currently opt-in precisely because of that cost. Turning it
+on unconditionally is the decision this needs.
+
+**Meanwhile:** `conflictDetection: true` turns the silent divergence into a
+visible `CONFLICT` dead-letter, which is the safer failure. Partitioning writes by
+owner avoids it entirely. Pinned by the "does not yet converge when two devices
+edit the same row concurrently" test.
+
+### 1. Connectivity vs retry budget
+
+Sync runs whether or not the device is online. Offline transport failures still
+increment `attemptCount` and schedule backoff. A long offline window can exhaust
+`maxAttempts` and dead-letter events the server never saw.
+
+**Direction:** only charge the retry budget when a server response was received
+(or gate push on an injectable `isOnline` / `navigator.onLine`).
+
+### 2. Outbox durability vs state durability
+
+`persistedCollectionOptions` wraps our mutation hook: it awaits the hook (the
+outbox append) and *then* calls `persistAndConfirmCollectionMutations` to write
+collection state. So the previously documented hazard — state persisting with no
+event — does not occur: if the outbox append rejects, state is never written and
+TanStack rolls the optimistic mutation back. That direction is pinned by the
+"rolls the row back when the outbox append fails" test.
+
+The remaining exposure is the mirror image. The outbox append has already
+committed by the time the state write runs, so a state write that fails leaves an
+event with no local row. The mutation rejects and the optimistic value is rolled
+back, but the event survives, gets pushed, and materialises on every *other*
+device. The authoring client then skips it on pull as its own echo and never
+materialises it — the one device that originated the row is the only one without
+it.
+
+The two writes want to be one transaction. LiveStore co-commits eventlog and
+state; the hook model cannot, since the two collections are persisted
+independently. Closing it properly means either a shared transaction across the
+outbox and the target collection, or reconciling the outbox against collection
+state at startup.
+
+### 3. `rowversions` growth
+
+With `conflictDetection: true`, one index row per touched key forever, including
+deletes. `pruneSyncedEvents` ignores it.
+
+### 4. Byte-size batching
+
+`pushBatchSize` counts events, not bytes. Large payloads (or one huge
+transaction) can still blow body limits. LiveStore-style ~900KB caps would
+complement tx-aware batching.
+
+### 5. Per-call leader only
+
+`localSeq` allocation has a TOCTOU window across tabs; `eventId` (uuidv7) breaks
+ties so order stays deterministic. Cross-tab write leadership is still TanStack's
+job, not ours.
+
+### 6. No `AbortSignal`
+
+A sync cannot be cancelled. bfcache / background tabs can hold the Web Lock
+until the browser times it out.
+
+### 7. Unbounded pull pages
+
+Non-advancing cursors stop the loop; a server that always advances with
+`hasMore: true` does not. A `maxPullPages` safety valve is missing.
+
+### 8. Status scan cost
+
+Full outbox scan on every mutation when a sync-status subscriber is attached.
+
+### Scope limits (not bugs)
+
+No rebase, no realtime subscribe, manual pruning only, plain upsert
+materialization, full-object updates (not diffs), no cold-start rebuild from
+event 0.
+
+### Example server notes
+
+The demo API does per-event conflict/dedup queries (fine for demos) and has no
+auth or user scoping — do not treat it as a production sync backend.
+
+---
+
+## Testing notes
+
+- Prefer `createMockSyncBackend()` over hand-rolled transports: it assigns
+  `globalSeq`, dedupes by `eventId`, paginates, and can inject outages /
+  rejections / backend resets.
+- Assert `pushBatchSizes` when testing batching; assert deadletter reasons for
+  conflict vs rejection vs max attempts.
+- Hook tests should prove a throwing hook does **not** fail insert/sync.
+- Repo vitest via `vite-plus` may lack native bindings in some environments;
+  a plain vitest config with `unstubGlobals: true` is enough for this package's
+  unit tests.
+
+---
+
 ## Future Directions
 
-1. **Event pruning** — compact old synced events after N days
-2. **Real-time subscribe** — SSE/WebSocket for instant server push
-3. **Selective sync** — sync only specific collections or subsets
-4. **Batched writes** — group rapid mutations into single events
-5. **Undo/redo** — leverage event log for time-travel
-6. **Schema migrations** — event upcasters for evolving payload shapes
+Highest-impact production gaps are detailed under [Open issues](#open-issues-ordered-by-production-risk). Product directions after those:
+
+1. **Real-time subscribe** — SSE/WebSocket for instant server push, replacing polling
+2. **Automatic rebase** — roll back divergent local events, apply upstream, re-apply on top. Events are invertible (`previous` is recorded), so the raw material exists
+3. **Long-lived leader** — a shared worker owning all writes, rather than per-call election
+4. **Automatic pruning** — a retention policy rather than a manual `pruneSyncedEvents()` call
+5. **Selective sync** — sync only specific collections or subsets
+6. **Undo/redo** — leverage the invertible event log for time-travel
 7. **Compression** — delta encoding for update events (store diff, not full object)
-8. **Retry queue** — exponential backoff for failed pushes
-9. **Multi-user awareness** — optional user_id scoping on the event table
-10. **Rebuild from events** — optional cold-start replay for disaster recovery
+8. **Multi-user awareness** — optional user_id scoping on the event table
+9. **Rebuild from events** — optional cold-start replay for disaster recovery