@happyvertical/smrt-web 0.38.2 → 0.38.4

package/AGENTS.md

@@ -28,6 +28,260 @@ mutations without hand-wiring cache keys or fetch/state.
   (`T[]`, `{ data }` envelopes, `{ error }` bodies → thrown
   `SmrtWebRequestError`).
 
+## The capability extension seam (#1755)
+
+`createSmrtCollection` takes an optional `capabilities: SmrtWebCapability[]` —
+plug-ins that hook the collection lifecycle so the three follow-on client slices
+(offline outbox #1762, persistence #1764, live SSE invalidation #1763-client)
+each live in their OWN module instead of contending on `index.ts`. Capabilities
+run in **array order** at six fixed points:
+
+1. `contributeCacheKey(ctx)` — ONCE, before construction. Returned segments
+   extend the base `smrt:(scope:)name` scheme so a capability can partition the
+   cache. Segments are spliced into the queryKey **just before the collection
+   name, so the name stays the LAST segment** — `invalidateRelated()`'s predicate
+   (and thus relationship-derived invalidation #1761 and a capability's own
+   `ctx.invalidate()`) identifies a collection by `key[key.length - 1]`, so a
+   name-last key is required or a collection using this hook would silently stop
+   invalidating. `cacheId` is an opaque engine id the predicate never reads, so it
+   appends.
+2. `warmStart(ctx)` — ONCE, before the first read. Returns rows that seed the
+   cache (the persistence rehydrate-from-disk path). **`initialData` wins**: it
+   is the fresher same-request SSR truth, so `warmStart` is consulted only when
+   `initialData` is undefined; the FIRST capability that returns rows contributes.
+   A **sync** return seeds inline; an **async** return is captured and
+   `SmrtWebCollection.preload()` AWAITS it before the engine's own load, so an
+   async rehydrate reliably suppresses the first `list()` on the preload path (a
+   subscribe-driven read racing an unresolved warmStart may still fetch once —
+   bounded, self-heals via the atomic seed updater).
+3. `wrapMutation(envelope, ctx)` — per mutation, BEFORE the fetcher. Return
+   `{ handled: true, result }` to take over the write (the fetcher is skipped and
+   `result` reconciles the optimistic row — the offline path); return
+   `{ handled: false }`/`undefined` to fall through. The FIRST `{ handled: true }`
+   wins and later capabilities' `wrapMutation` are skipped (`runWrapMutation`).
+   When a write is handled offline, the engine's post-mutation refetch AND
+   `invalidateRelated()` are **suppressed** (the handler returns `{ refetch:
+   false }`) — the offline write never hit the server, so a refetch of the server
+   list would DROP the optimistic row #1762's outbox must keep until it replays.
+   A mixed batch is conservative: any handled mutation suppresses the whole
+   handler's refetch (common case is one mutation per transaction). An unhandled
+   write refetches/invalidates exactly as before the seam.
+4. `onSettled(envelope, outcome, ctx)` — per mutation, after it settles, on BOTH
+   a successful persist (`{ ok: true, result }`) AND a fetcher throw
+   (`{ ok: false, error }`). Never swallows the throw — a rejected fetcher still
+   rolls the optimistic state back.
+5. `onAttach(ctx)` — ONCE, right after the engine collection is constructed. The
+   ONLY place a capability wires an external (non-mutation) trigger such as an
+   SSE subscription; `ctx.invalidate()` enters the same relationship-derived
+   invalidation the factory runs post-mutation.
+6. `teardown(ctx)` — inside `cleanup()`, AFTER the engine's own cleanup; awaited
+   if async.
+
+**Hook error isolation.** Every hook invocation is wrapped so one misbehaving
+capability cannot break the collection: a throwing `contributeCacheKey`,
+`warmStart` (sync throw or async reject), `onSettled`, `onAttach`, or `teardown`
+is logged via `console.warn` and skipped — a successful mutation still commits
+(no rollback), construction still completes, `cleanup()` still resolves, later
+capabilities' hooks still run, and an async warmStart rejection never escapes as
+an unhandled rejection.
+
+The context (`SmrtWebCapabilityContext`) and mutation envelope
+(`SmrtWebMutationEnvelope`) are typed **entirely in SMRT-owned terms**
+(`SmrtWebCollectionDefinition`, `SmrtCrudFetchers`, `SmrtWebRow`, plain TS) — no
+`@tanstack/*` type, so the seam stays inside the engine boundary. A capability
+needing deeper engine access reaches it through the existing
+`getEngineCollection()` unknown-bridge from its own module, never by widening
+these types.
+
+**No-op guarantee.** With `capabilities` undefined or `[]` every code path is
+byte-for-byte the collection of today: zero contributeCacheKey/warmStart/
+onAttach/teardown calls, `wrapMutation` always falls through to the real
+fetcher, `onSettled` runs nothing, and relationship-derived invalidation fires
+exactly as before. This PR ships the plug-in point and the shared durable-store
+foundation but **zero concrete capabilities** by design.
+
+### Shared durable-store foundation (#1755)
+
+`durable-store.ts` is the ONE SMRT-layer namespacing + wipe registry both the
+future outbox (#1762) and persistence (#1764) slices build on — pure
+bookkeeping, ZERO `@tanstack/*` imports.
+
+- `durableStoreNamespace(key)` — deterministic
+  `smrt-web:${apiBase}:${tenantId ?? '-'}:${identityId ?? '-'}:${manifestHash}`,
+  so a logout, tenant switch, or schema change each land on a different
+  namespace (`manifestHash` source is #1764's call; this layer is source-agnostic).
+- `registerDurableResource(namespace, resource)` → unregister; `wipeDurableStore(namespace)`
+  clears every registered `DurableResource` under a namespace (best-effort — a
+  rejected `clear()` doesn't abort the sweep) then drops it; a safe no-op on an
+  unknown/empty namespace.
+
+Rationale: the persistence slice's storage (SQLite-WASM/OPFS) and the outbox's
+IndexedDB queue are **separate storage engines**, so the shared foundation lives
+one level up — each slice keys its own storage primitive under a shared
+namespace, and `wipe()` clears both through the registry without the two modules
+importing each other. The outbox (#1762) is the first live consumer:
+`durableStoreNamespace(config.namespace)` is its IndexedDB dbName + leader-lock
+root, and it registers its queue as an `outbox` `DurableResource` so `wipe()`
+empties it.
+
+## Offline outbox (#1762)
+
+The first concrete capability over the seam — durable offline writes for
+opted-in collections. `offlineOutbox(config)` (from the root entry) returns a
+`SmrtWebCapability`; add it to a collection's `capabilities` array and mutations
+are captured in a durable IndexedDB queue that survives reloads/crashes, then
+replayed FIFO against the sync-apply batch contract (#1759) when connectivity
+returns, with exponential-backoff retries. A collection **without** it is
+byte-for-byte unaffected (the seam's no-op guarantee) — that IS the "opt-in per
+model" acceptance criterion.
+
+**Config** (`OfflineOutboxConfig`): `object` (the SAME generated definition
+passed to `createSmrtCollection` — its `name` is the sync-apply `object` route
+segment); `namespace` (a `DurableStoreKey` — folds api/tenant/identity/manifest;
+`manifestHash` is opaque caller-supplied config here, canonical source is
+#1764's call); `syncApplyBasePath` (default `/api/v1`; set `/api` for the
+generated SvelteKit route); `fetchFn`; `backoff` (`initialDelayMs`=1000,
+`multiplier`=2, `maxDelayMs`=60000); `onSyncStateChange` / `onConflict`
+(push callbacks, NOT a store — smrt-svelte wraps them later).
+
+**Hand-rolled, NOT `@tanstack/offline-transactions`.** That layer's public API
+is engine-typed (`Collection<…>`, `mutationFns`), so importing it would emit a
+`@tanstack/` specifier into `dist/*.d.ts` and FAIL
+`check-smrt-web-engine-boundary.mjs` (an unconditional dist-wide scan). It also
+pins `@tanstack/db` exactly and competes with smrt-web's own mutation lifecycle.
+So the outbox is a raw IndexedDB FIFO queue + native Web Locks, with ZERO new
+runtime dependency (only `fake-indexeddb` as a test devDep). The public surface
+is engine-free by construction (the boundary check is the proof).
+
+**Replay is sync-apply-ONLY** (`offline/engine.ts` → `POST
+{basePath}/sync/apply`), never `ctx.fetchers.create`. This is load-bearing:
+the normal REST create strips the client id (#1540) and mints a NEW server id,
+which would orphan the optimistic row the outbox is keeping; sync-apply's
+strict-insert path preserves the client UUID, so replay reconciles the exact
+optimistic row. `wrapMutation` enqueues then returns `{ handled: true, result:
+envelope.data }`, and the factory suppresses the post-mutation refetch +
+`invalidateRelated()` (its `{ refetch: false }` path) so the optimistic row is
+not dropped by a refetch of a server list that has never seen the offline write.
+
+**Idempotency / no duplicates.** Rows carry client-generated UUIDs and the
+endpoint is idempotent (`_insertOnly` create + no-op re-apply), so a batch that
+was sent but whose response was lost is blindly re-sent with no duplicate rows —
+the ambiguous-failure AC. Result → durable-transition mapping follows the
+sync-apply-contract's "Web outbox (#1762)" consumer notes: `applied` → remove /
+`synced`; `conflict` → remove + fire `onConflict` / `synced` (a conflict is a
+RESOLVED outcome, not a failure); retryable `write_failed` → keep + backoff /
+`pending`; `auth_required`/`forbidden` → PAUSE the loop until re-auth, keep
+queued; other terminal rejections → remove / `failed`; network/non-200/lost
+response → whole batch stays `pending`.
+
+**Shared, namespace-keyed engine** (`offline/engine.ts`,
+`getOrCreateOutboxEngine`): N collections under the same `namespace` share ONE
+ref-counted engine = ONE IndexedDB db + ONE leader lock + ONE FIFO queue. This
+is REQUIRED for correctness, not an optimization — independent per-collection
+locks would let two tabs each win a different collection's lock and both replay.
+The last collection to detach (via `teardown`) disposes the engine; the durable
+ROWS survive for the next load.
+
+**Web Locks leader election** (`offline/leader.ts`): with multiple tabs, exactly
+one replays the queue. A tab requests an EXCLUSIVE `navigator.locks` lock keyed
+`smrt-web-outbox-leader:<namespace>` and holds it while leader; the browser
+auto-releases on tab crash/close (no heartbeat) so the next tab takes over
+instantly. **Single-tab fallback (documented gap):** no `navigator.locks` →
+warn once + acquire leadership unconditionally; the outbox still replays but the
+multi-tab exactly-one-replayer guarantee does not hold across fallback tabs
+(NOT a BroadcastChannel shim in v1).
+
+**Observable state + bridge.** State events route by collection `object` (NOT by
+itemId), so rows REHYDRATED from IndexedDB after a reload still reach the
+reloaded collection's `onSyncStateChange` even though this session never
+enqueued them. `getOutboxHandle(durableStoreNamespace(key))` (a bridge like
+`getEngineCollection`) exposes `snapshot()` + `retry(itemId)` for trusted
+callers.
+
+**Reload-visibility gap (this slice's scope).** The outbox does NOT rehydrate
+the READ cache after a reload — a reloaded tab's `collection.toArray()` will not
+show captured-offline rows until a fetch runs; that read-side rehydrate is
+#1764's `warmStart`. So durability is proven via `OutboxHandle.snapshot()` / the
+raw IndexedDB store, not `collection.toArray()`. The WRITE side (capture →
+durable → exactly-once replay) is complete here.
+
+**Durable-store integration.** The engine registers its queue as an `outbox`
+`DurableResource` under `durableStoreNamespace(config.namespace)`, so
+`wipeDurableStore(namespace)` (a logout / tenant-switch) empties the queue; the
+namespace is also the IndexedDB dbName and the leader-lock root.
+
+### Live invalidation (#1763-client)
+
+`sse-client.ts` is the client half of live cache invalidation — ONE app-wide
+subscriber that turns the #1763 server's coarse change signals into collection
+refetches, so a dashboard reflects another session's writes without a manual
+refresh. It is a capability built on the seam above (`onAttach` registers the
+external trigger, `teardown` unregisters); it never touches the engine —
+`ctx.invalidate()` is the refetch primitive.
+
+- `createSmrtWebEventSubscriber(config)` — the ONE app-wide instance (mirror
+  `createSmrtWebClient`: construct once, pass to every collection; NOT
+  auto-derived per collection — one EventSource / one poll loop feeds all).
+  Config: `{ eventsUrl, changesUrl, fetchFn?, eventSourceFactory?,
+  pollIntervalMs?=5000, withCredentials?=true }`. Public surface:
+  `{ transport: 'sse'|'polling'|'idle', registerTable(table, invalidate) →
+  unregister, invalidateAll(), close() }`.
+- `liveInvalidation({ subscriber, tableName })` — the thin per-collection
+  capability. `tableName` is **EXPLICIT** config: a `SmrtWebCollectionDefinition`
+  has no physical-table field and STI children share one base table, so the
+  subscriber (which keys signals by physical table) must be told the table, not
+  guess it.
+
+**Wire contract it consumes** (both channels are generated by the #1763 server
+half in `packages/core/src/generators/`):
+
+- Push — the `_events` SSE route (`events-route.ts`): **NAMED** events
+  `event: change` / `event: resync`, `data` is `{table, operation, rowId,
+  tenantId}`, and the cursor `seq` is carried **only** in the SSE `id:` field
+  (the browser mirrors it to `MessageEvent.lastEventId`). `resync` uses that
+  `id:` as the fresh horizon for any later polling downgrade. Heartbeats are
+  `: heartbeat` comment lines EventSource ignores natively.
+- Pull — the `_changes` route (`changes-route.ts`): `GET {changesUrl}?since=
+  &tables=` → `{changes, cursor, resyncRequired?, resyncCursor?}`, the full
+  fallback.
+
+**The NAMED-event gotcha.** The frames are named, so the subscriber wires
+`es.addEventListener('change', …)` / `('resync', …)` — **`onmessage` never
+fires** for a named event and would silently receive nothing. A `change` frame's
+`data` is JSON-parsed **defensively**: malformed input is logged + dropped, never
+thrown back into the EventSource message loop (a throw there breaks later
+delivery).
+
+**No tenant logic (server enforces).** The wire carries only a signal, never a
+row payload; the subscriber just maps `table → invalidate`, and the refetch
+re-reads through the authorized collection routes. So authorization and
+tenant-scoping stay **entirely on the read path** — the `_events` stream is
+auth-guarded and tenant-scoped at connection open, `_changes` per request. This
+module does no tenant filtering and needs no identity.
+
+**SSE vs polling — feature-detect once, downgrade-on-fatal.** Construction
+feature-detects EventSource: present → `connectSse()`; absent → `startPolling()`.
+`transport` reflects it. A transient SSE `onerror` (readyState still OPEN) needs
+no code — the browser auto-reconnects, resending Last-Event-ID. A **fatal** error
+(readyState CLOSED — server 401 / route disabled) downgrades to polling **for the
+subscriber's life (no flap-back)**.
+
+**Polling fallback + resync cursor.** `poll()` skips work until at least one
+table is registered, then fetches `{changesUrl}?since=${lastSeq ?? 0}&tables=`
+for the current registered physical tables; each returned change invalidates
+its table and the cursor advances to the page cursor. `resyncRequired: true`
+(HTTP 200, with `cursor` still echoing the rejected value) invalidates
+everything and resumes from `resyncCursor`, the server's current horizon after
+the client performs a full refetch. A fetch **rejection** is a separate path:
+caught + logged, the interval keeps ticking (self-heals).
+
+**Idempotent → no client-side seq dedup.** Re-invalidating on a replayed change
+(a reconnect replays the tail) is safe — invalidation only schedules a background
+refetch, and relationship-derived invalidation is itself idempotent. So a
+replayed signal STILL fires, which is exactly what makes a reconnect miss no
+invalidation. `lastSeq` is a resume cursor for the poll fallback, not a dedup
+filter.
+
 ## The engine-absorption boundary (ratified conditions, #1761)
 
 1. **No engine types in the public API.** `@tanstack/*` types must never appear