reflectdb 0.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (57) hide show
  1. package/LICENSE +21 -0
  2. package/README.md +1260 -0
  3. package/dist/cjs/client/index.cjs +1252 -0
  4. package/dist/cjs/client/index.d.cts +629 -0
  5. package/dist/cjs/client/storage/indexeddb.cjs +253 -0
  6. package/dist/cjs/client/storage/indexeddb.d.cts +85 -0
  7. package/dist/cjs/core/index.cjs +202 -0
  8. package/dist/cjs/core/index.d.cts +473 -0
  9. package/dist/cjs/react/index.cjs +1512 -0
  10. package/dist/cjs/react/index.d.cts +748 -0
  11. package/dist/cjs/server/drizzle.cjs +413 -0
  12. package/dist/cjs/server/drizzle.d.cts +233 -0
  13. package/dist/cjs/server/index.cjs +4486 -0
  14. package/dist/cjs/server/index.d.cts +1988 -0
  15. package/dist/cjs/svelte/index.cjs +1414 -0
  16. package/dist/cjs/svelte/index.d.cts +645 -0
  17. package/dist/cjs/transport/bun-ws.cjs +244 -0
  18. package/dist/cjs/transport/bun-ws.d.cts +215 -0
  19. package/dist/cjs/transport/polling.cjs +285 -0
  20. package/dist/cjs/transport/polling.d.cts +210 -0
  21. package/dist/cjs/transport/sse.cjs +312 -0
  22. package/dist/cjs/transport/sse.d.cts +205 -0
  23. package/dist/cjs/transport/ws.cjs +330 -0
  24. package/dist/cjs/transport/ws.d.cts +236 -0
  25. package/dist/cjs/vanilla/index.cjs +1430 -0
  26. package/dist/cjs/vanilla/index.d.cts +657 -0
  27. package/dist/client/index.d.ts +629 -0
  28. package/dist/client/index.js +106 -0
  29. package/dist/client/storage/indexeddb.d.ts +85 -0
  30. package/dist/client/storage/indexeddb.js +213 -0
  31. package/dist/core/index.d.ts +473 -0
  32. package/dist/core/index.js +56 -0
  33. package/dist/react/index.d.ts +748 -0
  34. package/dist/react/index.js +366 -0
  35. package/dist/server/drizzle.d.ts +233 -0
  36. package/dist/server/drizzle.js +9 -0
  37. package/dist/server/index.d.ts +1988 -0
  38. package/dist/server/index.js +3959 -0
  39. package/dist/shared/esm-3tkwvysa.js +54 -0
  40. package/dist/shared/esm-b7xs9cde.js +4 -0
  41. package/dist/shared/esm-rw7jjtrv.js +58 -0
  42. package/dist/shared/esm-wkwx6bd9.js +25 -0
  43. package/dist/shared/esm-ytrd3hbq.js +1007 -0
  44. package/dist/shared/esm-z1xse19c.js +369 -0
  45. package/dist/svelte/index.d.ts +645 -0
  46. package/dist/svelte/index.js +262 -0
  47. package/dist/transport/bun-ws.d.ts +215 -0
  48. package/dist/transport/bun-ws.js +140 -0
  49. package/dist/transport/polling.d.ts +210 -0
  50. package/dist/transport/polling.js +181 -0
  51. package/dist/transport/sse.d.ts +205 -0
  52. package/dist/transport/sse.js +208 -0
  53. package/dist/transport/ws.d.ts +236 -0
  54. package/dist/transport/ws.js +226 -0
  55. package/dist/vanilla/index.d.ts +657 -0
  56. package/dist/vanilla/index.js +278 -0
  57. package/package.json +253 -0
package/README.md ADDED
@@ -0,0 +1,1260 @@
1
+ # reflectdb
2
+
3
+ [![npm version](https://img.shields.io/npm/v/reflectdb.svg)](https://www.npmjs.com/package/reflectdb)
4
+ [![npm downloads](https://img.shields.io/npm/dm/reflectdb.svg)](https://www.npmjs.com/package/reflectdb)
5
+ [![CI](https://github.com/TimMikeladze/reflectdb/actions/workflows/ci.yml/badge.svg)](https://github.com/TimMikeladze/reflectdb/actions/workflows/ci.yml)
6
+ [![types](https://img.shields.io/npm/types/reflectdb.svg)](https://www.typescriptlang.org)
7
+ [![license](https://img.shields.io/npm/l/reflectdb.svg)](./LICENSE)
8
+
9
+ A real-time sync engine for TypeScript. Keeps a server-side database in sync with any number of browser clients — offline-first, with optimistic local writes, automatic conflict resolution, and end-to-end type inference.
10
+
11
+ You bring your own types and your own database. reflectdb handles the protocol, the op log, conflicts, reconnection, and subscriptions.
12
+
13
+ ```
14
+ ┌──────────────┐ writes ┌──────────────┐ writes ┌──────────────┐
15
+ │ Browser A │ ──────────▶ │ Server │ ◀────────── │ Browser B │
16
+ │ (optimistic) │ deltas │ (authoritive)│ deltas │ (optimistic) │
17
+ │ │ ◀────────── │ │ ──────────▶ │ │
18
+ └──────────────┘ └──────────────┘ └──────────────┘
19
+ ▲ │
20
+ │ offline ▼
21
+ └────── IndexedDB ───── op log (in-memory / SQLite / Postgres)
22
+ ```
23
+
24
+ ## Table of Contents
25
+
26
+ - [Why reflectdb](#why-reflectdb)
27
+ - [Features](#features)
28
+ - [Use Cases](#use-cases)
29
+ - [Installation](#installation)
30
+ - [Quick Start](#quick-start)
31
+ - [Recipes](#recipes)
32
+ - [WebSocket sync with SQLite + Drizzle](#websocket-sync-with-sqlite--drizzle)
33
+ - [Typed params for multi-tenant queries](#typed-params-for-multi-tenant-queries)
34
+ - [Authentication and room-based access control](#authentication-and-room-based-access-control)
35
+ - [Per-column merge for collaborative editing](#per-column-merge-for-collaborative-editing)
36
+ - [Custom conflict resolvers](#custom-conflict-resolvers)
37
+ - [Validating client payloads](#validating-client-payloads)
38
+ - [Ephemeral messages (cursors, presence, typing)](#ephemeral-messages-cursors-presence-typing)
39
+ - [Per-user query results](#per-user-query-results)
40
+ - [Server-driven game loops](#server-driven-game-loops)
41
+ - [Windowed sync and pagination](#windowed-sync-and-pagination)
42
+ - [Auto-generated REST API](#auto-generated-rest-api)
43
+ - [High availability with Postgres](#high-availability-with-postgres)
44
+ - [Whiteboard + Pictionary example](#whiteboard--pictionary-example)
45
+ - [Architecture](#architecture)
46
+ - [Core Concepts](#core-concepts)
47
+ - [Hybrid Logical Clocks](#hybrid-logical-clocks)
48
+ - [Conflict Resolution](#conflict-resolution)
49
+ - [The Sync Protocol](#the-sync-protocol)
50
+ - [Two stores, one sync](#two-stores-one-sync)
51
+ - [The Op Log and Resume](#the-op-log-and-resume)
52
+ - [API Reference](#api-reference)
53
+ - [`reflectdb/core`](#reflectdbcore)
54
+ - [`reflectdb/server`](#reflectdbserver)
55
+ - [`reflectdb/client`](#reflectdbclient)
56
+ - [`reflectdb/react`](#reflectdbreact)
57
+ - [`reflectdb/svelte`](#reflectdbsvelte)
58
+ - [`reflectdb/vanilla`](#reflectdbvanilla)
59
+ - [`reflectdb/transport/*`](#reflectdbtransport)
60
+ - [Configuration Reference](#configuration-reference)
61
+ - [Query definition](#query-definition)
62
+ - [Server configuration](#server-configuration)
63
+ - [`implement()` options](#implement-options)
64
+ - [Rate limiting](#rate-limiting)
65
+ - [Compaction](#compaction)
66
+ - [Client configuration](#client-configuration)
67
+ - [Storage adapters](#storage-adapters)
68
+ - [Transport configuration](#transport-configuration)
69
+ - [Development](#development)
70
+ - [License](#license)
71
+
72
+ ## Why reflectdb
73
+
74
+ Most real-time sync libraries force you to choose: CRDTs (powerful but opaque), or simple pub/sub (fast but brittle). reflectdb sits in the middle — **per-row operations** with **hybrid logical clocks** for causal ordering, validated through a server-side pipeline so your database stays authoritative.
75
+
76
+ You define your schema once, and the same types flow to both sides:
77
+
78
+ ```ts
79
+ const { rows, insert } = useSync("todos");
80
+ // ^? Todo[] ^? (id, { title, done, createdAt? }) => void
81
+ ```
82
+
83
+ No code generation. No glue layer. No second source of truth.
84
+
85
+ **Bring your own stack.** reflectdb is agnostic about:
86
+
87
+ - **Your database** — any TypeScript ORM, raw SQL driver, Map, or REST API works. The `query`/`mutate` callbacks hand you `db` untouched.
88
+ - **Your row types** — plain TypeScript types, Drizzle `$inferSelect`, Kysely, Prisma, anything. Declare them with `t<MyRow>()`.
89
+ - **Your HTTP server** — Bun, Node, Deno, Cloudflare Workers, anything fetch-compatible. Transports expose handler functions you wire to routes.
90
+
91
+ Optional bits (use what you want):
92
+
93
+ - **Drizzle ORM** — if you point `table` at a Drizzle table, row types are auto-inferred.
94
+ - **Server op log storage** — SQLite (for single-node) or Postgres (for HA). Omit it and the op log is in-memory.
95
+ - **React / Svelte bindings** — use the core client directly if you prefer.
96
+
97
+ ## Features
98
+
99
+ - **Real-time sync** over WebSocket, Server-Sent Events, or HTTP long-polling
100
+ - **Offline-first** — optimistic local writes, queued and replayed on reconnect
101
+ - **End-to-end type safety** — schema defines row types, query params, writable fields, and which columns the server owns
102
+ - **Per-row and per-column conflict resolution** — `lww`, `merge`, `server`, or a custom resolver
103
+ - **Causal ordering** via hybrid logical clocks (HLC) — no dependence on synchronized wall clocks
104
+ - **Pluggable storage** — in-memory, SQLite, or Postgres for the server op log; memory or IndexedDB for the browser
105
+ - **Auto-generated REST** — `server.rest()` turns your schema into CRUD endpoints that broadcast deltas
106
+ - **Room-based access control** — scope clients to `org/:orgId` or arbitrary patterns
107
+ - **Rate limiting** — global and per-table, fail-open
108
+ - **Op log compaction** — configurable retention for old accepted ops
109
+ - **High availability** — shared Postgres + optional cross-instance polling
110
+ - **Framework bindings** — React hooks, Svelte stores, and a vanilla-JS helper; the core client works anywhere
111
+ - **Ephemeral channels** — presence, cursors, typing indicators that never touch the op log
112
+ - **Windowed sync** — paginate large tables with `loadMore` + `useTotalCount`
113
+
114
+ ## Use Cases
115
+
116
+ - Collaborative editing (docs, whiteboards, spreadsheets)
117
+ - Multi-device note apps, todo apps, inbox-like UIs
118
+ - Live dashboards where multiple clients view and edit the same state
119
+ - Local-first apps that need to work offline and merge on reconnect
120
+ - Admin tools that should "just update" when someone else changes a row
121
+ - Field-service or retail apps on spotty networks
122
+ - Games or canvases with presence indicators and live cursors
123
+
124
+ ## Installation
125
+
126
+ ```bash
127
+ bun add reflectdb
128
+ # or
129
+ npm install reflectdb
130
+ ```
131
+
132
+ Ships both ESM and CommonJS, so `import` and `require` both work:
133
+
134
+ ```ts
135
+ import { defineSyncQueries, t } from "reflectdb"; // or "reflectdb/core"
136
+ ```
137
+ ```js
138
+ const { defineSyncQueries, t } = require("reflectdb");
139
+ ```
140
+
141
+ Everything else is a subpath — `reflectdb/server`, `reflectdb/client`, `reflectdb/react`, and so on. The bare `reflectdb` specifier is an alias for `reflectdb/core`, the surface both sides share.
142
+
143
+ Peer dependencies are all optional:
144
+
145
+ ```bash
146
+ bun add react # for reflectdb/react
147
+ bun add drizzle-orm # if you want auto-inferred row types from Drizzle tables
148
+ # Svelte + vanilla have no peer deps
149
+ ```
150
+
151
+ ## Quick Start
152
+
153
+ A complete sync server in ~30 lines. No ORM, no database — just plain types and an in-memory Map.
154
+
155
+ ### 1. Define your schema
156
+
157
+ ```ts
158
+ // schema.ts
159
+ import { defineSyncQueries, t } from "reflectdb/core";
160
+
161
+ export type Todo = {
162
+ id: string;
163
+ title: string;
164
+ done: boolean;
165
+ createdAt: Date;
166
+ };
167
+
168
+ export const queries = defineSyncQueries({
169
+ todos: {
170
+ row: t<Todo>(),
171
+ conflict: "lww",
172
+ serverSet: ["createdAt"], // server always sets this, clients cannot
173
+ },
174
+ });
175
+ ```
176
+
177
+ ### 2. Create the server
178
+
179
+ ```ts
180
+ // server.ts
181
+ import { serve } from "bun";
182
+ import { createSyncServer } from "reflectdb/server";
183
+ import { createWsServerTransport } from "reflectdb/transport/ws";
184
+ import { queries, type Todo } from "./schema";
185
+
186
+ const todos = new Map<string, Todo>();
187
+ const transport = createWsServerTransport();
188
+
189
+ const server = createSyncServer({ queries, transport, serverId: "s1" });
190
+
191
+ server.auth(async (req) => {
192
+ // validate req.headers.get("authorization")
193
+ return { userId: "user-1" };
194
+ });
195
+
196
+ server.implement("todos", {
197
+ query: () => [...todos.values()],
198
+ mutate: async (op) => {
199
+ if (op.type === "delete") todos.delete(op.rowId);
200
+ else todos.set(op.rowId, { id: op.rowId, ...(op.payload as Partial<Todo>) } as Todo);
201
+ },
202
+ serverSet: { createdAt: () => new Date() },
203
+ });
204
+
205
+ // Wire WebSocket handlers to your HTTP server
206
+ serve({
207
+ port: 3001,
208
+ fetch(req, srv) {
209
+ const url = new URL(req.url);
210
+ if (url.pathname === "/sync") {
211
+ const clientId = crypto.randomUUID();
212
+ if (srv.upgrade(req, { data: { clientId } })) return;
213
+ }
214
+ return new Response("ok");
215
+ },
216
+ websocket: {
217
+ open(ws) { transport.handleOpen(ws.data.clientId, ws); },
218
+ message(ws, data) { transport.handleMessage(ws.data.clientId, String(data)); },
219
+ close(ws) { transport.handleClose(ws.data.clientId); },
220
+ pong(ws) { transport.handlePong(ws.data.clientId); },
221
+ },
222
+ });
223
+ ```
224
+
225
+ ### 3. Connect from the browser
226
+
227
+ ```tsx
228
+ // app.tsx
229
+ import { SyncProvider, useSync, useSyncStatus } from "reflectdb/react";
230
+ import { createIndexedDBStorage } from "reflectdb/client/storage/indexeddb";
231
+
232
+ export function App() {
233
+ return (
234
+ <SyncProvider
235
+ url="ws://localhost:3001/sync"
236
+ token="..."
237
+ tables={["todos"]}
238
+ storage={createIndexedDBStorage({ dbName: "myapp" })}
239
+ >
240
+ <TodoList />
241
+ </SyncProvider>
242
+ );
243
+ }
244
+
245
+ function TodoList() {
246
+ const { rows, insert, update, remove } = useSync("todos");
247
+ const status = useSyncStatus();
248
+
249
+ return (
250
+ <div>
251
+ <p>Status: {status}</p>
252
+ {rows.map((t) => (
253
+ <label key={t.id}>
254
+ <input type="checkbox" checked={t.done} onChange={() => update(t.id, { done: !t.done })} />
255
+ {t.title}
256
+ <button onClick={() => remove(t.id)}>x</button>
257
+ </label>
258
+ ))}
259
+ <button onClick={() => insert(crypto.randomUUID(), { title: "New", done: false })}>
260
+ Add
261
+ </button>
262
+ </div>
263
+ );
264
+ }
265
+ ```
266
+
267
+ Open two tabs — edits in one appear in the other within a round-trip. Close the laptop, edit offline, reopen — pending ops replay automatically.
268
+
269
+ ## Recipes
270
+
271
+ The repo ships one end-to-end example — [`examples/whiteboard/`](./examples/whiteboard/) — a collaborative drawing app with two modes (freeform and **Pictionary**). It exercises the patterns below in one place. The snippets here are minimal, copy-paste-friendly references; see the example for how they fit together.
272
+
273
+ ### WebSocket sync with SQLite + Drizzle
274
+
275
+ If you use Drizzle, point `table` at it and row types flow automatically. Swap the Map for `bun:sqlite` + Drizzle and add a persistent op log:
276
+
277
+ ```ts
278
+ import { Database } from "bun:sqlite";
279
+ import { drizzle } from "drizzle-orm/bun-sqlite";
280
+ import { sqliteTable, text, integer } from "drizzle-orm/sqlite-core";
281
+ import { eq } from "drizzle-orm";
282
+ import { defineSyncQueries } from "reflectdb/core";
283
+ import { createSyncServer, createSqliteStorage } from "reflectdb/server";
284
+
285
+ const todos = sqliteTable("todos", {
286
+ id: text("id").primaryKey(),
287
+ title: text("title").notNull(),
288
+ done: integer("done", { mode: "boolean" }).notNull().default(false),
289
+ createdAt: integer("created_at", { mode: "timestamp" }).notNull(),
290
+ });
291
+
292
+ const queries = defineSyncQueries({
293
+ todos: { table: todos, conflict: "lww", serverSet: ["createdAt"] },
294
+ });
295
+
296
+ const db = drizzle(new Database("app.db"));
297
+ const storage = createSqliteStorage({ path: "sync.db" });
298
+
299
+ const server = createSyncServer({ queries, db, transport, storage, serverId: "s1" });
300
+
301
+ server.implement("todos", {
302
+ query: (_ctx, db) => db.select().from(todos),
303
+ mutate: async (op, _ctx, db) => {
304
+ if (op.type === "delete") {
305
+ await db.delete(todos).where(eq(todos.id, op.rowId));
306
+ } else {
307
+ await db.insert(todos)
308
+ .values({ id: op.rowId, ...op.payload })
309
+ .onConflictDoUpdate({ target: todos.id, set: op.payload });
310
+ }
311
+ },
312
+ serverSet: { createdAt: () => new Date() },
313
+ });
314
+ ```
315
+
316
+ ### Typed params for multi-tenant queries
317
+
318
+ Declare query params with `t<T>()` so the client must pass them and the server can use them to scope queries:
319
+
320
+ ```ts
321
+ import { defineSyncQueries, t } from "reflectdb/core";
322
+
323
+ type Post = { id: string; title: string; orgId: string };
324
+
325
+ const queries = defineSyncQueries({
326
+ posts: {
327
+ row: t<Post>(),
328
+ params: t<{ orgId: string }>(),
329
+ tables: ["posts"], // change-detection hint for delta computation
330
+ pk: "id",
331
+ conflict: "lww",
332
+ readonly: ["orgId"], // clients cannot write this
333
+ },
334
+ });
335
+
336
+ // server
337
+ server.implement("posts", {
338
+ query: (ctx, kyselyDb) =>
339
+ kyselyDb.selectFrom("posts").where("orgId", "=", ctx.params.orgId).selectAll().execute(),
340
+ mutate: async (op, ctx, kyselyDb) => { /* ... */ },
341
+ });
342
+
343
+ // client
344
+ client.sync("posts", { orgId: "org-42" });
345
+ ```
346
+
347
+ Works with any ORM or raw driver.
348
+
349
+ ### Authentication and room-based access control
350
+
351
+ `auth()` runs on every connection. Return an `AuthContext` — anything with a `userId`. It's passed to every `query`, `mutate`, and `authorize` call.
352
+
353
+ ```ts
354
+ server.auth(async (req) => {
355
+ const token = req.headers.get("authorization")?.replace("Bearer ", "");
356
+ const session = await validateToken(token);
357
+ if (!session) throw new Error("unauthorized");
358
+ return { userId: session.userId, orgId: session.orgId };
359
+ });
360
+ ```
361
+
362
+ For multi-tenant apps, use `room()` to pin a client to a subset of data:
363
+
364
+ ```ts
365
+ server.room("org/:orgId", async ({ params, auth }) => {
366
+ if (!auth.memberships.includes(params.orgId)) {
367
+ throw new Error("not a member of this org");
368
+ }
369
+ return { scope: { orgId: params.orgId } };
370
+ });
371
+ ```
372
+
373
+ The whiteboard example wires this up with [better-auth](https://better-auth.com) — see [`examples/whiteboard/auth.ts`](./examples/whiteboard/auth.ts).
374
+
375
+ ### Per-column merge for collaborative editing
376
+
377
+ When two users edit different fields of the same row, `lww` would throw one write away. `merge` keeps both:
378
+
379
+ ```ts
380
+ const queries = defineSyncQueries({
381
+ docs: { row: t<Doc>(), conflict: "merge" }, // per-column HLCs
382
+ });
383
+ ```
384
+
385
+ ```
386
+ User A writes { title: "Hello" } at HLC 100
387
+ User B writes { body: "world" } at HLC 200
388
+ → Result: { title: "Hello", body: "world" } (both accepted)
389
+ ```
390
+
391
+ ### Custom conflict resolvers
392
+
393
+ For domain logic — counters, highest-bid-wins, append-only lists — supply a resolver:
394
+
395
+ ```ts
396
+ const queries = defineSyncQueries({
397
+ auctions: {
398
+ row: t<Auction>(),
399
+ conflict: {
400
+ policy: "custom",
401
+ resolve: (incoming, existing) => {
402
+ const bid = (incoming.payload.bid as number) ?? 0;
403
+ if (bid <= (existing.row?.bid as number ?? 0)) {
404
+ throw new Error("bid too low"); // rejects the op
405
+ }
406
+ return { row: { ...existing.row, ...incoming.payload } };
407
+ },
408
+ },
409
+ },
410
+ });
411
+ ```
412
+
413
+ ### Validating client payloads
414
+
415
+ `t<MyRow>()` is a **compile-time** phantom — it erases at runtime. reflectdb validates
416
+ protocol structure (an op's `payload` must be a non-array object or null) but never
417
+ its contents, so a client can send `{ title: 12345 }` or extra keys and they reach
418
+ your `mutate` untouched. `readonly` and `serverSet` strip named fields; they don't
419
+ type-check what's left.
420
+
421
+ Validate in `mutate`, with whatever library you already use — reflectdb has no opinion
422
+ and no dependency here:
423
+
424
+ ```ts
425
+ import { z } from "zod";
426
+ import { MutationError } from "reflectdb/core";
427
+
428
+ const Todo = z.object({
429
+ id: z.string(),
430
+ title: z.string().min(1).max(200),
431
+ done: z.boolean(),
432
+ }).strict(); // reject unknown keys instead of passing them through
433
+
434
+ server.implement("todos", {
435
+ query: (ctx, db) => db.select().from(todos),
436
+ mutate: async (op, ctx, db) => {
437
+ if (op.type === "delete") {
438
+ await db.delete(todos).where(eq(todos.id, op.rowId));
439
+ return;
440
+ }
441
+ // Updates carry a partial delta, not a whole row.
442
+ const schema = op.type === "insert" ? Todo : Todo.partial();
443
+ const parsed = schema.safeParse(op.payload);
444
+ if (!parsed.success) {
445
+ throw new MutationError("outside_shape", parsed.error.message);
446
+ }
447
+ await db.insert(todos).values({ id: op.rowId, ...parsed.data })
448
+ .onConflictDoUpdate({ target: todos.id, set: parsed.data });
449
+ },
450
+ });
451
+ ```
452
+
453
+ Two details that matter:
454
+
455
+ - **Throw `MutationError`, not a plain `Error`.** Only `MutationError` carries an
456
+ `ErrorReason` through to the client's `onError`; anything else is reported as
457
+ `server_error`.
458
+ - **Write the parsed value, not `op.payload`.** Writing the raw payload after
459
+ validating it defeats `.strict()` and any coercion the schema applied — and it
460
+ also feeds unvalidated data into reflectdb's mirror, which is what conflict
461
+ resolution compares against.
462
+
463
+ The same applies to `authorize`, and to writes arriving through `server.rest()` —
464
+ both run the identical pipeline.
465
+
466
+ ### Ephemeral messages (cursors, presence, typing)
467
+
468
+ Ephemeral events are room-scoped broadcasts that bypass the op log — ideal for high-frequency signals:
469
+
470
+ ```tsx
471
+ import { useEphemeral } from "reflectdb/react";
472
+
473
+ const { events, broadcast } = useEphemeral({
474
+ key: "cursor",
475
+ userId: currentUserId,
476
+ ttlMs: 10_000,
477
+ });
478
+
479
+ // on mouse move
480
+ broadcast({ x: e.clientX, y: e.clientY });
481
+
482
+ // render peers
483
+ Object.values(events).map((c) => <Cursor x={c.x} y={c.y} />);
484
+ ```
485
+
486
+ The whiteboard renders peer cursors this way — see [`examples/whiteboard/app.tsx`](./examples/whiteboard/app.tsx).
487
+
488
+ ### Per-user query results
489
+
490
+ A `query` callback is just a function — it can return different rows depending on the caller's `auth`. reflectdb re-runs it whenever the listed `tables` change, so each subscriber gets a personalized view that stays live.
491
+
492
+ The whiteboard uses this to keep the round's secret word out of the wire for everyone except the active drawer:
493
+
494
+ ```ts
495
+ const queries = defineSyncQueries({
496
+ roundWord: {
497
+ row: t<{ id: string; gameId: string; word: string }>(),
498
+ params: t<{ gameId: string }>(),
499
+ tables: ["games", "game_secrets"], // re-run on these
500
+ },
501
+ });
502
+
503
+ server.implement("roundWord", {
504
+ query: async (ctx, db) => {
505
+ const game = await db.select().from(games).where(eq(games.id, ctx.params.gameId)).get();
506
+ if (!game || game.state !== "drawing") return [];
507
+ if (game.currentDrawerId !== ctx.auth.userId) return []; // guessers see []
508
+ const secret = await db.select().from(gameSecrets)
509
+ .where(eq(gameSecrets.gameId, ctx.params.gameId)).get();
510
+ return secret?.word ? [{ id: ctx.params.gameId, gameId: ctx.params.gameId, word: secret.word }] : [];
511
+ },
512
+ mutate: async () => { throw new Error("read-only"); },
513
+ tables: ["games", "game_secrets"],
514
+ });
515
+ ```
516
+
517
+ The `game_secrets` table isn't registered in `defineSyncQueries`, so it's never broadcast directly. Calling `server.notifyChange("game_secrets")` from the engine fans out the recomputed `roundWord` result to whichever client is now the drawer.
518
+
519
+ ### Server-driven game loops
520
+
521
+ Some apps need state that advances on a clock, not on user input — round timers, expiring claims, scheduled rotations. Pair a `setInterval` with `notifyChange` and the server stays the single source of truth.
522
+
523
+ ```ts
524
+ setInterval(async () => {
525
+ const now = Date.now();
526
+ const active = await db.select().from(games).where(eq(games.mode, "pictionary"));
527
+ for (const g of active) {
528
+ if (g.state === "drawing" && now >= g.roundEndsAt) {
529
+ await endRound(g.id); // raw SQL writes
530
+ await server.notifyChange("games"); // fan-out to subscribers
531
+ }
532
+ }
533
+ }, 500);
534
+ ```
535
+
536
+ Pair with a per-row mutex if a tick can outrun its interval. The whiteboard example does both — see [`examples/whiteboard/server.tsx`](./examples/whiteboard/server.tsx).
537
+
538
+ ### Windowed sync and pagination
539
+
540
+ For large tables, sync a sliding window instead of the whole set:
541
+
542
+ ```ts
543
+ const queries = defineSyncQueries({
544
+ messages: {
545
+ row: t<Message>(),
546
+ conflict: "lww",
547
+ countHints: true, // emit count_changed deltas
548
+ },
549
+ });
550
+
551
+ // React
552
+ const { rows } = useSync("messages", { window: 50 });
553
+ const total = useTotalCount("messages");
554
+ const loadMore = useLoadMore("messages");
555
+
556
+ // show "Load 50 more" when rows.length < total
557
+ ```
558
+
559
+ Make the window a **real** limit by reading `ctx.limit` in the query and supplying a `count`:
560
+
561
+ ```ts
562
+ server.implement("messages", {
563
+ query: ({ params, limit }, db) =>
564
+ db.select().from(messages)
565
+ .where(eq(messages.roomId, params.roomId))
566
+ .orderBy(desc(messages.createdAt))
567
+ .limit(limit ?? 1000),
568
+ count: async ({ params }, db) =>
569
+ (await db.select({ n: count() }).from(messages)
570
+ .where(eq(messages.roomId, params.roomId)))[0].n,
571
+ });
572
+ ```
573
+
574
+ Without them, the server fetches every matching row on every broadcast and slices in JS — pagination reduces bytes on the wire and nothing else. `ctx.limit` is `undefined` when the caller genuinely needs the full set, so a plain `limit ?? <max>` is safe.
575
+
576
+ A window is an **entitlement**, not a row count: a subscriber with `window: 50` whose query matched 3 rows still receives the next 47 inserts, and `loadMore(20)` widens the entitlement by 20 regardless of how many rows actually arrived. Reconnecting restores the widened window, not the initial one.
577
+
578
+ ### Auto-generated REST API
579
+
580
+ `server.rest()` returns a fetch-style handler that responds to CRUD URLs derived from your schema:
581
+
582
+ ```ts
583
+ const rest = server.rest({ prefix: "/api" });
584
+
585
+ serve({
586
+ port: 3001,
587
+ async fetch(req, srv) {
588
+ const url = new URL(req.url);
589
+ if (url.pathname.startsWith("/api/")) return rest(req);
590
+ // ...WebSocket upgrade, etc.
591
+ return new Response("ok");
592
+ },
593
+ });
594
+ ```
595
+
596
+ Endpoints generated for every `implement()`'d table:
597
+
598
+ ```
599
+ GET /api/<table> → list (supports ?where=…&limit=&offset=)
600
+ GET /api/<table>/:id → single row
601
+ POST /api/<table> → insert (body = row, or array = batch)
602
+ PATCH /api/<table>/:id → update
603
+ DELETE /api/<table>/:id → delete
604
+ ```
605
+
606
+ REST writes go through the same pipeline as sync writes and broadcast deltas to connected clients.
607
+
608
+ ### High availability with Postgres
609
+
610
+ Share a Postgres op log between server instances. Clients reconnecting to a different instance resume seamlessly from their HLC watermark.
611
+
612
+ ```ts
613
+ import pg from "pg";
614
+ import { createPostgresStorage } from "reflectdb/server";
615
+
616
+ const pool = new pg.Pool({ connectionString: process.env.DATABASE_URL });
617
+
618
+ const server = createSyncServer({
619
+ queries, db, transport,
620
+ storage: createPostgresStorage(pool),
621
+ serverId: process.env.FLY_ALLOC_ID,
622
+ poll: 500, // 500ms cross-instance poll for active-active
623
+ });
624
+ ```
625
+
626
+ | Mode | Config | Use case |
627
+ |------|--------|----------|
628
+ | **Failover only** | Shared Postgres, no `poll` | Clients resume on reconnect |
629
+ | **Active-active** | Shared Postgres + `poll: 500` | Real-time cross-instance updates |
630
+
631
+ Each poll tick first probes the shared op log's head HLC; an idle tick costs one `MAX(hlc)` query and broadcasts nothing. Only tables that actually changed are re-broadcast. The tick also re-merges the shared clock watermark, so an instance whose wall clock lags its peers stops stamping writes below HLCs clients have already seen.
632
+
633
+ ## Whiteboard + Pictionary example
634
+
635
+ A complete React + Bun + Drizzle app that exercises most of reflectdb in one place: [`examples/whiteboard/`](./examples/whiteboard/).
636
+
637
+ ```bash
638
+ cd examples/whiteboard
639
+ bun install
640
+ bun dev
641
+ # open http://localhost:3003 in two tabs
642
+ ```
643
+
644
+ Two modes:
645
+
646
+ - **Freeform draw** — every player can draw on a shared canvas. Strokes are LWW per row.
647
+ - **Pictionary** — players take turns drawing while the others guess in chat. The server picks a word, runs a per-round timer, awards points based on remaining time, advances the drawer, and ends the game after N full rotations.
648
+
649
+ What it demonstrates:
650
+
651
+ | Pattern | Where |
652
+ |---------|-------|
653
+ | Drizzle-typed schema, SQLite op log | [`schema.ts`](./examples/whiteboard/schema.ts) |
654
+ | WebSocket transport on Bun | [`server.tsx`](./examples/whiteboard/server.tsx) |
655
+ | Authentication via better-auth (email/password + anonymous) | [`auth.ts`](./examples/whiteboard/auth.ts) |
656
+ | `params`-scoped queries (`strokes`, `messages` per game) | [`server.tsx`](./examples/whiteboard/server.tsx) |
657
+ | Per-user query results — only the drawer receives the secret word | `roundWord` in [`server.tsx`](./examples/whiteboard/server.tsx) |
658
+ | Server-side game loop with a mutex + `notifyChange` | `tick`, `withLock` in [`server.tsx`](./examples/whiteboard/server.tsx) |
659
+ | Server-side guess detection (text replacement so the answer never broadcasts) | `mutateMessageWithGuesses` in [`server.tsx`](./examples/whiteboard/server.tsx) |
660
+ | `readonly` field enforcement to keep the engine state out of client hands | [`schema.ts`](./examples/whiteboard/schema.ts) |
661
+ | Ephemeral cursors per game, scoped via `key: \`cursor:${gameId}\`` | [`app.tsx`](./examples/whiteboard/app.tsx) |
662
+ | Per-table rate limiting (loose for strokes, tight for chat) | `server.rateLimit` in [`server.tsx`](./examples/whiteboard/server.tsx) |
663
+
664
+ ## Architecture
665
+
666
+ ```
667
+ ┌─────────────────────────────────────────────────────────────────────────────┐
668
+ │ SHARED CORE (core/) │
669
+ │ │
670
+ │ defineSyncQueries({ ... }) ── one schema, shared by every layer │
671
+ │ │
672
+ │ ┌──────────────┐ ┌──────────────┐ ┌──────────────────────────────────┐ │
673
+ │ │ types.ts │ │ hlc.ts │ │ schema.ts │ │
674
+ │ │ • SyncOp │ │ • HLC │ │ • SyncQueryDef │ │
675
+ │ │ • Messages │ │ • send/recv │ │ • InferRow / InferParams │ │
676
+ │ │ • ErrorReason│ │ • pack/cmp │ │ • t<T>() phantom helper │ │
677
+ │ │ • Protocol │ │ │ │ • ConflictPolicy │ │
678
+ │ └──────────────┘ └──────────────┘ └──────────────────────────────────┘ │
679
+ └───────────────────────────────┬─────────────────────────────────────────────┘
680
+ ┌───────────────┴────────────────┐
681
+ ▼ ▼
682
+ ┌───────────────────────────────────┐ ┌──────────────────────────────────────┐
683
+ │ SERVER (server/) │ │ CLIENT (client/) │
684
+ │ │ │ │
685
+ │ createSyncServer<TQueries>() │ │ createSyncClient<TQueries>() │
686
+ │ ├─ .implement(name, opts) │ │ ├─ .sync(name, params?) │
687
+ │ ├─ .auth(token → AuthContext) │ │ ├─ .insert/.update/.delete │
688
+ │ ├─ .room(pattern, cb) │ │ ├─ .subscribe / .subscribeTable │
689
+ │ ├─ .rateLimit({...}) │ │ ├─ .getRows / .getRow / .getState │
690
+ │ ├─ .compaction({...}) │ │ ├─ .loadMore / .getTotalCount │
691
+ │ ├─ .rest({ prefix }) │ │ └─ .sendEphemeral / .subscribeEph. │
692
+ │ ├─ .notifyChange(table) │ │ │
693
+ │ └─ .close() │ │ Internal: │
694
+ │ │ │ • SyncClient (state machine) │
695
+ │ Pipeline (per op): │ │ • ClientStore (row cache + queue) │
696
+ │ 1. clock drift check │ │ • OpCreator (HLC stamping) │
697
+ │ 2. rate limit (fail-open) │ │ │
698
+ │ 3. batch-size check │ │ State machine: │
699
+ │ 4. readonly enforcement │ │ hydrating → disconnected → … │
700
+ │ 5. serverSet injection │ │ connecting → bootstrapping → synced│
701
+ │ 6. conflict resolution* │ │ │
702
+ │ │ │ Storage adapters: │
703
+ │ (* skipped by eager modes) │ │ • memory (ephemeral) │
704
+ │ │ │ • indexeddb (persistent) │
705
+ │ BroadcastEngine (per write): │ │ │
706
+ │ group subscribers → run query │ │ │
707
+ │ once per group → diff per │ │ │
708
+ │ client → send → commit cache │ │ │
709
+ │ │ │ │
710
+ │ Op log storage (optional): │ │ │
711
+ │ • in-memory (default) │ │ │
712
+ │ • sqlite (bun:sqlite) │ │ │
713
+ │ • postgres (any pg-compatible) │ │ │
714
+ └───────────────────────────────────┘ └──────────────────────────────────────┘
715
+
716
+ ┌─────────────────────────────────────────────────────────────────────────────┐
717
+ │ TRANSPORT LAYER (transport/) │
718
+ │ │
719
+ │ ┌──────────────────┐ ┌──────────────────┐ ┌──────────────────────────┐ │
720
+ │ │ WebSocket │ │ SSE │ │ Polling │ │
721
+ │ │ real-time │ │ event-stream + │ │ 3 HTTP endpoints — │ │
722
+ │ │ bi-directional │ │ POST back-chan │ │ works anywhere HTTP does│ │
723
+ │ └──────────────────┘ └──────────────────┘ └──────────────────────────┘ │
724
+ └─────────────────────────────────────────────────────────────────────────────┘
725
+
726
+ ┌─────────────────────────────────────────────────────────────────────────────┐
727
+ │ FRAMEWORK BINDINGS (react/, svelte/, vanilla/) │
728
+ │ │
729
+ │ • createSyncReact(queries) → typed hooks + <SyncProvider> │
730
+ │ • createSyncSvelte(queries) → typed Svelte stores │
731
+ │ • createSyncVanilla(queries) → typed callback API │
732
+ └─────────────────────────────────────────────────────────────────────────────┘
733
+ ```
734
+
735
+ ## Core Concepts
736
+
737
+ ### Hybrid Logical Clocks
738
+
739
+ reflectdb uses [hybrid logical clocks](https://muratbuffalo.blogspot.com/2014/07/hybrid-logical-clocks.html) (HLCs) to order events across machines without requiring synchronized clocks.
740
+
741
+ An HLC has three parts:
742
+
743
+ | Component | Purpose |
744
+ |-----------|---------|
745
+ | `ms` | Physical wall time |
746
+ | `counter` | Logical counter (breaks ties) |
747
+ | `nodeId` | Machine that generated it |
748
+
749
+ HLCs pack to zero-padded strings (`0000001711234567890.0003.client-abc`), so **string comparison gives correct causal ordering** — no parsing needed. Conflict resolution is essentially free.
750
+
751
+ Two operations define the clock:
752
+
753
+ - **send** (`sendHlc`): advance `max(wall, lastMs)`; increment counter on tie, else reset.
754
+ - **receive** (`receiveHlc`): advance past both local and remote state. Remote timestamps are clamped to `wall + MAX_CLOCK_DRIFT_MS` (default 5 min) so a runaway client can't push the clock into the future.
755
+
756
+ The clock **ratchets forward** through every exchange, so causal ordering is preserved across the network.
757
+
758
+ ### Conflict Resolution
759
+
760
+ Four built-in policies, chosen per-query:
761
+
762
+ ```ts
763
+ defineSyncQueries({
764
+ posts: { row: t<Post>(), conflict: "lww" },
765
+ docs: { row: t<Doc>(), conflict: "merge" },
766
+ config: { row: t<Config>(), conflict: "server" },
767
+ scores: { row: t<Score>(), conflict: { policy: "custom", resolve: fn } },
768
+ });
769
+ ```
770
+
771
+ | Policy | Granularity | Concurrent edits to different fields | Use case |
772
+ |---------|-------------|--------------------------------------|----------|
773
+ | `lww` | Row | One wins, the other is lost | Simple data, rare conflicts |
774
+ | `merge` | Column | Both preserved | Collaborative editing |
775
+ | `server`| Row | Only first write; all others rejected | Config, reference data |
776
+ | custom | You choose | Your logic | Counters, "highest bid wins", business rules |
777
+
778
+ A custom resolver receives the incoming op, the existing row + per-column clocks, and metadata, and returns the resolved row. Throw to reject.
779
+
780
+ `merge` is a **server-side** guarantee: the server resolves per column using the client op HLCs held in its mirror, so two clients editing different fields both land. The per-column clocks a client sees on a broadcast are a different domain — a diff-driven broadcast can't attribute a column to the op that produced it, so every column changed in one broadcast carries that broadcast's HLC. Client-side merge orders *broadcasts* against each other and against local optimistic state; it does not reconstruct per-column causality between clients.
781
+
782
+ Both eager broadcast modes skip conflict resolution entirely — writes land last-writer-wins regardless of the declared policy.
783
+
784
+ ### The Sync Protocol
785
+
786
+ ```
787
+ 1. client ──▶ hello server ──▶ hello_ack (protocol, serverId)
788
+ 2. client ──▶ sync_declare server ──▶ snapshot / bootstrap_complete
789
+ 3. client ──▶ ops (optimistic) server runs pipeline
790
+ 4. server ──▶ ack / reject
791
+ 5. server ──▶ delta (broadcast to subscribers)
792
+ 6. client reconnects ──▶ resume (watermark HLC) server ──▶ missed deltas
793
+ ```
794
+
795
+ All messages are JSON; the transport is just a pipe. WebSocket gives bi-directional real-time; SSE gives server-push with POST for upstream; polling is stateless HTTP for constrained environments.
796
+
797
+ ### Two stores, one sync
798
+
799
+ reflectdb keeps its own store alongside yours, and it helps to know which one answers what:
800
+
801
+ | Read | Source |
802
+ |------|--------|
803
+ | Snapshots (`bootstrap`, `resume`) | **Your database**, via the `query` callback |
804
+ | Broadcast deltas | **Your database**, diffed against a per-client cached result set |
805
+ | Conflict resolution (`lww` / `merge` / `server` / custom) | **reflectdb's mirror** — a JSONB row store plus per-column HLCs |
806
+ | Which tables changed since an HLC | **reflectdb's op log** |
807
+
808
+ A write therefore lands in two places: your `mutate` callback commits to your database, and reflectdb commits the mirror row plus its op-log entry. Those are **separate commits** — "atomic" in this codebase means the mirror row and its op-log entry commit together, not that they commit with your write. Consequences worth designing around:
809
+
810
+ - A crash between the two leaves your database ahead of the mirror. Clients still converge (snapshots come from your database), but conflict resolution decides against slightly stale state until the next write.
811
+ - If `mutate` transforms the payload, or database defaults/triggers rewrite it, or something writes the table out of band, the mirror drifts from what clients actually see. Keep `mutate` a faithful application of `op.payload` when conflict policy is load-bearing, and route out-of-band writes through `server.applyServerOp` / `server.emit` / `server.tx`.
812
+
813
+ ### The Op Log and Resume
814
+
815
+ Every accepted mutation is appended to the server's op log with its HLC. On reconnect, the client sends its last seen HLC as a watermark, and the server replays only what the client missed. This makes cross-server failover automatic when the log is shared (Postgres).
816
+
817
+ Old ops are compacted on a schedule based on client inactivity and minimum op age. Reconnecting clients whose watermark has been compacted receive a fresh bootstrap.
818
+
819
+ ## API Reference
820
+
821
+ ### `reflectdb/core`
822
+
823
+ ```ts
824
+ import {
825
+ defineSyncQueries, t,
826
+ createHlc, sendHlc, receiveHlc, packHlc, unpackHlc, compareHlc,
827
+ MutationError, TransportSendError, isErrorReason, reasonFromError,
828
+ PROTOCOL_VERSION, MAX_CLOCK_DRIFT_MS, MAX_BATCH_SIZE,
829
+ TOMBSTONE_RETENTION_MS, SERVER_TOMBSTONE_RETENTION_MS,
830
+ } from "reflectdb/core";
831
+ ```
832
+
833
+ | Export | Description |
834
+ |--------|-------------|
835
+ | `defineSyncQueries(map)` | Identity function that pins your schema's literal types. Feed its result to both server and client. |
836
+ | `t<T>()` | Phantom helper to declare a row or params type. Returns `undefined as T`. |
837
+ | `createHlc(nodeId)` / `sendHlc` / `receiveHlc` | HLC constructors and transitions. |
838
+ | `packHlc` / `unpackHlc` / `compareHlc` | Serialize, deserialize, compare HLC values. |
839
+ | `MutationError(reason, message?)` | Throw from `mutate`/`authorize` to reject a write with a specific `ErrorReason`. |
840
+ | `TransportSendError(clientId, message)` | Throw from a custom `ServerTransport.send` when a frame did not reach the peer. |
841
+ | `isErrorReason(v)` / `reasonFromError(e)` | Validate / extract an `ErrorReason`. |
842
+
843
+ Types: `HLC`, `SyncOp`, `OpType`, `OpStatus`, `ClientMessage`, `ServerMessage`, `ErrorReason`, `ConflictPolicy`, `ConflictResolver`, `SyncQueryDef`, `InferRow`, `InferParams`, `InferWritableRow`, `RequiresParams`, `RateLimitConfig`, `CompactionConfig`, `ShapeConfig`, `AuthContext`, `DrizzleTableLike`.
844
+
845
+ ### `reflectdb/server`
846
+
847
+ ```ts
848
+ import {
849
+ createServer, createSyncServer,
850
+ createSqliteStorage, createPostgresStorage,
851
+ resolveConflict, processOp,
852
+ enforceClockDrift, enforceReadonly, enforceServerSet, enforceBatchSize, createRateLimiter,
853
+ MutationError,
854
+ } from "reflectdb/server";
855
+ ```
856
+
857
+ **`createSyncServer<TQueries, TDb, TAuth>(config)`** — the typed entry point. Returns a server with:
858
+
859
+ | Method | Purpose |
860
+ |--------|---------|
861
+ | `.implement(name, options)` | Register a query handler (required for every query in the schema). |
862
+ | `.auth(callback)` | Validate the connection request and return an `AuthContext`. |
863
+ | `.room(pattern, callback)` | Scope clients to a subset of data, matched against URL-style patterns (`org/:orgId`). |
864
+ | `.rateLimit(config)` | Set per-user/per-table limits. Fail-open on limiter errors. |
865
+ | `.compaction(config)` | Configure op-log compaction. |
866
+ | `.rest({ prefix })` | Generate a CRUD fetch handler. |
867
+ | `.minSchemaVersion(n)` | Reject clients on older schema versions. |
868
+ | `.notifyChange(table)` | Manually trigger a broadcast (for external writes). |
869
+ | `.runCompaction()` | Manually run one compaction pass. |
870
+ | `.close()` | Shut down, disconnect clients, close storage. |
871
+
872
+ `createServer()` is the lower-level untyped variant — use it only if you need to register queries dynamically or don't have a `defineSyncQueries` map.
873
+
874
+ ### `reflectdb/client`
875
+
876
+ ```ts
877
+ import {
878
+ createSyncClient,
879
+ SyncClient,
880
+ ClientStore,
881
+ createMemoryStorage,
882
+ createOpCreator,
883
+ } from "reflectdb/client";
884
+
885
+ import { createIndexedDBStorage } from "reflectdb/client/storage/indexeddb";
886
+ ```
887
+
888
+ **`createSyncClient<TQueries>(config)`** — fully typed client. Methods:
889
+
890
+ | Category | Methods |
891
+ |----------|---------|
892
+ | Lifecycle | `init()`, `connect()`, `bootstrap()`, `resume()`, `push()`, `close()` |
893
+ | Subscriptions | `sync(name, params?)`, `unsync(name)` |
894
+ | Mutations | `insert(name, id, payload)`, `update(name, id, patch)`, `delete(name, id)`, `batch(ops)` |
895
+ | Reads | `getRows(name)`, `getRow(name, id)`, `getState()`, `getVersion()`, `getPendingCount()` |
896
+ | Observation | `subscribe(listener)`, `subscribeTable(name, listener)` (returns unsubscribe fn) |
897
+ | Windowing | `loadMore(name, count)`, `getTotalCount(name)` |
898
+ | Ephemeral | `sendEphemeral({ key, userId, data, ttlMs? })`, `subscribeEphemeral(key, listener)` |
899
+
900
+ State machine: `hydrating → disconnected → connecting → bootstrapping → synced`. Reconnects with exponential backoff (capped by `maxReconnectDelayMs`, default 30s).
901
+
902
+ ### `reflectdb/react`
903
+
904
+ ```ts
905
+ import {
906
+ SyncProvider, useSyncClient,
907
+ useSync, useSyncStatus, useRow,
908
+ usePendingCount, useEphemeral,
909
+ useTotalCount, useLoadMore,
910
+ createSyncReact,
911
+ } from "reflectdb/react";
912
+ ```
913
+
914
+ **`<SyncProvider>` props:**
915
+
916
+ | Prop | Type | Description |
917
+ |------|------|-------------|
918
+ | `url` | `string` | WebSocket URL (required) |
919
+ | `token` | `string` | Auth token passed to `server.auth()` (required) |
920
+ | `tables` | `string[]` | Tables to auto-sync on mount |
921
+ | `clientId` | `string` | Stable ID for this client (generated if omitted) |
922
+ | `storage` | `ClientStorageAdapter` | Defaults to memory |
923
+ | `onReauth` | `() => Promise<string>` | Called when server revokes auth |
924
+ | `onError` | `(e) => void` | Connection / sync error callback |
925
+
926
+ **Hooks:**
927
+
928
+ | Hook | Returns |
929
+ |------|---------|
930
+ | `useSync(table, options?)` | `{ rows, insert, update, remove, loading }` — options: `{ params?, includeDeleted?, window? }` |
931
+ | `useSyncStatus()` | `"hydrating" \| "disconnected" \| "connecting" \| "bootstrapping" \| "synced"` |
932
+ | `useRow(table, id)` | Single row or `null` |
933
+ | `usePendingCount()` | Total unsynced op count |
934
+ | `useEphemeral({ key, userId, ttlMs? })` | `{ events, broadcast }` |
935
+ | `useTotalCount(table)` | Server-side count (requires `countHints: true`) |
936
+ | `useLoadMore(table)` | Function to expand the sync window |
937
+
938
+ **`createSyncReact<TQueries>()`** returns the same hook set with row and param types inferred from your schema.
939
+
940
+ ### `reflectdb/svelte`
941
+
942
+ ```ts
943
+ import { createSyncStore, createSyncSvelte, createBrowserWsTransport } from "reflectdb/svelte";
944
+ ```
945
+
946
+ `createSyncStore(config)` returns a `SyncStore`:
947
+
948
+ ```ts
949
+ const store = createSyncStore({ url, token, tables: ["notes"] });
950
+
951
+ const { rows, insert, update, remove } = store.sync<Note>("notes");
952
+ // rows is a Readable<Note[]> — subscribe with Svelte's $rows
953
+
954
+ store.status // Readable<SyncClientState>
955
+ store.pendingCount // Readable<number>
956
+
957
+ store.connect();
958
+ store.onStateChange((s) => …);
959
+ store.onError((e) => …);
960
+ ```
961
+
962
+ `createSyncSvelte(queries)` returns fully-typed store factories.
963
+
964
+ ### `reflectdb/vanilla`
965
+
966
+ ```ts
967
+ import { createSync, createSyncVanilla, createBrowserWsTransport } from "reflectdb/vanilla";
968
+
969
+ const sync = createSync({ url, token, tables: ["notes"] });
970
+ const notes = sync.sync<Note>("notes");
971
+
972
+ notes.onChange((rows) => render(rows));
973
+ notes.insert(id, { title: "…" });
974
+
975
+ sync.onStateChange((s) => …);
976
+ sync.onPendingChange((n) => …);
977
+ sync.onError((e) => …);
978
+ sync.connect();
979
+ ```
980
+
981
+ Also supports ephemeral: `sync.sendEphemeral({ key, userId, data })`, `sync.onEphemeral(key, listener)`.
982
+
983
+ ### `reflectdb/transport/*`
984
+
985
+ ```ts
986
+ import { createWsServerTransport, isOriginAllowed } from "reflectdb/transport/ws";
987
+ import { createBunWsServerTransport } from "reflectdb/transport/bun-ws";
988
+ import { createSseServerTransport } from "reflectdb/transport/sse";
989
+ import { createPollingServerTransport, pollingBodyTooLarge } from "reflectdb/transport/polling";
990
+ ```
991
+
992
+ Each server transport returns a `ServerTransport` object plus framework-agnostic handlers (`handleOpen`, `handleMessage`, `handleClose`, `handlePong` for WS; `handleSubscribe`, `handleMessage`, `handleDisconnect`, `createEventStream` for SSE; `handleConnect`, `handlePoll`, `handleSend`, `handleDisconnect` for polling). Wire them to your HTTP server's routes — reflectdb does not ship a specific HTTP server.
993
+
994
+ `reflectdb/transport/bun-ws` is the same WebSocket transport shaped for `Bun.serve`: `createBunWsServerTransport()` returns `{ transport, websocket }`, where `websocket` is the handlers object you pass straight to `Bun.serve({ websocket })`. Use it only under Bun; `reflectdb/transport/ws` is the runtime-agnostic one.
995
+
996
+ Client-side, use the `createBrowserWsTransport(url)` helper exported from `reflectdb/svelte` or `reflectdb/vanilla`, or let `<SyncProvider>` create one internally.
997
+
998
+ ## Configuration Reference
999
+
1000
+ ### Query definition
1001
+
1002
+ Every entry in `defineSyncQueries({ ... })`:
1003
+
1004
+ ```ts
1005
+ {
1006
+ // declare the row type — choose ONE:
1007
+ row: t<MyRow>(), // plain type (recommended default)
1008
+ // OR
1009
+ table: someDrizzleTable, // auto-infers row type + table list + pk
1010
+
1011
+ // optional:
1012
+ params: t<{ orgId: string }>(), // typed query params (required on sync() if declared)
1013
+ tables: ["posts", "post_tags"], // change-detection tables; defaults to the query key
1014
+ pk: "id", // primary-key column name (default "id")
1015
+ conflict: "lww", // "lww" | "merge" | "server" | { policy: "custom", resolve }
1016
+ readonly: ["createdBy"], // fields the client cannot write
1017
+ serverSet: ["createdAt", "updatedAt"], // fields the server always sets — required in `implement.serverSet`
1018
+ countHints: true, // emit count_changed deltas for windowed sync
1019
+ }
1020
+ ```
1021
+
1022
+ `conflict` resolves the incoming op against **reflectdb's mirror** (its own JSONB row store and per-column clocks), not against your database. The two agree as long as every write goes through reflectdb and `mutate` persists the resolved payload verbatim — see [Two stores, one sync](#two-stores-one-sync).
1023
+
1024
+ ### Server configuration
1025
+
1026
+ ```ts
1027
+ createSyncServer({
1028
+ queries, // from defineSyncQueries()
1029
+ db, // optional — anything you want handed to query/mutate callbacks
1030
+ transport, // ServerTransport (required)
1031
+ storage, // StorageAdapter (optional; defaults to in-memory op log)
1032
+ serverId: "server-1", // unique within the deployment
1033
+ poll: 500, // ms — enable HA active-active polling
1034
+ maxConnectionsPerUser: 10, // backpressure guard
1035
+ queryTimeoutMs: 5_000, // abort a broadcast query that hangs (0 = disabled)
1036
+ maxBroadcastConcurrency: 8, // subscriber groups queried in parallel per broadcast
1037
+ allowAnonymous: false, // serve connections with no auth() callback
1038
+ onEvent: (event) => { … }, // lifecycle telemetry
1039
+ });
1040
+ ```
1041
+
1042
+ A server with no `auth()` callback rejects the handshake — every message after `hello` would fail the authentication gate anyway. Pass `allowAnonymous: true` to serve unauthenticated clients; each session gets an `anon:<clientId>` identity.
1043
+
1044
+ ### `implement()` options
1045
+
1046
+ ```ts
1047
+ server.implement("todos", {
1048
+ query: (ctx, db) => /* fetch rows — return anything iterable */,
1049
+ mutate: async (op, ctx, db) => { /* apply op.type/op.rowId/op.payload */ },
1050
+ authorize: async (action, ctx, db) => {
1051
+ // action.type is "read" | "write"
1052
+ // throw to deny
1053
+ },
1054
+ serverSet: { createdAt: () => new Date() }, // required if schema declares serverSet fields
1055
+ broadcast: "consistent", // "consistent" | "eager" | "eager-durable"
1056
+ flushInterval: 50, // ms, for eager broadcasting
1057
+ maxBufferSize: 100, // ops per eager batch
1058
+ tables: ["todos"], // override change-detection set
1059
+ count: (ctx, db) => db.count(…), // total row count for windowed queries
1060
+ groupBy: ({ auth }) => String(auth.orgId), // collapse subscribers into one query execution
1061
+ room: "org/:orgId", // require this room pattern on every subscription
1062
+ });
1063
+ ```
1064
+
1065
+ `broadcast` modes:
1066
+
1067
+ - **`consistent`** (default): run the conflict pipeline, persist, then broadcast by diffing each subscriber's re-executed query result.
1068
+ - **`eager-durable`**: skip conflict resolution; run `mutate`, persist to reflectdb's mirror atomically, then broadcast the delta directly. The recommended low-latency mode.
1069
+ - **`eager`**: same, but the mirror write is batched in the background — a crash can lose it. Only safe when `mutate` is durable to your own database.
1070
+
1071
+ Both eager modes still enforce `readonly`, `serverSet`, clock drift, the batch cap and rate limits. What they skip is **conflict resolution**: a declared `conflict` policy does not apply and writes land last-writer-wins.
1072
+
1073
+ #### Scaling broadcasts with `groupBy`
1074
+
1075
+ A write re-executes each dependent query once per subscriber *group*. Groups default to `(auth, params, roomKey)`, so with per-user auth they collapse to roughly one per connected client — N clients means N query executions per write.
1076
+
1077
+ `groupBy` returns the coarser key a query actually depends on:
1078
+
1079
+ ```ts
1080
+ server.implement("posts", {
1081
+ query: ({ auth }, db) => db.select().from(posts).where(eq(posts.orgId, auth.orgId)),
1082
+ // Results depend only on orgId — every member of an org shares one execution.
1083
+ groupBy: ({ auth }) => String(auth.orgId),
1084
+ });
1085
+ ```
1086
+
1087
+ Two clients sharing a key **must** be entitled to byte-identical rows. Collapsing clients that aren't leaks rows across the boundary.
1088
+
1089
+ ### Rate limiting
1090
+
1091
+ ```ts
1092
+ server.rateLimit({
1093
+ opsPerSecond: 20,
1094
+ opsPerMinute: 600,
1095
+ batchesPerMinute: 120,
1096
+ perTable: {
1097
+ todos: { opsPerSecond: 10 },
1098
+ },
1099
+ ephemeralPerSecond: 60, // presence/cursor ceiling per client (default 60; 0 disables)
1100
+ });
1101
+ ```
1102
+
1103
+ The limiter is **fail-open**: if the limiter itself errors, ops still flow. Clients that exceed their limit receive `ErrorReason: "rate_limited"`.
1104
+
1105
+ Ephemeral messages are metered separately and **always** — even without a `rateLimit()` call — because each one fans out to every room subscriber, which makes an unmetered channel an amplification vector. Dropped messages surface as an `ephemeral_rate_limited` event on `onEvent`.
1106
+
1107
+ ### Compaction
1108
+
1109
+ ```ts
1110
+ server.compaction({
1111
+ clientInactivityTimeout: "24h", // clients idle longer are ignored
1112
+ interval: "1h", // compaction interval
1113
+ minOpAge: "5m", // don't compact ops younger than this
1114
+ });
1115
+ ```
1116
+
1117
+ Durations accept `ms`, `s`, `m`, `h`, `d`.
1118
+
1119
+ ### Client configuration
1120
+
1121
+ ```ts
1122
+ createSyncClient({
1123
+ queries,
1124
+ clientId: "browser-xyz", // required; keep stable across reloads
1125
+ transport: createBrowserWsTransport("ws://…"),
1126
+ token: "auth-token", // required
1127
+ storage: createIndexedDBStorage({ dbName: "app" }),
1128
+ autoSync: true, // auto-sync param-less queries on connect
1129
+ maxReconnectDelayMs: 30_000,
1130
+ hydrateAllTables: false, // true = read every stored row at boot
1131
+ onSync: () => { … }, // fires once bootstrap completes
1132
+ onError: (e) => { … },
1133
+ onReauth: async () => newToken, // called after "auth_revoked"
1134
+ });
1135
+ ```
1136
+
1137
+ At boot the client restores its persisted subscriptions first and hydrates only those tables — rows only ever reach local storage through a subscription, so nothing reachable is skipped. Set `hydrateAllTables: true` if you read rows for a table before calling `sync()` on it.
1138
+
1139
+ ### Storage adapters
1140
+
1141
+ #### Server op log
1142
+
1143
+ | Adapter | Import | Best for |
1144
+ |---------|--------|----------|
1145
+ | _(none)_ | omit `storage` | In-memory op log; ephemeral, single node |
1146
+ | `createSqliteStorage({ path?, db? })` | `reflectdb/server` | Single server, development, embedded — **Bun only** |
1147
+ | `createPostgresStorage(poolOrConfig)` | `reflectdb/server` | Multi-server HA, production |
1148
+
1149
+ `createPostgresStorage` accepts any object with `query(text, values) => { rows }` — `pg.Pool`, `pg.Client`, `@neondatabase/serverless`, etc. Optional config: `{ client, tablePrefix: "_reflectdb" }`.
1150
+
1151
+ `createSqliteStorage` is backed by `bun:sqlite` and is resolved lazily, so importing `reflectdb/server` on Node is fine — only calling `createSqliteStorage` there throws. On Node, use `createPostgresStorage` (or omit `storage` for the in-memory op log).
1152
+
1153
+ #### Client
1154
+
1155
+ | Adapter | Import | Best for |
1156
+ |---------|--------|----------|
1157
+ | `createMemoryStorage()` | `reflectdb/client` | Testing, SSR, short sessions |
1158
+ | `createIndexedDBStorage({ dbName, version?, migrate? })` | `reflectdb/client/storage/indexeddb` | Production browser apps |
1159
+
1160
+ ### Transport configuration
1161
+
1162
+ #### WebSocket
1163
+
1164
+ ```ts
1165
+ createWsServerTransport({
1166
+ maxMessageBytes: 1_000_000, // reject larger frames
1167
+ pingIntervalMs: 30_000, // heartbeat; 0 disables
1168
+ pongTimeoutMs: 60_000, // close half-open connections
1169
+ maxBufferedBytes: 8_000_000, // outbound backpressure ceiling; 0 disables
1170
+ });
1171
+ ```
1172
+
1173
+ Returns a `ServerTransport` plus `handleOpen`, `handleMessage`, `handleClose`, `handlePong`. Wire these to your HTTP server's WebSocket callbacks (see [Quick Start](#quick-start)). Use `isOriginAllowed(req, ["https://app.example"])` in your upgrade handler for CORS.
1174
+
1175
+ #### Server-Sent Events
1176
+
1177
+ ```ts
1178
+ createSseServerTransport({
1179
+ replayBufferSize: 256, // Last-Event-ID replay window
1180
+ });
1181
+ ```
1182
+
1183
+ Two endpoints to wire: `GET /sync/events/:clientId` (SSE stream) and `POST /sync/messages/:clientId` (client → server).
1184
+
1185
+ #### HTTP long-polling
1186
+
1187
+ ```ts
1188
+ createPollingServerTransport({
1189
+ maxQueueLen: 1000,
1190
+ idleTimeoutMs: 60_000,
1191
+ reaperIntervalMs: 30_000,
1192
+ maxMessageBytes: 1_000_000,
1193
+ });
1194
+ ```
1195
+
1196
+ Three endpoints: `POST /sync/connect/:id`, `GET /sync/poll/:id`, `POST /sync/send/:id`. Use the `pollingBodyTooLarge()` helper in your request handler.
1197
+
1198
+ #### Writing your own transport
1199
+
1200
+ `ServerTransport.send` **must reject when the frame did not reach the peer** — unknown or closed socket, full outbound queue, backpressure limit. The broadcast engine treats a resolved `send` as "this delta landed" and only then commits the client's cached result set; a transport that swallows failures makes the server believe a client holds rows it never received, and the divergence persists until reconnect. Throw `TransportSendError` from `reflectdb/core` so callers can distinguish delivery failures from bugs.
1201
+
1202
+ ## Development
1203
+
1204
+ ### Prerequisites
1205
+
1206
+ - [Bun](https://bun.sh) 1.0+
1207
+ - Node.js 22+ and npm — dev tooling, and `bun run verify:node`, which checks the published package works for Node consumers in both ESM and CommonJS
1208
+
1209
+ ### Setup
1210
+
1211
+ ```bash
1212
+ git clone https://github.com/TimMikeladze/reflectdb.git
1213
+ cd reflectdb
1214
+ bun install
1215
+ ```
1216
+
1217
+ ### Scripts
1218
+
1219
+ | Command | Description |
1220
+ |---------|-------------|
1221
+ | `bun test` | Run the full test suite |
1222
+ | `bun test --watch` | Watch mode |
1223
+ | `bun test --coverage` | Coverage report |
1224
+ | `bun run build` | Build with `bunup` (ESM + types) |
1225
+ | `bun run type-check` | TypeScript strict check |
1226
+ | `bun run lint` | Lint with `oxlint` |
1227
+ | `bun run format` | Format with `oxfmt` |
1228
+ | `bun run verify:exports` | Check the `exports` map against `dist/`, and type-check the emitted declarations without ambient Bun/React globals (run after `build`) |
1229
+ | `bun run verify:node` | Install the packed tarball into a throwaway Node project and check every subpath imports, requires, and type-checks there under both export conditions (run after `build`; needs `node` + `npm`) |
1230
+
1231
+ ### Project Structure
1232
+
1233
+ ```
1234
+ src/
1235
+ ├── core/ HLC, types, schema
1236
+ ├── server/ createSyncServer, pipeline, session, handler
1237
+ │ │ broadcast-engine, result-cache, eager-buffer,
1238
+ │ │ compaction-manager, replay-detector, ephemeral-manager
1239
+ │ └── storage/ SQLite + Postgres adapters
1240
+ ├── client/ sync-client, store, ops, typed-client
1241
+ │ └── storage/ memory + IndexedDB adapters
1242
+ ├── transport/ WebSocket (runtime-agnostic + Bun.serve), SSE, polling
1243
+ ├── react/ <SyncProvider>, hooks, typed factory
1244
+ ├── svelte/ createSyncStore, typed factory
1245
+ └── vanilla/ createSync, typed factory
1246
+ ```
1247
+
1248
+ ### Tech stack
1249
+
1250
+ - Runtime: Bun
1251
+ - Language: TypeScript (strict, ESM only)
1252
+ - Build: bunup
1253
+ - Test: `bun:test`
1254
+ - Lint: oxlint
1255
+ - Format: oxfmt
1256
+ - CI: GitHub Actions (Ubuntu + macOS + Windows)
1257
+
1258
+ ## License
1259
+
1260
+ MIT