reflectdb 0.1.3 → 0.2.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -18,7 +18,7 @@ You bring your own types and your own database. reflectdb handles the protocol,
18
18
  └──────────────┘ └──────────────┘ └──────────────┘
19
19
  ▲ │
20
20
  │ offline ▼
21
- └────── IndexedDB ───── op log (in-memory / SQLite / Postgres)
21
+ └────── IndexedDB ───── op log (in-memory / SQLite / Postgres / S3)
22
22
  ```
23
23
 
24
24
  ## Table of Contents
@@ -45,8 +45,13 @@ You bring your own types and your own database. reflectdb handles the protocol,
45
45
  - [Windowed sync and pagination](#windowed-sync-and-pagination)
46
46
  - [Auto-generated REST API](#auto-generated-rest-api)
47
47
  - [High availability with Postgres](#high-availability-with-postgres)
48
+ - [Sync with no database at all](#sync-with-no-database-at-all)
49
+ - [Serverless sync on Vercel](#serverless-sync-on-vercel)
50
+ - [htmx 4 bindings](#htmx-4-bindings)
48
51
  - [Whiteboard + Pictionary example](#whiteboard--pictionary-example)
49
52
  - [Infinite Tetris example](#infinite-tetris-example)
53
+ - [Multiplayer kanban example](#multiplayer-kanban-example)
54
+ - [htmx todos example](#htmx-todos-example)
50
55
  - [Architecture](#architecture)
51
56
  - [Core Concepts](#core-concepts)
52
57
  - [Hybrid Logical Clocks](#hybrid-logical-clocks)
@@ -57,10 +62,12 @@ You bring your own types and your own database. reflectdb handles the protocol,
57
62
  - [API Reference](#api-reference)
58
63
  - [`reflectdb/core`](#reflectdbcore)
59
64
  - [`reflectdb/server`](#reflectdbserver)
65
+ - [`reflectdb/server/storage/object`](#reflectdbserverstorageobject)
60
66
  - [`reflectdb/client`](#reflectdbclient)
61
67
  - [`reflectdb/react`](#reflectdbreact)
62
68
  - [`reflectdb/svelte`](#reflectdbsvelte)
63
69
  - [`reflectdb/vanilla`](#reflectdbvanilla)
70
+ - [`reflectdb/htmx`](#reflectdbhtmx)
64
71
  - [`reflectdb/transport/*`](#reflectdbtransport)
65
72
  - [Configuration Reference](#configuration-reference)
66
73
  - [Query definition](#query-definition)
@@ -81,11 +88,16 @@ You bring your own types and your own database. reflectdb handles the protocol,
81
88
  | Demo | Try it | What it demonstrates |
82
89
  |------|--------|----------------------|
83
90
  | **Infinite multiplayer Tetris** | [Play live](https://reflectdb-tetris.fly.dev/) · [source](./examples/tetris/) | Optimistic input prediction, server reconciliation and gravity, a live leaderboard, per-player progression, and Bun SQLite persistence in one perpetual game. Open two tabs to add another player. |
91
+ | **Multiplayer kanban** | [Open the board](https://reflectdb-kanban.vercel.app/) · [source](./examples/kanban/) | A board whose entire durable state is an S3 bucket — no Postgres, no SQLite, no volume — running on Vercel functions. Shows leaseless optimistic concurrency and serverless SSE. Open two tabs and drag a card. |
92
+ | **htmx 4 todos** | [Open it](https://reflectdb-htmx-todos.fly.dev/) · [source](./examples/htmx-todos/) | htmx owns the DOM, reflectdb owns the data. Attributes point at `reflect:` actions instead of server routes, so the server renders no HTML at all — every fragment comes from the local store. Open two tabs, or stop typing and go offline. The list resets to its seed rows every minute. |
84
93
  | **Collaborative whiteboard** | [Draw live](https://reflectdb-whiteboard.fly.dev/) · [source](./examples/whiteboard/) | Freeform drawing by default, optional Pictionary rounds, guest-authenticated rooms, ephemeral cursors, chat, presence, and per-user query results. Rooms and everything in them are deleted 30 minutes after they are created. Open two tabs to draw with yourself. |
85
94
 
86
- Both demos run on one auto-stopping Fly Machine with no volume, so the first load
87
- after an idle period may take a moment. Their data is intentionally ephemeral
88
- across deployments and Machine replacement.
95
+ Tetris, the whiteboard and the htmx todos each run on one auto-stopping Fly
96
+ Machine with no volume, so the first load after an idle period may take a
97
+ moment. The kanban board has no machine to wake — it is Vercel functions and a
98
+ bucket — but every board resets on a five-minute window, and the htmx todos
99
+ reset every minute. All four keep their data intentionally ephemeral across
100
+ deployments and Machine replacement.
89
101
 
90
102
  ## Why reflectdb
91
103
 
@@ -109,7 +121,7 @@ No code generation. No glue layer. No second source of truth.
109
121
  Optional bits (use what you want):
110
122
 
111
123
  - **Drizzle ORM** — if you point `table` at a Drizzle table, row types are auto-inferred.
112
- - **Server op log storage** — SQLite (for single-node) or Postgres (for HA). Omit it and the op log is in-memory.
124
+ - **Server op log storage** — SQLite (single-node), Postgres (HA), or an S3-compatible bucket (no database at all). Omit it and the op log is in-memory.
113
125
  - **React / Svelte bindings** — use the core client directly if you prefer.
114
126
 
115
127
  ## Features
@@ -119,13 +131,15 @@ Optional bits (use what you want):
119
131
  - **End-to-end type safety** — schema defines row types, query params, writable fields, and which columns the server owns
120
132
  - **Per-row and per-column conflict resolution** — `lww`, `merge`, `server`, or a custom resolver
121
133
  - **Causal ordering** via hybrid logical clocks (HLC) — no dependence on synchronized wall clocks
122
- - **Pluggable storage** — in-memory, SQLite, or Postgres for the server op log; memory or IndexedDB for the browser
134
+ - **Pluggable storage** — in-memory, SQLite, Postgres, or S3-compatible object storage for the server op log; memory or IndexedDB for the browser
123
135
  - **Auto-generated REST** — `server.rest()` turns your schema into CRUD endpoints that broadcast deltas
124
136
  - **Room-based access control** — scope clients to `org/:orgId` or arbitrary patterns
125
137
  - **Rate limiting** — global and per-table, fail-open
126
138
  - **Op log compaction** — configurable retention for old accepted ops
127
139
  - **High availability** — shared Postgres + optional cross-instance polling
128
- - **Framework bindings** — React hooks, Svelte stores, and a vanilla-JS helper; the core client works anywhere
140
+ - **No database at all** — `createObjectStorage` runs a room with an S3-compatible bucket as the only durable store, group-committing one object per batch
141
+ - **Runs serverless** — SSE in `serverless` mode answers each POST with the replies it produced, so sync works on Vercel, Lambda or Workers
142
+ - **Framework bindings** — React hooks, Svelte stores, a vanilla-JS helper, and htmx 4 attribute bindings; the core client works anywhere
129
143
  - **Ephemeral channels** — presence, cursors, typing indicators that never touch the op log, with a room snapshot on join and a pluggable adapter (Redis included) so presence spans a fleet
130
144
  - **Typed presence** — `presence()` in the schema, `usePresence()` in the component, key derived for you
131
145
  - **Read-only views** — `view()` entries that recompute on their dependencies and reject writes at both levels
@@ -141,6 +155,7 @@ Optional bits (use what you want):
141
155
  - Admin tools that should "just update" when someone else changes a row
142
156
  - Field-service or retail apps on spotty networks
143
157
  - Games or canvases with presence indicators and live cursors
158
+ - Serverless deployments with no database to attach and no machine to keep warm
144
159
 
145
160
  ## Installation
146
161
 
@@ -165,13 +180,14 @@ Peer dependencies are all optional:
165
180
 
166
181
  ```bash
167
182
  bun add react # for reflectdb/react
183
+ bun add htmx.org@^4 # for reflectdb/htmx
168
184
  bun add drizzle-orm # if you want auto-inferred row types from Drizzle tables
169
185
  # Svelte + vanilla have no peer deps
170
186
  ```
171
187
 
172
188
  ## Quick Start
173
189
 
174
- A complete sync server in ~30 lines. No ORM, no database — just plain types and an in-memory Map.
190
+ A complete sync server in ~30 lines. No ORM, no database — just plain types and an in-memory Map, handed to reflectdb as `db`.
175
191
 
176
192
  ### 1. Define your schema
177
193
 
@@ -207,7 +223,10 @@ import { queries, type Todo } from "./schema";
207
223
  const todos = new Map<string, Todo>();
208
224
  const transport = createWsServerTransport();
209
225
 
210
- const server = createSyncServer({ queries, transport, serverId: "s1" });
226
+ // `db` is whatever holds your data — an ORM handle, a pool, or a plain Map.
227
+ // It is not optional: reflectdb only runs a `query` callback when it has a
228
+ // `db` to pass it, so leaving it out makes every snapshot come back empty.
229
+ const server = createSyncServer({ queries, db: todos, transport, serverId: "s1" });
211
230
 
212
231
  server.auth(async (req) => {
213
232
  // validate req.headers.get("authorization")
@@ -215,7 +234,7 @@ server.auth(async (req) => {
215
234
  });
216
235
 
217
236
  server.implement("todos", {
218
- query: () => [...todos.values()],
237
+ query: (_ctx, db) => [...db.values()],
219
238
  mutate: async (op) => {
220
239
  if (op.type === "delete") todos.delete(op.rowId);
221
240
  else todos.set(op.rowId, { id: op.rowId, ...(op.payload as Partial<Todo>) } as Todo);
@@ -860,6 +879,145 @@ const server = createSyncServer({
860
879
 
861
880
  Each poll tick first probes the shared op log's head HLC; an idle tick costs one `MAX(hlc)` query and broadcasts nothing. Only tables that actually changed are re-broadcast. The tick also re-merges the shared clock watermark, so an instance whose wall clock lags its peers stops stamping writes below HLCs clients have already seen.
862
881
 
882
+ ### Sync with no database at all
883
+
884
+ `createObjectStorage` runs a room with an S3-compatible bucket as the only durable store — no Postgres, no SQLite, no volume. Works with AWS S3, Cloudflare R2, Tigris, MinIO and GCS.
885
+
886
+ ```ts
887
+ import { createObjectStorage } from "reflectdb/server/storage/object";
888
+
889
+ const storage = createObjectStorage({
890
+ store: {
891
+ provider: "tigris", // or "aws" | "r2" | "minio" | "gcs"
892
+ bucket: process.env.S3_BUCKET!,
893
+ credentials: {
894
+ keyId: process.env.S3_ACCESS_KEY_ID!,
895
+ secret: process.env.S3_SECRET_ACCESS_KEY!,
896
+ },
897
+ },
898
+ roomId: "board-42",
899
+ });
900
+
901
+ const server = createSyncServer({ queries, db, transport, storage });
902
+
903
+ // Flush and release the lease so the next machine takes over immediately.
904
+ process.on("SIGTERM", () => storage.close().then(() => process.exit(0)));
905
+ ```
906
+
907
+ Row state is authoritative **in memory** and the bucket is durability only, so reads never touch the network. A write appends to a buffer that group-commits one object per batch, then advances a compare-and-swapped manifest — the log's single linearization point. The flush loop is self-clocking, so there is no flush interval to tune, and an idle room issues no requests at all.
908
+
909
+ The trade is that a room has exactly one writer, elected with a lease. Route each room to one instance: this adapter replaces shared-database HA polling with room affinity rather than layering on top of it. Where you cannot promise that routing, see [Serverless sync on Vercel](#serverless-sync-on-vercel).
910
+
911
+ Every knob and its default is in [Object storage (no database)](#object-storage-no-database); the design, the failure modes and the known limits are in [`docs/object-storage.md`](./docs/object-storage.md).
912
+
913
+ ### Serverless sync on Vercel
914
+
915
+ Serverless functions break two assumptions a long-lived server gets for free. Both have a flag, and [`examples/kanban`](./examples/kanban/) is a deployed board that uses them.
916
+
917
+ **Any request can land on any instance,** so there is no single writer to elect. Drop the lease and let instances race on the manifest CAS instead:
918
+
919
+ ```ts
920
+ const storage = createObjectStorage({
921
+ store: { /* … */ },
922
+ roomId,
923
+ concurrency: "optimistic", // no lease; the loser of a CAS re-reads and retries
924
+ });
925
+ ```
926
+
927
+ That rests on the same guarantee the lease mode does — the CAS is what keeps the data correct, and the lease was only ever an optimization. What it costs is that in-memory state is no longer authoritative, because another invocation may have committed since this one last looked. Call `await storage.refresh()` — one manifest GET, `true` when something moved — before a read that must be current.
928
+
929
+ **A function cannot hold a WebSocket, and SSE is one-way.** The POST and the event stream are two separate invocations, so a reply to a POST can never reach a stream that process does not own. `serverless: true` returns those replies in the POST's own response instead:
930
+
931
+ ```ts
932
+ const transport = createSseServerTransport({ serverless: true });
933
+ const handler = new MessageHandler({ transport, serverId, db, allowAnonymous: true });
934
+ handler.setStorage(storage);
935
+
936
+ // POST /api/sync/messages — the replies this message produced
937
+ const messages = await transport.collectReplies(clientId, message, () =>
938
+ handler.whenIdle(clientId),
939
+ );
940
+ return Response.json({ messages });
941
+
942
+ // GET /api/sync/events — the stream carries only OTHER clients' changes
943
+ setInterval(async () => {
944
+ if (await storage.refresh()) await handler.pollRemoteChanges();
945
+ }, 250);
946
+ ```
947
+
948
+ `MessageHandler` is exported from `reflectdb/server`; the example drives it directly rather than through `createSyncServer`, because a serverless route needs `whenIdle` and `pollRemoteChanges`. Set the matching `serverless: true` on `createSseClientTransport`. Two more things a serverless deployment owns, both worked through in the kanban example: the client's session and subscriptions are rebuilt per invocation, because the `hello` and `sync_declare` went to a different process, and the stream instance must `bootstrap` so its result cache holds what the client is actually holding — the broadcast engine sends a diff against that cache, and an empty one makes every existing row look new.
949
+
950
+ ### htmx 4 bindings
951
+
952
+ `reflectdb/htmx` lets htmx drive the DOM while reflectdb owns the data. Bindings point at a `reflect:` action instead of a server route, so reads and writes resolve against the local store — optimistic, offline-capable, and re-rendered whenever a peer's change arrives. htmx 4 core ships no SSE or WebSocket support of its own; sync stays reflectdb's job.
953
+
954
+ ```ts
955
+ import htmx from "htmx.org";
956
+ import { createHtmxSync } from "reflectdb/htmx";
957
+
958
+ const reflect = createHtmxSync({
959
+ htmx,
960
+ url: "ws://localhost:3001/sync",
961
+ token,
962
+ tables: ["todos"],
963
+ });
964
+
965
+ reflect.view<Todo>("todos", ({ rows }) =>
966
+ rows
967
+ .map(
968
+ (todo) => `
969
+ <li>
970
+ <input type="checkbox" ${todo.done ? "checked" : ""}
971
+ hx-put="reflect:todos/${todo.id}"
972
+ hx-vals='{"done": "${!todo.done}"}'>
973
+ ${escapeHtml(todo.text)}
974
+ <button hx-delete="reflect:todos/${todo.id}">&times;</button>
975
+ </li>`,
976
+ )
977
+ .join(""),
978
+ );
979
+
980
+ // Form bodies arrive as strings — coerce before they reach the op log.
981
+ reflect.parse<Todo>("todos", (payload) => ({
982
+ ...payload,
983
+ done: payload.done === "true",
984
+ }));
985
+
986
+ await reflect.connect();
987
+ ```
988
+
989
+ ```html
990
+ <ul hx-get="reflect:todos" hx-trigger="load" hx-swap="innerMorph"></ul>
991
+
992
+ <form hx-post="reflect:todos">
993
+ <input name="text" required>
994
+ <input type="hidden" name="done" value="false">
995
+ <button>Add</button>
996
+ </form>
997
+ ```
998
+
999
+ The action grammar is REST-shaped, so the attributes read like ordinary htmx:
1000
+
1001
+ ```
1002
+ GET reflect:<table> → render the collection view
1003
+ GET reflect:<table>/:id → render one row
1004
+ POST reflect:<table> → insert (row id from the body's `id`, else generated)
1005
+ PUT reflect:<table>/:id → update
1006
+ PATCH reflect:<table>/:id → update
1007
+ DELETE reflect:<table>/:id → delete
1008
+ ```
1009
+
1010
+ Writes answer `204 No Content`, so htmx swaps nothing where the write happened. The store change then re-renders every bound element a beat later — one render path whether the edit came from this tab, another tab, or the server.
1011
+
1012
+ Worth knowing:
1013
+
1014
+ - An element binds by making its first `reflect:` read, so give collection bindings `hx-trigger="load"`.
1015
+ - Use an inner swap (`innerHTML`, `innerMorph`) on collection bindings. `outerHTML` replaces the bound element itself, and a replacement still carrying `hx-trigger="load"` would re-request forever.
1016
+ - `hx-swap="innerMorph"` is usually what you want: htmx 4 morphs in place, so focus and caret position survive a re-render.
1017
+ - Query params reach the view for filtering (`reflect:todos?done=false`); they do not change the server subscription. Declare that with `tables`, or with `reflect.sync.sync(table, { params })`.
1018
+ - A row read whose row is missing answers `204`, leaving existing markup alone instead of blanking it.
1019
+ - A view returns a raw HTML string — escape interpolated values yourself.
1020
+
863
1021
  ## Whiteboard + Pictionary example
864
1022
 
865
1023
  A complete React + Bun + Drizzle app that exercises most of reflectdb in one
@@ -931,6 +1089,81 @@ database. The included Fly.io config runs on one auto-stopping 256 MB Machine.
931
1089
  | Headless game rules and top-out reset tests | [`game.ts`](./examples/tetris/game.ts) / [`game.test.ts`](./examples/tetris/game.test.ts) |
932
1090
  | Bun SQLite persistence and restart tests | [`database.ts`](./examples/tetris/database.ts) / [`database.test.ts`](./examples/tetris/database.test.ts) |
933
1091
 
1092
+ ## Multiplayer kanban example
1093
+
1094
+ A shared board whose entire durable state is an S3-compatible bucket, deployed as
1095
+ Vercel functions: [`examples/kanban/`](./examples/kanban/). It is live at
1096
+ [reflectdb-kanban.vercel.app](https://reflectdb-kanban.vercel.app/). No Postgres,
1097
+ no SQLite, no volume, no Redis.
1098
+
1099
+ ```bash
1100
+ cd examples/kanban
1101
+ bun install
1102
+ KANBAN_LOCAL_DIR=.data vercel dev
1103
+ # open http://localhost:3000 in two tabs and drag a card
1104
+ ```
1105
+
1106
+ `KANBAN_LOCAL_DIR` swaps the bucket for a directory, so the example runs with no
1107
+ credentials — the filesystem driver has the same CAS semantics and the whole
1108
+ conformance suite runs against both. `vite` alone serves the UI but not `/api`,
1109
+ so the board will not connect without `vercel dev`.
1110
+
1111
+ The board is open to anyone with the link and `?board=<slug>` makes a new one.
1112
+ Every board resets to its starting cards on a five-minute window, claimed with a
1113
+ single `If-None-Match: *` write so exactly one of N racing invocations does the
1114
+ work — a cron job would not run on Vercel's Hobby tier, and would leave an idle
1115
+ board costing something.
1116
+
1117
+ | Pattern | Where |
1118
+ |---------|-------|
1119
+ | Object storage as the only durable store | `createObjectStorage` in [`lib/board.ts`](./examples/kanban/lib/board.ts) |
1120
+ | `concurrency: "optimistic"` — no lease, instances race on the manifest CAS | [`lib/board.ts`](./examples/kanban/lib/board.ts) |
1121
+ | Serverless SSE — replies returned from the POST that produced them | [`api/sync/messages.ts`](./examples/kanban/api/sync/messages.ts) |
1122
+ | `storage.refresh()` → `handler.pollRemoteChanges()` as the stream's poll loop | [`api/sync/events.ts`](./examples/kanban/api/sync/events.ts) |
1123
+ | Rebuilding a client's session, subscription and result cache per invocation | `restoreSubscription` in [`api/sync/events.ts`](./examples/kanban/api/sync/events.ts) |
1124
+ | Per-column merge so a rename and a drag on the same card both land | `cards` in [`lib/board.ts`](./examples/kanban/lib/board.ts) |
1125
+ | Payload validation with `MutationError` rather than coercion | `cards.mutate` in [`lib/board.ts`](./examples/kanban/lib/board.ts) |
1126
+ | Lazy periodic reset claimed with create-if-absent, applied through `applyServerOp` | [`lib/reset.ts`](./examples/kanban/lib/reset.ts) |
1127
+ | Fractional positioning for drag-and-drop ordering | [`schema.ts`](./examples/kanban/schema.ts) |
1128
+ | Bundling the API routes so Vercel's Node builder never sees a `.ts` specifier | [`scripts/build-kanban.ts`](./scripts/build-kanban.ts) / [`vercel.json`](./vercel.json) |
1129
+
1130
+ Deploy from the repository root rather than the example directory — the example
1131
+ imports reflectdb from `src/`, and the root `vercel.json` runs a build that
1132
+ bundles the two API routes into `.vercel/output` itself. Pushing to `main` does
1133
+ it; the demo's Vercel project is linked to this repository at the repository
1134
+ root. The variables to set, and the one extra `storage.init()` step MinIO needs,
1135
+ are in [`examples/kanban/README.md`](./examples/kanban/README.md).
1136
+
1137
+ ## htmx todos example
1138
+
1139
+ A todo list where htmx 4 owns the DOM and reflectdb owns the data:
1140
+ [`examples/htmx-todos/`](./examples/htmx-todos/). It is live at
1141
+ [reflectdb-htmx-todos.fly.dev](https://reflectdb-htmx-todos.fly.dev/). The
1142
+ server renders no HTML — every fragment is produced in the browser from the
1143
+ local store. The deployed list resets to its seed rows every minute.
1144
+
1145
+ ```bash
1146
+ cd examples/htmx-todos
1147
+ bun install
1148
+ bun run dev
1149
+ # open http://localhost:3005 in two tabs
1150
+ ```
1151
+
1152
+ The server takes the first free port at or above `PORT` (default 3005), so it
1153
+ does not collide with the other examples.
1154
+
1155
+ | Pattern | Where |
1156
+ |---------|-------|
1157
+ | `reflect:` actions on ordinary htmx attributes | [`index.html`](./examples/htmx-todos/index.html) |
1158
+ | One view function rendering the whole list from local rows | [`client.ts`](./examples/htmx-todos/client.ts) |
1159
+ | `parse` building a patch, so a checkbox toggle keeps the text it never sent | [`client.ts`](./examples/htmx-todos/client.ts) |
1160
+ | A counter riding along as an `hx-swap-oob` element | [`client.ts`](./examples/htmx-todos/client.ts) |
1161
+ | Filters as query params, kept across peers' edits | [`index.html`](./examples/htmx-todos/index.html) |
1162
+ | A periodic reset routed through `applyServerOp`, so it reaches every open tab | [`server.ts`](./examples/htmx-todos/server.ts) |
1163
+
1164
+ Stop the server and keep typing: writes land in the DOM immediately, the badge
1165
+ counts them, and they drain on reconnect.
1166
+
934
1167
  ## Architecture
935
1168
 
936
1169
  ```
@@ -982,6 +1215,7 @@ database. The included Fly.io config runs on one auto-stopping 256 MB Machine.
982
1215
  │ • in-memory (default) │ │ │
983
1216
  │ • sqlite (bun:sqlite) │ │ │
984
1217
  │ • postgres (any pg-compatible) │ │ │
1218
+ │ • object (any S3-compatible) │ │ │
985
1219
  └───────────────────────────────────┘ └──────────────────────────────────────┘
986
1220
 
987
1221
  ┌─────────────────────────────────────────────────────────────────────────────┐
@@ -995,14 +1229,60 @@ database. The included Fly.io config runs on one auto-stopping 256 MB Machine.
995
1229
  └─────────────────────────────────────────────────────────────────────────────┘
996
1230
 
997
1231
  ┌─────────────────────────────────────────────────────────────────────────────┐
998
- FRAMEWORK BINDINGS (react/, svelte/, vanilla/)
1232
+ FRAMEWORK BINDINGS (react/, svelte/, vanilla/, htmx/)
999
1233
  │ │
1000
1234
  │ • createSyncReact(queries) → typed hooks + <SyncProvider> │
1001
1235
  │ • createSyncSvelte(queries) → typed Svelte stores │
1002
1236
  │ • createSyncVanilla(queries) → typed callback API │
1237
+ │ • createSyncHtmx(queries) → typed views behind reflect: attributes │
1003
1238
  └─────────────────────────────────────────────────────────────────────────────┘
1004
1239
  ```
1005
1240
 
1241
+ ### Object storage as the durable store
1242
+
1243
+ `createObjectStorage` replaces that op-log box with an S3-compatible bucket, and
1244
+ nothing else in the diagram changes. The bucket is durability, never the read
1245
+ path:
1246
+
1247
+ ```
1248
+ ┌──────────────────────────────────────┐
1249
+ clients ───────▶ │ writer instance (one per room) │
1250
+ │ │
1251
+ │ in-memory authoritative state │ ◀── every read
1252
+ │ rows · per-column HLCs · op ring │ lands here
1253
+ │ reserveOp set · meta │
1254
+ │ │
1255
+ │ write buffer ──▶ group commit │
1256
+ └───────────────────┬──────────────────┘
1257
+ │ one PUT per batch,
1258
+ │ then one CAS
1259
+
1260
+ ┌──────────────────────────────────────┐
1261
+ │ object store — S3 · R2 · Tigris · │
1262
+ │ MinIO · GCS │
1263
+ │ │
1264
+ │ _lease writer election │
1265
+ │ _manifest the CAS'd commit point │
1266
+ │ wal/ immutable op batches │
1267
+ │ snap/ materialized rows │
1268
+ └──────────────────────────────────────┘
1269
+ ```
1270
+
1271
+ Reads (`getRow`, `getRows`, `getOpsSince`, `reserveOp`) hit memory and never
1272
+ touch the network. A write mutates memory and appends to a buffer; the buffer
1273
+ flushes as one object per batch, and the commit is the compare-and-swap that adds
1274
+ that segment to `_manifest`. Boot is one manifest GET, the newest snapshot, and
1275
+ the segments the manifest still lists.
1276
+
1277
+ Only `_lease` and `_manifest` are ever overwritten, and only via CAS. Everything
1278
+ else is write-once, which is what makes concurrent readers safe — and what lets
1279
+ `concurrency: "optimistic"` drop the lease entirely on a platform that cannot
1280
+ route a room to one instance.
1281
+
1282
+ Full design, provider compatibility, durability model and known limits:
1283
+ [`docs/object-storage.md`](./docs/object-storage.md).
1284
+
1285
+
1006
1286
  ## Core Concepts
1007
1287
 
1008
1288
  ### Hybrid Logical Clocks
@@ -1073,7 +1353,7 @@ reflectdb keeps its own store alongside yours, and it helps to know which one an
1073
1353
 
1074
1354
  | Read | Source |
1075
1355
  |------|--------|
1076
- | Snapshots (`bootstrap`, `resume`) | **Your database**, via the `query` callback |
1356
+ | Snapshots (`bootstrap`, `resume`) | **Your database**, via the `query` callback — which only runs when `db` was passed to `createSyncServer` |
1077
1357
  | Broadcast deltas | **Your database**, diffed against a per-client cached result set |
1078
1358
  | Conflict resolution (`lww` / `merge` / `server` / custom) | **reflectdb's mirror** — a JSONB row store plus per-column HLCs |
1079
1359
  | Which tables changed since an HLC | **reflectdb's op log** |
@@ -1168,6 +1448,42 @@ See [Server-driven game loops](#server-driven-game-loops) for `interval` / `lock
1168
1448
 
1169
1449
  `createServer()` is the lower-level untyped variant — use it only if you need to register queries dynamically or don't have a `defineSyncQueries` map.
1170
1450
 
1451
+ ### `reflectdb/server/storage/object`
1452
+
1453
+ ```ts
1454
+ import {
1455
+ createObjectStorage,
1456
+ createS3Driver, createFilesystemDriver, createMemoryDriver,
1457
+ PreconditionFailedError, BackpressureError, NotWriterError, MemoryLimitExceededError,
1458
+ IncompleteStateError, roomPrefix,
1459
+ } from "reflectdb/server/storage/object";
1460
+ ```
1461
+
1462
+ **`createObjectStorage(config)`** — a storage adapter backed by an S3-compatible bucket. It satisfies the same `StorageAdapter` contract as the SQLite and Postgres adapters, plus a lifecycle and observability surface the design needs:
1463
+
1464
+ | Member | Purpose |
1465
+ |--------|---------|
1466
+ | `init()` | Boot the room — manifest, snapshot, WAL replay. Idempotent, and implied by the first call to anything else; call it explicitly to surface boot failures at startup rather than on the first query. On MinIO it also seeds `_lease` and `_manifest`, which makes it a **deploy step** rather than something N servers race. |
1467
+ | `refresh()` | Fold in whatever another instance has committed. Returns `true` when the room actually moved. Costs one GET, and nothing further when `commitSeq` has not changed. Only meaningful under `concurrency: "optimistic"`. |
1468
+ | `flush()` | Resolve once everything buffered is durable. |
1469
+ | `close()` | Stop accepting writes, flush, stop the flush loop, release the lease. Every step is bounded by `shutdownFlushMs`, so a hung store cannot hold a `SIGTERM` open. |
1470
+ | `health` | `"healthy"`, `"degraded"` or `"unavailable"`, with `onHealthChange(cb)` to observe transitions. |
1471
+ | `durableHlc` | Highest HLC known to be on the store, with `onDurable(cb)` fired per batch. |
1472
+
1473
+ Errors are typed so a caller can tell "someone else moved first" from a transport failure: `PreconditionFailedError` (a CAS lost), `BackpressureError` (the write buffer hit `batch.maxBufferBytes`), `NotWriterError` (this instance lost or never held the lease), `MemoryLimitExceededError` (room state exceeded `memory.maxRoomBytes`), `IncompleteStateError` (the manifest names an object the store does not have — the room refuses to boot rather than present the loss as an empty room).
1474
+
1475
+ `roomPrefix(roomId)` returns the key prefix a room writes under, for tooling that has to address those keys from outside the adapter — an admin wipe, or a disposable room clearing itself after an `IncompleteStateError`.
1476
+
1477
+ Pass `store` and the S3 driver is built for you; pass `driver` to supply one directly:
1478
+
1479
+ | Driver | Signature | Use |
1480
+ |--------|-----------|-----|
1481
+ | `createS3Driver` | `(config: StoreConfig)` | Any S3-compatible bucket. SigV4 over `fetch` with WebCrypto — no `node:crypto`, so the browser build stays clean. |
1482
+ | `createFilesystemDriver` | `(rootDir: string)` | Local development and tests with no network. Same CAS semantics via atomic rename, but a filesystem cannot make compare-then-rename atomic across processes — single process only. |
1483
+ | `createMemoryDriver` | `(options?)` | Tests, with fault injection (412 storms, 500s, latency) and `casWildcard: false` to reproduce MinIO. |
1484
+
1485
+ See [Object storage (no database)](#object-storage-no-database) for the configuration, and [`docs/object-storage.md`](./docs/object-storage.md) for the design.
1486
+
1171
1487
  ### `reflectdb/client`
1172
1488
 
1173
1489
  ```ts
@@ -1272,19 +1588,40 @@ store.onError((e) => …);
1272
1588
  ```ts
1273
1589
  import { createSync, createSyncVanilla, createBrowserWsTransport } from "reflectdb/vanilla";
1274
1590
 
1275
- const sync = createSync({ url, token, tables: ["notes"] });
1591
+ const sync = createSync({ url, token, tables: ["notes"], onError: (e) => … });
1276
1592
  const notes = sync.sync<Note>("notes");
1277
1593
 
1278
- notes.onChange((rows) => render(rows));
1594
+ notes.onChange(() => render(notes.getRows()));
1279
1595
  notes.insert(id, { title: "…" });
1280
1596
 
1281
1597
  sync.onStateChange((s) => …);
1282
1598
  sync.onPendingChange((n) => …);
1283
- sync.onError((e) => …);
1284
1599
  sync.connect();
1285
1600
  ```
1286
1601
 
1287
- Also supports ephemeral: `sync.sendEphemeral({ key, userId, data })`, `sync.onEphemeral(key, listener)`.
1602
+ Also supports ephemeral: `sync.ephemeral({ key, userId, ttlMs })` returns a binding with `broadcast(data)`, `getEvents()` and `onChange(listener)`.
1603
+
1604
+ ### `reflectdb/htmx`
1605
+
1606
+ ```ts
1607
+ import { createHtmxSync, createSyncHtmx } from "reflectdb/htmx";
1608
+
1609
+ const reflect = createHtmxSync({ htmx, url, token, tables: ["todos"] });
1610
+
1611
+ reflect.view<Todo>("todos", ({ rows, rowId, params }) => "<li>…</li>");
1612
+ reflect.parse<Todo>("todos", (payload) => ({ …payload, done: payload.done === "true" }));
1613
+
1614
+ reflect.install(); // attach the reflect: shim (connect() does this too)
1615
+ reflect.uninstall(); // detach it and drop every element binding
1616
+ reflect.refresh("todos"); // re-render bound elements on demand
1617
+ reflect.sync; // the underlying VanillaSync, for anything attributes cannot express
1618
+
1619
+ await reflect.connect();
1620
+ ```
1621
+
1622
+ Requires htmx 4 — import the instance yourself and pass it in. The adapter listens for `htmx:config:request` and sets `ctx.fetch` on `reflect:` actions, so htmx still performs the swap, OOB handling, settling and history it would for a server response.
1623
+
1624
+ `createSyncHtmx<typeof queries>()` returns the same factory with `view`, `parse` and `refresh` narrowed to your schema's table names and row types. See [htmx 4 bindings](#htmx-4-bindings) for the action grammar.
1288
1625
 
1289
1626
  ### `reflectdb/transport/*`
1290
1627
 
@@ -1502,11 +1839,105 @@ At boot the client restores its persisted subscriptions first and hydrates only
1502
1839
  | _(none)_ | omit `storage` | In-memory op log; ephemeral, single node |
1503
1840
  | `createSqliteStorage({ path?, db? })` | `reflectdb/server` | Single server, development, embedded — **Bun only** |
1504
1841
  | `createPostgresStorage(poolOrConfig)` | `reflectdb/server` | Multi-server HA, production |
1842
+ | `createObjectStorage({ store, roomId })` | `reflectdb/server/storage/object` | S3-compatible object storage as the only durable store — no database |
1505
1843
 
1506
1844
  `createPostgresStorage` accepts any object with `query(text, values) => { rows }` — `pg.Pool`, `pg.Client`, `@neondatabase/serverless`, etc. Optional config: `{ client, tablePrefix: "_reflectdb" }`.
1507
1845
 
1508
1846
  `createSqliteStorage` is backed by `bun:sqlite` and is resolved lazily, so importing `reflectdb/server` on Node is fine — only calling `createSqliteStorage` there throws. On Node, use `createPostgresStorage` (or omit `storage` for the in-memory op log).
1509
1847
 
1848
+ #### Object storage (no database)
1849
+
1850
+ `createObjectStorage` runs a room with S3-compatible object storage as the only
1851
+ durable store. Works with AWS S3, Cloudflare R2, Tigris, MinIO and GCS.
1852
+
1853
+ ```ts
1854
+ import { createObjectStorage } from "reflectdb/server/storage/object";
1855
+
1856
+ const storage = createObjectStorage({
1857
+ store: {
1858
+ provider: "tigris", // or "aws" | "r2" | "minio" | "gcs"
1859
+ bucket: "my-app",
1860
+ credentials: {
1861
+ keyId: process.env.AWS_ACCESS_KEY_ID!,
1862
+ secret: process.env.AWS_SECRET_ACCESS_KEY!,
1863
+ },
1864
+ },
1865
+ roomId: "board-42",
1866
+ });
1867
+
1868
+ const server = createSyncServer({ queries, db, transport, storage });
1869
+
1870
+ // Flush and release the lease so the next machine takes over immediately.
1871
+ process.on("SIGTERM", () => storage.close().then(() => process.exit(0)));
1872
+ ```
1873
+
1874
+ Row state is authoritative **in memory** and the object store is durability
1875
+ only, so reads never touch the network. Writes group-commit: one PUT per batch,
1876
+ self-clocking, with no flush interval to tune. An idle room issues zero requests.
1877
+
1878
+ Because state is in memory and a room has exactly one writer, **route each room
1879
+ to one instance** — this adapter replaces shared-database HA polling with room
1880
+ affinity rather than layering on top of it.
1881
+
1882
+ Where you cannot promise that routing — serverless functions, where any request
1883
+ lands on any instance — set `concurrency: "optimistic"`. It drops the lease and
1884
+ lets instances race on the manifest CAS instead, with the loser re-reading and
1885
+ retrying. That rests on the same guarantee the lease mode does: the CAS is what
1886
+ keeps the data correct, and the lease was only ever an optimization. In exchange,
1887
+ in-memory state is no longer authoritative, so call `await storage.refresh()`
1888
+ (one manifest GET) before a read that must be current.
1889
+
1890
+ Defaults are correct rather than fast: `durability: "durable"` acks a write only
1891
+ once it is on the store. `"buffered"` acks earlier and is lossy on crash until
1892
+ the durable-watermark protocol lands.
1893
+
1894
+ The `SIGTERM` handler above matters — without it a deploy drops whatever is
1895
+ buffered, and the next writer waits out `lease.ttlMs` before taking the room.
1896
+
1897
+ MinIO needs a one-time `await storage.init()` as a **deploy step** (it rejects
1898
+ the `If-None-Match: *` wildcard, so the room has to be seeded); on every other
1899
+ provider `init()` is a no-op.
1900
+
1901
+ `store` describes the bucket. `provider` fills in `endpoint` and `urlStyle`, so most deployments set three fields and nothing else:
1902
+
1903
+ | Field | Required | Notes |
1904
+ |-------|----------|-------|
1905
+ | `bucket` | yes | |
1906
+ | `credentials` | yes | `{ keyId, secret, sessionToken? }`. |
1907
+ | `provider` | no | `"aws"`, `"r2"`, `"tigris"`, `"minio"` or `"gcs"` — a preset for `endpoint`, `region` and `urlStyle`. |
1908
+ | `accountId` | for R2 | Derives `https://<id>.r2.cloudflarestorage.com`. |
1909
+ | `endpoint` / `region` / `urlStyle` | no | Override the preset. MinIO has no default endpoint, and a Fly-provisioned Tigris bucket uses `fly.storage.tigris.dev` rather than the preset's `t3.storage.dev`. |
1910
+ | `prefix` | no | Key prefix, so one bucket holds many apps. Keys are `<prefix>/rooms/<roomId>/…`. |
1911
+
1912
+ Pass `driver` instead of `store` to supply a driver directly — `createFilesystemDriver(dir)` for local development with no credentials, `createMemoryDriver()` for tests.
1913
+
1914
+ Everything else, with its default:
1915
+
1916
+ | Option | Default | Purpose |
1917
+ |--------|---------|---------|
1918
+ | `roomId` | — | Required. One room per adapter; it is the key prefix and the routing key. |
1919
+ | `writerId` | random | Identifies this writer in the lease. Set it to make ownership legible in logs. |
1920
+ | `durability` | `"durable"` | Ack after the manifest CAS. `"buffered"` acks on memory apply and is lossy on crash until the durable-watermark protocol lands. |
1921
+ | `concurrency` | `"single-writer"` | `"optimistic"` drops the lease where a room cannot be routed to one instance. See [Serverless sync on Vercel](#serverless-sync-on-vercel). |
1922
+ | `retentionMs` | `Infinity` | How long accepted ops are kept. |
1923
+ | `batch.maxBytes` | 4 MiB | Largest single WAL segment. |
1924
+ | `batch.minLingerMs` | `5` | Coalesces ops arriving in the same event-loop tick. Not a flush interval — there is none, the flush loop is self-clocking. |
1925
+ | `batch.maxBufferBytes` | 64 MiB | Buffer ceiling before the backpressure policy fires. |
1926
+ | `batch.onBackpressure` | `"reject"` | Throw `BackpressureError` so backpressure reaches the client. `"degrade"` keeps accepting, stops promising durability, and flips `health`. |
1927
+ | `compaction.afterSegments` | `200` | Snapshot once the manifest lists this many segments. Boot costs one GET per listed segment, so lower it for rooms that are read cold more often than they are written. |
1928
+ | `compaction.afterBytes` | 64 MiB | Or this many bytes, whichever comes first. |
1929
+ | `compaction.gcGraceMs` | `3_600_000` | Delay before deleting superseded segments, so a reader holding the old manifest does not 404. |
1930
+ | `lease.ttlMs` / `lease.renewMs` | `300_000` / `120_000` | Long on purpose. `close()` releases the lease, so the TTL bounds only *unclean* failover — and a long one is what keeps an idle room free. |
1931
+ | `lease.mode` | `"on-write"` | A room holding connected clients but taking no writes renews nothing. |
1932
+ | `memory.maxTotalBytes` / `memory.maxRoomBytes` | `Infinity` | Global across rooms, then per room. State is authoritative in memory, so this is a cliff rather than a slope — set it and the ceiling arrives as `MemoryLimitExceededError` instead of an OOM. |
1933
+ | `memory.onExceeded` | `"reject"` | `"evict"` and `"spill"` are accepted as configuration but throw at the limit; neither is implemented yet. |
1934
+ | `memory.idleEvictMs` | `300_000` | Zero-client rooms flush, release the lease and drop their state. |
1935
+ | `shutdownFlushMs` | `5000` | Bounds every step of `close()`, so a hung store cannot hold a `SIGTERM` open. |
1936
+ | `onDurable` / `onHealthChange` | — | Callbacks. The same values are readable as `storage.durableHlc` and `storage.health`. |
1937
+
1938
+ Full design, the configuration reference and the known limits:
1939
+ [`docs/object-storage.md`](./docs/object-storage.md).
1940
+
1510
1941
  #### Client
1511
1942
 
1512
1943
  | Adapter | Import | Best for |
@@ -1592,11 +2023,42 @@ Returns a `ServerTransport` plus `handleOpen`, `handleMessage`, `handleClose`, `
1592
2023
  ```ts
1593
2024
  createSseServerTransport({
1594
2025
  replayBufferSize: 256, // Last-Event-ID replay window
2026
+ serverless: false, // see below
1595
2027
  });
1596
2028
  ```
1597
2029
 
1598
2030
  Two endpoints to wire: `GET /sync/events/:clientId` (SSE stream) and `POST /sync/messages/:clientId` (client → server).
1599
2031
 
2032
+ ##### Serverless SSE
2033
+
2034
+ SSE is server→client only: normally a client POSTs a message and every reply —
2035
+ `hello_ack`, snapshots, op acks — comes back down the held stream. That works
2036
+ only while the POST and the stream are handled by the **same process**.
2037
+
2038
+ On Vercel, Lambda or Workers they are two separate invocations, so those replies
2039
+ would be enqueued onto a stream the POST's process does not have, and the client
2040
+ hangs at the handshake. Set `serverless: true` on both halves:
2041
+
2042
+ ```ts
2043
+ // server: return the replies the POST produced
2044
+ const messages = await transport.collectReplies(clientId, message, () =>
2045
+ handler.whenIdle(clientId),
2046
+ );
2047
+ return Response.json({ messages });
2048
+
2049
+ // client
2050
+ createSseClientTransport({ eventUrl, messageUrl, serverless: true });
2051
+ ```
2052
+
2053
+ The stream is then left doing the one thing it can: pushing *other* clients'
2054
+ changes. Leave it off for a single-process server — replies stream normally
2055
+ there, and turning it on would deliver each one twice.
2056
+
2057
+ Two things a serverless deployment must also handle, both shown in
2058
+ [`examples/kanban`](./examples/kanban/): the session is rebuilt per invocation
2059
+ (nothing remembers that this client subscribed), and the stream instance has to
2060
+ poll storage to notice writes made elsewhere.
2061
+
1600
2062
  #### HTTP long-polling
1601
2063
 
1602
2064
  ```ts
@@ -1663,6 +2125,7 @@ reference:
1663
2125
  | `og/index.html` | `public/og.png` | 1200x630 at 2x | reflectdb.dev |
1664
2126
  | `og/tetris.html` | `public/og-tetris.png` | 1200x630 at 2x | the Tetris demo — served from reflectdb.dev, since the Fly Machine sleeps |
1665
2127
  | `og/whiteboard.html` | `public/og-whiteboard.png` | 1200x630 at 2x | the whiteboard demo, served from reflectdb.dev for the same reason |
2128
+ | `og/kanban.html` | `public/og-kanban.png` | 1200x630 at 2x | the kanban demo, so all four cards regenerate together |
1666
2129
  | `og/github.html` | `public/og-github.png` | 1280x640 at 2x | this repository's social preview, uploaded by hand under Settings → Social preview |
1667
2130
 
1668
2131
  Edit the HTML, not the PNGs. 1200x630 is the one ratio X, Facebook, LinkedIn,
@@ -1697,12 +2160,15 @@ src/
1697
2160
  │ │ broadcast-engine, result-cache, eager-buffer,
1698
2161
  │ │ compaction-manager, replay-detector, ephemeral-manager
1699
2162
  │ └── storage/ SQLite + Postgres adapters
2163
+ │ └── object/ S3-compatible object storage — manifest CAS, WAL,
2164
+ │ snapshots, memory / filesystem / S3 drivers
1700
2165
  ├── client/ sync-client, store, ops, typed-client
1701
2166
  │ └── storage/ memory + IndexedDB adapters
1702
2167
  ├── transport/ WebSocket (runtime-agnostic + Bun.serve), SSE, polling
1703
2168
  ├── react/ <SyncProvider>, hooks, typed factory
1704
2169
  ├── svelte/ createSyncStore, typed factory
1705
- └── vanilla/ createSync, typed factory
2170
+ ├── vanilla/ createSync, typed factory
2171
+ └── htmx/ createHtmxSync, reflect: action router, typed factory
1706
2172
  ```
1707
2173
 
1708
2174
  ### Tech stack