reflectdb 0.1.3 → 0.3.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -18,7 +18,7 @@ You bring your own types and your own database. reflectdb handles the protocol,
18
18
  └──────────────┘ └──────────────┘ └──────────────┘
19
19
  ▲ │
20
20
  │ offline ▼
21
- └────── IndexedDB ───── op log (in-memory / SQLite / Postgres)
21
+ └────── IndexedDB ───── op log (in-memory / SQLite / Postgres / S3)
22
22
  ```
23
23
 
24
24
  ## Table of Contents
@@ -45,8 +45,13 @@ You bring your own types and your own database. reflectdb handles the protocol,
45
45
  - [Windowed sync and pagination](#windowed-sync-and-pagination)
46
46
  - [Auto-generated REST API](#auto-generated-rest-api)
47
47
  - [High availability with Postgres](#high-availability-with-postgres)
48
+ - [Sync with no database at all](#sync-with-no-database-at-all)
49
+ - [Serverless sync on Vercel](#serverless-sync-on-vercel)
50
+ - [htmx 4 bindings](#htmx-4-bindings)
48
51
  - [Whiteboard + Pictionary example](#whiteboard--pictionary-example)
49
52
  - [Infinite Tetris example](#infinite-tetris-example)
53
+ - [Multiplayer kanban example](#multiplayer-kanban-example)
54
+ - [htmx todos example](#htmx-todos-example)
50
55
  - [Architecture](#architecture)
51
56
  - [Core Concepts](#core-concepts)
52
57
  - [Hybrid Logical Clocks](#hybrid-logical-clocks)
@@ -57,10 +62,12 @@ You bring your own types and your own database. reflectdb handles the protocol,
57
62
  - [API Reference](#api-reference)
58
63
  - [`reflectdb/core`](#reflectdbcore)
59
64
  - [`reflectdb/server`](#reflectdbserver)
65
+ - [`reflectdb/server/storage/object`](#reflectdbserverstorageobject)
60
66
  - [`reflectdb/client`](#reflectdbclient)
61
67
  - [`reflectdb/react`](#reflectdbreact)
62
68
  - [`reflectdb/svelte`](#reflectdbsvelte)
63
69
  - [`reflectdb/vanilla`](#reflectdbvanilla)
70
+ - [`reflectdb/htmx`](#reflectdbhtmx)
64
71
  - [`reflectdb/transport/*`](#reflectdbtransport)
65
72
  - [Configuration Reference](#configuration-reference)
66
73
  - [Query definition](#query-definition)
@@ -81,11 +88,16 @@ You bring your own types and your own database. reflectdb handles the protocol,
81
88
  | Demo | Try it | What it demonstrates |
82
89
  |------|--------|----------------------|
83
90
  | **Infinite multiplayer Tetris** | [Play live](https://reflectdb-tetris.fly.dev/) · [source](./examples/tetris/) | Optimistic input prediction, server reconciliation and gravity, a live leaderboard, per-player progression, and Bun SQLite persistence in one perpetual game. Open two tabs to add another player. |
91
+ | **Multiplayer kanban** | [Open the board](https://reflectdb-kanban.vercel.app/) · [source](./examples/kanban/) | A board whose entire durable state is an S3 bucket — no Postgres, no SQLite, no volume — running on Vercel functions. Shows leaseless optimistic concurrency and serverless SSE. Open two tabs and drag a card. |
92
+ | **htmx 4 todos** | [Open it](https://reflectdb-htmx-todos.fly.dev/) · [source](./examples/htmx-todos/) | htmx owns the DOM, reflectdb owns the data. Attributes point at `reflect:` actions instead of server routes, so the server renders no HTML at all — every fragment comes from the local store. Open two tabs, or stop typing and go offline. The list resets to its seed rows every minute. |
84
93
  | **Collaborative whiteboard** | [Draw live](https://reflectdb-whiteboard.fly.dev/) · [source](./examples/whiteboard/) | Freeform drawing by default, optional Pictionary rounds, guest-authenticated rooms, ephemeral cursors, chat, presence, and per-user query results. Rooms and everything in them are deleted 30 minutes after they are created. Open two tabs to draw with yourself. |
85
94
 
86
- Both demos run on one auto-stopping Fly Machine with no volume, so the first load
87
- after an idle period may take a moment. Their data is intentionally ephemeral
88
- across deployments and Machine replacement.
95
+ Tetris, the whiteboard and the htmx todos each run on one auto-stopping Fly
96
+ Machine with no volume, so the first load after an idle period may take a
97
+ moment. The kanban board has no machine to wake — it is Vercel functions and a
98
+ bucket — but every board resets on a five-minute window, and the htmx todos
99
+ reset every minute. All four keep their data intentionally ephemeral across
100
+ deployments and Machine replacement.
89
101
 
90
102
  ## Why reflectdb
91
103
 
@@ -109,7 +121,7 @@ No code generation. No glue layer. No second source of truth.
109
121
  Optional bits (use what you want):
110
122
 
111
123
  - **Drizzle ORM** — if you point `table` at a Drizzle table, row types are auto-inferred.
112
- - **Server op log storage** — SQLite (for single-node) or Postgres (for HA). Omit it and the op log is in-memory.
124
+ - **Server op log storage** — SQLite (single-node), Postgres (HA), or an S3-compatible bucket (no database at all). Omit it and the op log is in-memory.
113
125
  - **React / Svelte bindings** — use the core client directly if you prefer.
114
126
 
115
127
  ## Features
@@ -119,13 +131,15 @@ Optional bits (use what you want):
119
131
  - **End-to-end type safety** — schema defines row types, query params, writable fields, and which columns the server owns
120
132
  - **Per-row and per-column conflict resolution** — `lww`, `merge`, `server`, or a custom resolver
121
133
  - **Causal ordering** via hybrid logical clocks (HLC) — no dependence on synchronized wall clocks
122
- - **Pluggable storage** — in-memory, SQLite, or Postgres for the server op log; memory or IndexedDB for the browser
134
+ - **Pluggable storage** — in-memory, SQLite, Postgres, or S3-compatible object storage for the server op log; memory or IndexedDB for the browser
123
135
  - **Auto-generated REST** — `server.rest()` turns your schema into CRUD endpoints that broadcast deltas
124
136
  - **Room-based access control** — scope clients to `org/:orgId` or arbitrary patterns
125
137
  - **Rate limiting** — global and per-table, fail-open
126
138
  - **Op log compaction** — configurable retention for old accepted ops
127
139
  - **High availability** — shared Postgres + optional cross-instance polling
128
- - **Framework bindings** — React hooks, Svelte stores, and a vanilla-JS helper; the core client works anywhere
140
+ - **No database at all** — `createObjectStorage` runs a room with an S3-compatible bucket as the only durable store, group-committing one object per batch
141
+ - **Runs serverless** — SSE in `serverless` mode answers each POST with the replies it produced, so sync works on Vercel, Lambda or Workers
142
+ - **Framework bindings** — React hooks, Svelte stores, a vanilla-JS helper, and htmx 4 attribute bindings; the core client works anywhere
129
143
  - **Ephemeral channels** — presence, cursors, typing indicators that never touch the op log, with a room snapshot on join and a pluggable adapter (Redis included) so presence spans a fleet
130
144
  - **Typed presence** — `presence()` in the schema, `usePresence()` in the component, key derived for you
131
145
  - **Read-only views** — `view()` entries that recompute on their dependencies and reject writes at both levels
@@ -141,6 +155,7 @@ Optional bits (use what you want):
141
155
  - Admin tools that should "just update" when someone else changes a row
142
156
  - Field-service or retail apps on spotty networks
143
157
  - Games or canvases with presence indicators and live cursors
158
+ - Serverless deployments with no database to attach and no machine to keep warm
144
159
 
145
160
  ## Installation
146
161
 
@@ -165,13 +180,14 @@ Peer dependencies are all optional:
165
180
 
166
181
  ```bash
167
182
  bun add react # for reflectdb/react
183
+ bun add htmx.org@^4 # for reflectdb/htmx
168
184
  bun add drizzle-orm # if you want auto-inferred row types from Drizzle tables
169
185
  # Svelte + vanilla have no peer deps
170
186
  ```
171
187
 
172
188
  ## Quick Start
173
189
 
174
- A complete sync server in ~30 lines. No ORM, no database — just plain types and an in-memory Map.
190
+ A complete sync server in ~30 lines. No ORM, no database — just plain types and an in-memory Map, handed to reflectdb as `db`.
175
191
 
176
192
  ### 1. Define your schema
177
193
 
@@ -207,7 +223,10 @@ import { queries, type Todo } from "./schema";
207
223
  const todos = new Map<string, Todo>();
208
224
  const transport = createWsServerTransport();
209
225
 
210
- const server = createSyncServer({ queries, transport, serverId: "s1" });
226
+ // `db` is whatever holds your data — an ORM handle, a pool, or a plain Map.
227
+ // It is not optional: reflectdb only runs a `query` callback when it has a
228
+ // `db` to pass it, so leaving it out makes every snapshot come back empty.
229
+ const server = createSyncServer({ queries, db: todos, transport, serverId: "s1" });
211
230
 
212
231
  server.auth(async (req) => {
213
232
  // validate req.headers.get("authorization")
@@ -215,7 +234,7 @@ server.auth(async (req) => {
215
234
  });
216
235
 
217
236
  server.implement("todos", {
218
- query: () => [...todos.values()],
237
+ query: (_ctx, db) => [...db.values()],
219
238
  mutate: async (op) => {
220
239
  if (op.type === "delete") todos.delete(op.rowId);
221
240
  else todos.set(op.rowId, { id: op.rowId, ...(op.payload as Partial<Todo>) } as Todo);
@@ -860,6 +879,148 @@ const server = createSyncServer({
860
879
 
861
880
  Each poll tick first probes the shared op log's head HLC; an idle tick costs one `MAX(hlc)` query and broadcasts nothing. Only tables that actually changed are re-broadcast. The tick also re-merges the shared clock watermark, so an instance whose wall clock lags its peers stops stamping writes below HLCs clients have already seen.
862
881
 
882
+ ### Sync with no database at all
883
+
884
+ `createObjectStorage` runs a room with an S3-compatible bucket as the only durable store — no Postgres, no SQLite, no volume. Works with AWS S3, Cloudflare R2, Tigris, MinIO and GCS.
885
+
886
+ ```ts
887
+ import { createObjectStorage } from "reflectdb/server/storage/object";
888
+
889
+ const storage = createObjectStorage({
890
+ store: {
891
+ provider: "tigris", // or "aws" | "r2" | "minio" | "gcs"
892
+ bucket: process.env.S3_BUCKET!,
893
+ credentials: {
894
+ keyId: process.env.S3_ACCESS_KEY_ID!,
895
+ secret: process.env.S3_SECRET_ACCESS_KEY!,
896
+ },
897
+ },
898
+ roomId: "board-42",
899
+ });
900
+
901
+ const server = createSyncServer({ queries, db, transport, storage });
902
+
903
+ // Flush and release the lease so the next machine takes over immediately.
904
+ process.on("SIGTERM", () => storage.close().then(() => process.exit(0)));
905
+ ```
906
+
907
+ Row state is authoritative **in memory** and the bucket is durability only, so reads never touch the network. A write appends to a buffer that group-commits one object per batch, then advances a compare-and-swapped manifest — the log's single linearization point. The flush loop is self-clocking, so there is no flush interval to tune, and an idle room issues no requests at all.
908
+
909
+ The trade is that a room has exactly one writer, elected with a lease. Route each room to one instance: this adapter replaces shared-database HA polling with room affinity rather than layering on top of it. Where you cannot promise that routing, see [Serverless sync on Vercel](#serverless-sync-on-vercel).
910
+
911
+ Every knob and its default is in [Object storage (no database)](#object-storage-no-database); the design, the failure modes and the known limits are in [`docs/object-storage.md`](./docs/object-storage.md).
912
+
913
+ ### Serverless sync on Vercel
914
+
915
+ Serverless functions break two assumptions a long-lived server gets for free. Both have a flag, and [`examples/kanban`](./examples/kanban/) is a deployed board that uses them.
916
+
917
+ **Any request can land on any instance,** so there is no single writer to elect. Drop the lease and let instances race on the manifest CAS instead:
918
+
919
+ ```ts
920
+ const storage = createObjectStorage({
921
+ store: { /* … */ },
922
+ roomId,
923
+ concurrency: "optimistic", // no lease; the loser of a CAS re-reads and retries
924
+ });
925
+ ```
926
+
927
+ That rests on the same guarantee the lease mode does — the CAS is what keeps the data correct, and the lease was only ever an optimization. What it costs is that in-memory state is no longer authoritative, because another invocation may have committed since this one last looked. Call `await storage.refresh()` — one manifest GET, `true` when something moved — before a read that must be current.
928
+
929
+ **A function cannot hold a WebSocket, and SSE is one-way.** The POST and the event stream are two separate invocations, so a reply to a POST can never reach a stream that process does not own. `serverless: true` returns those replies in the POST's own response instead:
930
+
931
+ ```ts
932
+ const transport = createSseServerTransport({ serverless: true });
933
+ const handler = new MessageHandler({ transport, serverId, db, allowAnonymous: true });
934
+ handler.setStorage(storage);
935
+
936
+ // POST /api/sync/messages — the replies this message produced
937
+ const messages = await transport.collectReplies(clientId, message, () =>
938
+ handler.whenIdle(clientId),
939
+ );
940
+ return Response.json({ messages });
941
+
942
+ // GET /api/sync/events — the stream carries only OTHER clients' changes
943
+ setInterval(async () => {
944
+ if (await storage.refresh()) await handler.pollRemoteChanges();
945
+ }, 250);
946
+ ```
947
+
948
+ `MessageHandler` is exported from `reflectdb/server`; the example drives it directly rather than through `createSyncServer`, because a serverless route needs `whenIdle` and `pollRemoteChanges`. Set the matching `serverless: true` on `createSseClientTransport`. Two more things a serverless deployment owns, both worked through in the kanban example: the client's session and subscriptions are rebuilt per invocation, because the `hello` and `sync_declare` went to a different process, and the stream instance must `bootstrap` so its result cache holds what the client is actually holding — the broadcast engine sends a diff against that cache, and an empty one makes every existing row look new.
949
+
950
+ ### htmx 4 bindings
951
+
952
+ `reflectdb/htmx` lets htmx drive the DOM while reflectdb owns the data. Bindings point at a `reflect:` action instead of a server route, so reads and writes resolve against the local store — optimistic, offline-capable, and re-rendered whenever a peer's change arrives. htmx 4 core ships no SSE or WebSocket support of its own; sync stays reflectdb's job.
953
+
954
+ ```ts
955
+ import htmx from "htmx.org";
956
+ import { createHtmxSync } from "reflectdb/htmx";
957
+
958
+ const reflect = createHtmxSync({
959
+ htmx,
960
+ url: "ws://localhost:3001/sync",
961
+ token,
962
+ tables: ["todos"],
963
+ });
964
+
965
+ reflect.view<Todo>("todos", ({ rows }) =>
966
+ rows
967
+ .map(
968
+ (todo) => `
969
+ <li>
970
+ <input type="checkbox" ${todo.done ? "checked" : ""}
971
+ hx-put="reflect:todos/${todo.id}"
972
+ hx-vals='{"done": "${!todo.done}"}'>
973
+ ${escapeHtml(todo.text)}
974
+ <button hx-delete="reflect:todos/${todo.id}">&times;</button>
975
+ </li>`,
976
+ )
977
+ .join(""),
978
+ );
979
+
980
+ // Form bodies arrive as strings — coerce before they reach the op log.
981
+ reflect.parse<Todo>("todos", (payload) => ({
982
+ ...payload,
983
+ done: payload.done === "true",
984
+ }));
985
+
986
+ await reflect.connect();
987
+ ```
988
+
989
+ ```html
990
+ <ul hx-get="reflect:todos" hx-trigger="load" hx-swap="innerMorph"></ul>
991
+
992
+ <form hx-post="reflect:todos">
993
+ <input name="text" required>
994
+ <input type="hidden" name="done" value="false">
995
+ <button>Add</button>
996
+ </form>
997
+ ```
998
+
999
+ The action grammar is REST-shaped, so the attributes read like ordinary htmx:
1000
+
1001
+ ```
1002
+ GET reflect:<table> → render the collection view
1003
+ GET reflect:<table>/:id → render one row
1004
+ POST reflect:<table> → insert (row id from the body's `id`, else generated)
1005
+ a body `id` names the row; it is not stored as a column
1006
+ PUT reflect:<table>/:id → update
1007
+ PATCH reflect:<table>/:id → update
1008
+ DELETE reflect:<table>/:id → delete
1009
+ ```
1010
+
1011
+ Writes answer `204 No Content`, so htmx swaps nothing where the write happened. The store change then re-renders every bound element a beat later — one render path whether the edit came from this tab, another tab, or the server.
1012
+
1013
+ Worth knowing:
1014
+
1015
+ - An element binds by making its first `reflect:` read, so give collection bindings `hx-trigger="load"`.
1016
+ - Use an inner swap (`innerHTML`, `innerMorph`) on collection bindings. `outerHTML` replaces the bound element itself, and a replacement still carrying `hx-trigger="load"` would re-request forever.
1017
+ - `hx-swap="innerMorph"` is usually what you want: htmx 4 morphs in place, so focus and caret position survive a re-render.
1018
+ - Query params reach the view for filtering (`reflect:todos?done=false`); they do not change the server subscription. Declare that with `tables`, or with `reflect.sync.sync(table, { params })`.
1019
+ - A row read whose row is missing answers `204`, leaving existing markup alone instead of blanking it.
1020
+ - A view returns a raw HTML string — escape interpolated values yourself.
1021
+ - A bad action or an unregistered view answers 4xx/5xx, which htmx swaps nowhere by default, and a `reflect:` request never reaches the network tab. The adapter `console.error`s every one of them so the failure is not silent.
1022
+ - Checkboxes, radios and `<option>`s are re-synced from the rendered markup after a store-driven re-render. Once a user clicks one, the HTML spec stops letting the `checked`/`selected` attribute drive the property, and htmx's morph only fixes that up for `value` — so a peer's change would otherwise leave a ticked box on a row the store says is open.
1023
+
863
1024
  ## Whiteboard + Pictionary example
864
1025
 
865
1026
  A complete React + Bun + Drizzle app that exercises most of reflectdb in one
@@ -931,6 +1092,81 @@ database. The included Fly.io config runs on one auto-stopping 256 MB Machine.
931
1092
  | Headless game rules and top-out reset tests | [`game.ts`](./examples/tetris/game.ts) / [`game.test.ts`](./examples/tetris/game.test.ts) |
932
1093
  | Bun SQLite persistence and restart tests | [`database.ts`](./examples/tetris/database.ts) / [`database.test.ts`](./examples/tetris/database.test.ts) |
933
1094
 
1095
+ ## Multiplayer kanban example
1096
+
1097
+ A shared board whose entire durable state is an S3-compatible bucket, deployed as
1098
+ Vercel functions: [`examples/kanban/`](./examples/kanban/). It is live at
1099
+ [reflectdb-kanban.vercel.app](https://reflectdb-kanban.vercel.app/). No Postgres,
1100
+ no SQLite, no volume, no Redis.
1101
+
1102
+ ```bash
1103
+ cd examples/kanban
1104
+ bun install
1105
+ KANBAN_LOCAL_DIR=.data vercel dev
1106
+ # open http://localhost:3000 in two tabs and drag a card
1107
+ ```
1108
+
1109
+ `KANBAN_LOCAL_DIR` swaps the bucket for a directory, so the example runs with no
1110
+ credentials — the filesystem driver has the same CAS semantics and the whole
1111
+ conformance suite runs against both. `vite` alone serves the UI but not `/api`,
1112
+ so the board will not connect without `vercel dev`.
1113
+
1114
+ The board is open to anyone with the link and `?board=<slug>` makes a new one.
1115
+ Every board resets to its starting cards on a five-minute window, claimed with a
1116
+ single `If-None-Match: *` write so exactly one of N racing invocations does the
1117
+ work — a cron job would not run on Vercel's Hobby tier, and would leave an idle
1118
+ board costing something.
1119
+
1120
+ | Pattern | Where |
1121
+ |---------|-------|
1122
+ | Object storage as the only durable store | `createObjectStorage` in [`lib/board.ts`](./examples/kanban/lib/board.ts) |
1123
+ | `concurrency: "optimistic"` — no lease, instances race on the manifest CAS | [`lib/board.ts`](./examples/kanban/lib/board.ts) |
1124
+ | Serverless SSE — replies returned from the POST that produced them | [`api/sync/messages.ts`](./examples/kanban/api/sync/messages.ts) |
1125
+ | `storage.refresh()` → `handler.pollRemoteChanges()` as the stream's poll loop | [`api/sync/events.ts`](./examples/kanban/api/sync/events.ts) |
1126
+ | Rebuilding a client's session, subscription and result cache per invocation | `restoreSubscription` in [`api/sync/events.ts`](./examples/kanban/api/sync/events.ts) |
1127
+ | Per-column merge so a rename and a drag on the same card both land | `cards` in [`lib/board.ts`](./examples/kanban/lib/board.ts) |
1128
+ | Payload validation with `MutationError` rather than coercion | `cards.mutate` in [`lib/board.ts`](./examples/kanban/lib/board.ts) |
1129
+ | Lazy periodic reset claimed with create-if-absent, applied through `applyServerOp` | [`lib/reset.ts`](./examples/kanban/lib/reset.ts) |
1130
+ | Fractional positioning for drag-and-drop ordering | [`schema.ts`](./examples/kanban/schema.ts) |
1131
+ | Bundling the API routes so Vercel's Node builder never sees a `.ts` specifier | [`scripts/build-kanban.ts`](./scripts/build-kanban.ts) / [`vercel.json`](./vercel.json) |
1132
+
1133
+ Deploy from the repository root rather than the example directory — the example
1134
+ imports reflectdb from `src/`, and the root `vercel.json` runs a build that
1135
+ bundles the two API routes into `.vercel/output` itself. Pushing to `main` does
1136
+ it; the demo's Vercel project is linked to this repository at the repository
1137
+ root. The variables to set, and the one extra `storage.init()` step MinIO needs,
1138
+ are in [`examples/kanban/README.md`](./examples/kanban/README.md).
1139
+
1140
+ ## htmx todos example
1141
+
1142
+ A todo list where htmx 4 owns the DOM and reflectdb owns the data:
1143
+ [`examples/htmx-todos/`](./examples/htmx-todos/). It is live at
1144
+ [reflectdb-htmx-todos.fly.dev](https://reflectdb-htmx-todos.fly.dev/). The
1145
+ server renders no HTML — every fragment is produced in the browser from the
1146
+ local store. The deployed list resets to its seed rows every minute.
1147
+
1148
+ ```bash
1149
+ cd examples/htmx-todos
1150
+ bun install
1151
+ bun run dev
1152
+ # open http://localhost:3005 in two tabs
1153
+ ```
1154
+
1155
+ The server takes the first free port at or above `PORT` (default 3005), so it
1156
+ does not collide with the other examples.
1157
+
1158
+ | Pattern | Where |
1159
+ |---------|-------|
1160
+ | `reflect:` actions on ordinary htmx attributes | [`index.html`](./examples/htmx-todos/index.html) |
1161
+ | One view function rendering the whole list from local rows | [`client.ts`](./examples/htmx-todos/client.ts) |
1162
+ | `parse` building a patch, so a checkbox toggle keeps the text it never sent | [`client.ts`](./examples/htmx-todos/client.ts) |
1163
+ | A counter riding along as an `hx-swap-oob` element | [`client.ts`](./examples/htmx-todos/client.ts) |
1164
+ | Filters as query params, kept across peers' edits | [`index.html`](./examples/htmx-todos/index.html) |
1165
+ | A periodic reset routed through `applyServerOp`, so it reaches every open tab | [`server.ts`](./examples/htmx-todos/server.ts) |
1166
+
1167
+ Stop the server and keep typing: writes land in the DOM immediately, the badge
1168
+ counts them, and they drain on reconnect.
1169
+
934
1170
  ## Architecture
935
1171
 
936
1172
  ```
@@ -982,6 +1218,7 @@ database. The included Fly.io config runs on one auto-stopping 256 MB Machine.
982
1218
  │ • in-memory (default) │ │ │
983
1219
  │ • sqlite (bun:sqlite) │ │ │
984
1220
  │ • postgres (any pg-compatible) │ │ │
1221
+ │ • object (any S3-compatible) │ │ │
985
1222
  └───────────────────────────────────┘ └──────────────────────────────────────┘
986
1223
 
987
1224
  ┌─────────────────────────────────────────────────────────────────────────────┐
@@ -995,14 +1232,60 @@ database. The included Fly.io config runs on one auto-stopping 256 MB Machine.
995
1232
  └─────────────────────────────────────────────────────────────────────────────┘
996
1233
 
997
1234
  ┌─────────────────────────────────────────────────────────────────────────────┐
998
- FRAMEWORK BINDINGS (react/, svelte/, vanilla/)
1235
+ FRAMEWORK BINDINGS (react/, svelte/, vanilla/, htmx/)
999
1236
  │ │
1000
1237
  │ • createSyncReact(queries) → typed hooks + <SyncProvider> │
1001
1238
  │ • createSyncSvelte(queries) → typed Svelte stores │
1002
1239
  │ • createSyncVanilla(queries) → typed callback API │
1240
+ │ • createSyncHtmx(queries) → typed views behind reflect: attributes │
1003
1241
  └─────────────────────────────────────────────────────────────────────────────┘
1004
1242
  ```
1005
1243
 
1244
+ ### Object storage as the durable store
1245
+
1246
+ `createObjectStorage` replaces that op-log box with an S3-compatible bucket, and
1247
+ nothing else in the diagram changes. The bucket is durability, never the read
1248
+ path:
1249
+
1250
+ ```
1251
+ ┌──────────────────────────────────────┐
1252
+ clients ───────▶ │ writer instance (one per room) │
1253
+ │ │
1254
+ │ in-memory authoritative state │ ◀── every read
1255
+ │ rows · per-column HLCs · op ring │ lands here
1256
+ │ reserveOp set · meta │
1257
+ │ │
1258
+ │ write buffer ──▶ group commit │
1259
+ └───────────────────┬──────────────────┘
1260
+ │ one PUT per batch,
1261
+ │ then one CAS
1262
+
1263
+ ┌──────────────────────────────────────┐
1264
+ │ object store — S3 · R2 · Tigris · │
1265
+ │ MinIO · GCS │
1266
+ │ │
1267
+ │ _lease writer election │
1268
+ │ _manifest the CAS'd commit point │
1269
+ │ wal/ immutable op batches │
1270
+ │ snap/ materialized rows │
1271
+ └──────────────────────────────────────┘
1272
+ ```
1273
+
1274
+ Reads (`getRow`, `getRows`, `getOpsSince`, `reserveOp`) hit memory and never
1275
+ touch the network. A write mutates memory and appends to a buffer; the buffer
1276
+ flushes as one object per batch, and the commit is the compare-and-swap that adds
1277
+ that segment to `_manifest`. Boot is one manifest GET, the newest snapshot, and
1278
+ the segments the manifest still lists.
1279
+
1280
+ Only `_lease` and `_manifest` are ever overwritten, and only via CAS. Everything
1281
+ else is write-once, which is what makes concurrent readers safe — and what lets
1282
+ `concurrency: "optimistic"` drop the lease entirely on a platform that cannot
1283
+ route a room to one instance.
1284
+
1285
+ Full design, provider compatibility, durability model and known limits:
1286
+ [`docs/object-storage.md`](./docs/object-storage.md).
1287
+
1288
+
1006
1289
  ## Core Concepts
1007
1290
 
1008
1291
  ### Hybrid Logical Clocks
@@ -1073,7 +1356,7 @@ reflectdb keeps its own store alongside yours, and it helps to know which one an
1073
1356
 
1074
1357
  | Read | Source |
1075
1358
  |------|--------|
1076
- | Snapshots (`bootstrap`, `resume`) | **Your database**, via the `query` callback |
1359
+ | Snapshots (`bootstrap`, `resume`) | **Your database**, via the `query` callback — which only runs when `db` was passed to `createSyncServer` |
1077
1360
  | Broadcast deltas | **Your database**, diffed against a per-client cached result set |
1078
1361
  | Conflict resolution (`lww` / `merge` / `server` / custom) | **reflectdb's mirror** — a JSONB row store plus per-column HLCs |
1079
1362
  | Which tables changed since an HLC | **reflectdb's op log** |
@@ -1168,6 +1451,42 @@ See [Server-driven game loops](#server-driven-game-loops) for `interval` / `lock
1168
1451
 
1169
1452
  `createServer()` is the lower-level untyped variant — use it only if you need to register queries dynamically or don't have a `defineSyncQueries` map.
1170
1453
 
1454
+ ### `reflectdb/server/storage/object`
1455
+
1456
+ ```ts
1457
+ import {
1458
+ createObjectStorage,
1459
+ createS3Driver, createFilesystemDriver, createMemoryDriver,
1460
+ PreconditionFailedError, BackpressureError, NotWriterError, MemoryLimitExceededError,
1461
+ IncompleteStateError, roomPrefix,
1462
+ } from "reflectdb/server/storage/object";
1463
+ ```
1464
+
1465
+ **`createObjectStorage(config)`** — a storage adapter backed by an S3-compatible bucket. It satisfies the same `StorageAdapter` contract as the SQLite and Postgres adapters, plus a lifecycle and observability surface the design needs:
1466
+
1467
+ | Member | Purpose |
1468
+ |--------|---------|
1469
+ | `init()` | Boot the room — manifest, snapshot, WAL replay. Idempotent, and implied by the first call to anything else; call it explicitly to surface boot failures at startup rather than on the first query. On MinIO it also seeds `_lease` and `_manifest`, which makes it a **deploy step** rather than something N servers race. |
1470
+ | `refresh()` | Fold in whatever another instance has committed. Returns `true` when the room actually moved. Costs one GET, and nothing further when `commitSeq` has not changed. Only meaningful under `concurrency: "optimistic"`. |
1471
+ | `flush()` | Resolve once everything buffered is durable. |
1472
+ | `close()` | Stop accepting writes, flush, stop the flush loop, release the lease. Every step is bounded by `shutdownFlushMs`, so a hung store cannot hold a `SIGTERM` open. |
1473
+ | `health` | `"healthy"`, `"degraded"` or `"unavailable"`, with `onHealthChange(cb)` to observe transitions. |
1474
+ | `durableHlc` | Highest HLC known to be on the store, with `onDurable(cb)` fired per batch. |
1475
+
1476
+ Errors are typed so a caller can tell "someone else moved first" from a transport failure: `PreconditionFailedError` (a CAS lost), `BackpressureError` (the write buffer hit `batch.maxBufferBytes`), `NotWriterError` (this instance lost or never held the lease), `MemoryLimitExceededError` (room state exceeded `memory.maxRoomBytes`), `IncompleteStateError` (the manifest names an object the store does not have — the room refuses to boot rather than present the loss as an empty room).
1477
+
1478
+ `roomPrefix(roomId)` returns the key prefix a room writes under, for tooling that has to address those keys from outside the adapter — an admin wipe, or a disposable room clearing itself after an `IncompleteStateError`.
1479
+
1480
+ Pass `store` and the S3 driver is built for you; pass `driver` to supply one directly:
1481
+
1482
+ | Driver | Signature | Use |
1483
+ |--------|-----------|-----|
1484
+ | `createS3Driver` | `(config: StoreConfig)` | Any S3-compatible bucket. SigV4 over `fetch` with WebCrypto — no `node:crypto`, so the browser build stays clean. |
1485
+ | `createFilesystemDriver` | `(rootDir: string)` | Local development and tests with no network. Same CAS semantics via atomic rename, but a filesystem cannot make compare-then-rename atomic across processes — single process only. |
1486
+ | `createMemoryDriver` | `(options?)` | Tests, with fault injection (412 storms, 500s, latency) and `casWildcard: false` to reproduce MinIO. |
1487
+
1488
+ See [Object storage (no database)](#object-storage-no-database) for the configuration, and [`docs/object-storage.md`](./docs/object-storage.md) for the design.
1489
+
1171
1490
  ### `reflectdb/client`
1172
1491
 
1173
1492
  ```ts
@@ -1272,19 +1591,40 @@ store.onError((e) => …);
1272
1591
  ```ts
1273
1592
  import { createSync, createSyncVanilla, createBrowserWsTransport } from "reflectdb/vanilla";
1274
1593
 
1275
- const sync = createSync({ url, token, tables: ["notes"] });
1594
+ const sync = createSync({ url, token, tables: ["notes"], onError: (e) => … });
1276
1595
  const notes = sync.sync<Note>("notes");
1277
1596
 
1278
- notes.onChange((rows) => render(rows));
1597
+ notes.onChange(() => render(notes.getRows()));
1279
1598
  notes.insert(id, { title: "…" });
1280
1599
 
1281
1600
  sync.onStateChange((s) => …);
1282
1601
  sync.onPendingChange((n) => …);
1283
- sync.onError((e) => …);
1284
1602
  sync.connect();
1285
1603
  ```
1286
1604
 
1287
- Also supports ephemeral: `sync.sendEphemeral({ key, userId, data })`, `sync.onEphemeral(key, listener)`.
1605
+ Also supports ephemeral: `sync.ephemeral({ key, userId, ttlMs })` returns a binding with `broadcast(data)`, `getEvents()` and `onChange(listener)`.
1606
+
1607
+ ### `reflectdb/htmx`
1608
+
1609
+ ```ts
1610
+ import { createHtmxSync, createSyncHtmx } from "reflectdb/htmx";
1611
+
1612
+ const reflect = createHtmxSync({ htmx, url, token, tables: ["todos"] });
1613
+
1614
+ reflect.view<Todo>("todos", ({ rows, rowId, params }) => "<li>…</li>");
1615
+ reflect.parse<Todo>("todos", (payload) => ({ …payload, done: payload.done === "true" }));
1616
+
1617
+ reflect.install(); // attach the reflect: shim (connect() does this too)
1618
+ reflect.uninstall(); // detach it and drop every element binding
1619
+ reflect.refresh("todos"); // re-render bound elements on demand
1620
+ reflect.sync; // the underlying VanillaSync, for anything attributes cannot express
1621
+
1622
+ await reflect.connect();
1623
+ ```
1624
+
1625
+ Requires htmx 4 — import the instance yourself and pass it in. The adapter listens for `htmx:config:request` and sets `ctx.fetch` on `reflect:` actions, so htmx still performs the swap, OOB handling, settling and history it would for a server response.
1626
+
1627
+ `createSyncHtmx<typeof queries>()` returns the same factory with `view`, `parse` and `refresh` narrowed to your schema's table names and row types. See [htmx 4 bindings](#htmx-4-bindings) for the action grammar.
1288
1628
 
1289
1629
  ### `reflectdb/transport/*`
1290
1630
 
@@ -1502,11 +1842,105 @@ At boot the client restores its persisted subscriptions first and hydrates only
1502
1842
  | _(none)_ | omit `storage` | In-memory op log; ephemeral, single node |
1503
1843
  | `createSqliteStorage({ path?, db? })` | `reflectdb/server` | Single server, development, embedded — **Bun only** |
1504
1844
  | `createPostgresStorage(poolOrConfig)` | `reflectdb/server` | Multi-server HA, production |
1845
+ | `createObjectStorage({ store, roomId })` | `reflectdb/server/storage/object` | S3-compatible object storage as the only durable store — no database |
1505
1846
 
1506
1847
  `createPostgresStorage` accepts any object with `query(text, values) => { rows }` — `pg.Pool`, `pg.Client`, `@neondatabase/serverless`, etc. Optional config: `{ client, tablePrefix: "_reflectdb" }`.
1507
1848
 
1508
1849
  `createSqliteStorage` is backed by `bun:sqlite` and is resolved lazily, so importing `reflectdb/server` on Node is fine — only calling `createSqliteStorage` there throws. On Node, use `createPostgresStorage` (or omit `storage` for the in-memory op log).
1509
1850
 
1851
+ #### Object storage (no database)
1852
+
1853
+ `createObjectStorage` runs a room with S3-compatible object storage as the only
1854
+ durable store. Works with AWS S3, Cloudflare R2, Tigris, MinIO and GCS.
1855
+
1856
+ ```ts
1857
+ import { createObjectStorage } from "reflectdb/server/storage/object";
1858
+
1859
+ const storage = createObjectStorage({
1860
+ store: {
1861
+ provider: "tigris", // or "aws" | "r2" | "minio" | "gcs"
1862
+ bucket: "my-app",
1863
+ credentials: {
1864
+ keyId: process.env.AWS_ACCESS_KEY_ID!,
1865
+ secret: process.env.AWS_SECRET_ACCESS_KEY!,
1866
+ },
1867
+ },
1868
+ roomId: "board-42",
1869
+ });
1870
+
1871
+ const server = createSyncServer({ queries, db, transport, storage });
1872
+
1873
+ // Flush and release the lease so the next machine takes over immediately.
1874
+ process.on("SIGTERM", () => storage.close().then(() => process.exit(0)));
1875
+ ```
1876
+
1877
+ Row state is authoritative **in memory** and the object store is durability
1878
+ only, so reads never touch the network. Writes group-commit: one PUT per batch,
1879
+ self-clocking, with no flush interval to tune. An idle room issues zero requests.
1880
+
1881
+ Because state is in memory and a room has exactly one writer, **route each room
1882
+ to one instance** — this adapter replaces shared-database HA polling with room
1883
+ affinity rather than layering on top of it.
1884
+
1885
+ Where you cannot promise that routing — serverless functions, where any request
1886
+ lands on any instance — set `concurrency: "optimistic"`. It drops the lease and
1887
+ lets instances race on the manifest CAS instead, with the loser re-reading and
1888
+ retrying. That rests on the same guarantee the lease mode does: the CAS is what
1889
+ keeps the data correct, and the lease was only ever an optimization. In exchange,
1890
+ in-memory state is no longer authoritative, so call `await storage.refresh()`
1891
+ (one manifest GET) before a read that must be current.
1892
+
1893
+ Defaults are correct rather than fast: `durability: "durable"` acks a write only
1894
+ once it is on the store. `"buffered"` acks earlier and is lossy on crash until
1895
+ the durable-watermark protocol lands.
1896
+
1897
+ The `SIGTERM` handler above matters — without it a deploy drops whatever is
1898
+ buffered, and the next writer waits out `lease.ttlMs` before taking the room.
1899
+
1900
+ MinIO needs a one-time `await storage.init()` as a **deploy step** (it rejects
1901
+ the `If-None-Match: *` wildcard, so the room has to be seeded); on every other
1902
+ provider `init()` is a no-op.
1903
+
1904
+ `store` describes the bucket. `provider` fills in `endpoint` and `urlStyle`, so most deployments set three fields and nothing else:
1905
+
1906
+ | Field | Required | Notes |
1907
+ |-------|----------|-------|
1908
+ | `bucket` | yes | |
1909
+ | `credentials` | yes | `{ keyId, secret, sessionToken? }`. |
1910
+ | `provider` | no | `"aws"`, `"r2"`, `"tigris"`, `"minio"` or `"gcs"` — a preset for `endpoint`, `region` and `urlStyle`. |
1911
+ | `accountId` | for R2 | Derives `https://<id>.r2.cloudflarestorage.com`. |
1912
+ | `endpoint` / `region` / `urlStyle` | no | Override the preset. MinIO has no default endpoint, and a Fly-provisioned Tigris bucket uses `fly.storage.tigris.dev` rather than the preset's `t3.storage.dev`. |
1913
+ | `prefix` | no | Key prefix, so one bucket holds many apps. Keys are `<prefix>/rooms/<roomId>/…`. |
1914
+
1915
+ Pass `driver` instead of `store` to supply a driver directly — `createFilesystemDriver(dir)` for local development with no credentials, `createMemoryDriver()` for tests.
1916
+
1917
+ Everything else, with its default:
1918
+
1919
+ | Option | Default | Purpose |
1920
+ |--------|---------|---------|
1921
+ | `roomId` | — | Required. One room per adapter; it is the key prefix and the routing key. |
1922
+ | `writerId` | random | Identifies this writer in the lease. Set it to make ownership legible in logs. |
1923
+ | `durability` | `"durable"` | Ack after the manifest CAS. `"buffered"` acks on memory apply and is lossy on crash until the durable-watermark protocol lands. |
1924
+ | `concurrency` | `"single-writer"` | `"optimistic"` drops the lease where a room cannot be routed to one instance. See [Serverless sync on Vercel](#serverless-sync-on-vercel). |
1925
+ | `retentionMs` | `Infinity` | How long accepted ops are kept. |
1926
+ | `batch.maxBytes` | 4 MiB | Largest single WAL segment. |
1927
+ | `batch.minLingerMs` | `5` | Coalesces ops arriving in the same event-loop tick. Not a flush interval — there is none, the flush loop is self-clocking. |
1928
+ | `batch.maxBufferBytes` | 64 MiB | Buffer ceiling before the backpressure policy fires. |
1929
+ | `batch.onBackpressure` | `"reject"` | Throw `BackpressureError` so backpressure reaches the client. `"degrade"` keeps accepting, stops promising durability, and flips `health`. |
1930
+ | `compaction.afterSegments` | `200` | Snapshot once the manifest lists this many segments. Boot costs one GET per listed segment, so lower it for rooms that are read cold more often than they are written. |
1931
+ | `compaction.afterBytes` | 64 MiB | Or this many bytes, whichever comes first. |
1932
+ | `compaction.gcGraceMs` | `3_600_000` | Delay before deleting superseded segments, so a reader holding the old manifest does not 404. |
1933
+ | `lease.ttlMs` / `lease.renewMs` | `300_000` / `120_000` | Long on purpose. `close()` releases the lease, so the TTL bounds only *unclean* failover — and a long one is what keeps an idle room free. |
1934
+ | `lease.mode` | `"on-write"` | A room holding connected clients but taking no writes renews nothing. |
1935
+ | `memory.maxTotalBytes` / `memory.maxRoomBytes` | `Infinity` | Global across rooms, then per room. State is authoritative in memory, so this is a cliff rather than a slope — set it and the ceiling arrives as `MemoryLimitExceededError` instead of an OOM. |
1936
+ | `memory.onExceeded` | `"reject"` | `"evict"` and `"spill"` are accepted as configuration but throw at the limit; neither is implemented yet. |
1937
+ | `memory.idleEvictMs` | `300_000` | Zero-client rooms flush, release the lease and drop their state. |
1938
+ | `shutdownFlushMs` | `5000` | Bounds every step of `close()`, so a hung store cannot hold a `SIGTERM` open. |
1939
+ | `onDurable` / `onHealthChange` | — | Callbacks. The same values are readable as `storage.durableHlc` and `storage.health`. |
1940
+
1941
+ Full design, the configuration reference and the known limits:
1942
+ [`docs/object-storage.md`](./docs/object-storage.md).
1943
+
1510
1944
  #### Client
1511
1945
 
1512
1946
  | Adapter | Import | Best for |
@@ -1592,11 +2026,42 @@ Returns a `ServerTransport` plus `handleOpen`, `handleMessage`, `handleClose`, `
1592
2026
  ```ts
1593
2027
  createSseServerTransport({
1594
2028
  replayBufferSize: 256, // Last-Event-ID replay window
2029
+ serverless: false, // see below
1595
2030
  });
1596
2031
  ```
1597
2032
 
1598
2033
  Two endpoints to wire: `GET /sync/events/:clientId` (SSE stream) and `POST /sync/messages/:clientId` (client → server).
1599
2034
 
2035
+ ##### Serverless SSE
2036
+
2037
+ SSE is server→client only: normally a client POSTs a message and every reply —
2038
+ `hello_ack`, snapshots, op acks — comes back down the held stream. That works
2039
+ only while the POST and the stream are handled by the **same process**.
2040
+
2041
+ On Vercel, Lambda or Workers they are two separate invocations, so those replies
2042
+ would be enqueued onto a stream the POST's process does not have, and the client
2043
+ hangs at the handshake. Set `serverless: true` on both halves:
2044
+
2045
+ ```ts
2046
+ // server: return the replies the POST produced
2047
+ const messages = await transport.collectReplies(clientId, message, () =>
2048
+ handler.whenIdle(clientId),
2049
+ );
2050
+ return Response.json({ messages });
2051
+
2052
+ // client
2053
+ createSseClientTransport({ eventUrl, messageUrl, serverless: true });
2054
+ ```
2055
+
2056
+ The stream is then left doing the one thing it can: pushing *other* clients'
2057
+ changes. Leave it off for a single-process server — replies stream normally
2058
+ there, and turning it on would deliver each one twice.
2059
+
2060
+ Two things a serverless deployment must also handle, both shown in
2061
+ [`examples/kanban`](./examples/kanban/): the session is rebuilt per invocation
2062
+ (nothing remembers that this client subscribed), and the stream instance has to
2063
+ poll storage to notice writes made elsewhere.
2064
+
1600
2065
  #### HTTP long-polling
1601
2066
 
1602
2067
  ```ts
@@ -1663,6 +2128,7 @@ reference:
1663
2128
  | `og/index.html` | `public/og.png` | 1200x630 at 2x | reflectdb.dev |
1664
2129
  | `og/tetris.html` | `public/og-tetris.png` | 1200x630 at 2x | the Tetris demo — served from reflectdb.dev, since the Fly Machine sleeps |
1665
2130
  | `og/whiteboard.html` | `public/og-whiteboard.png` | 1200x630 at 2x | the whiteboard demo, served from reflectdb.dev for the same reason |
2131
+ | `og/kanban.html` | `public/og-kanban.png` | 1200x630 at 2x | the kanban demo, so all four cards regenerate together |
1666
2132
  | `og/github.html` | `public/og-github.png` | 1280x640 at 2x | this repository's social preview, uploaded by hand under Settings → Social preview |
1667
2133
 
1668
2134
  Edit the HTML, not the PNGs. 1200x630 is the one ratio X, Facebook, LinkedIn,
@@ -1697,12 +2163,15 @@ src/
1697
2163
  │ │ broadcast-engine, result-cache, eager-buffer,
1698
2164
  │ │ compaction-manager, replay-detector, ephemeral-manager
1699
2165
  │ └── storage/ SQLite + Postgres adapters
2166
+ │ └── object/ S3-compatible object storage — manifest CAS, WAL,
2167
+ │ snapshots, memory / filesystem / S3 drivers
1700
2168
  ├── client/ sync-client, store, ops, typed-client
1701
2169
  │ └── storage/ memory + IndexedDB adapters
1702
2170
  ├── transport/ WebSocket (runtime-agnostic + Bun.serve), SSE, polling
1703
2171
  ├── react/ <SyncProvider>, hooks, typed factory
1704
2172
  ├── svelte/ createSyncStore, typed factory
1705
- └── vanilla/ createSync, typed factory
2173
+ ├── vanilla/ createSync, typed factory
2174
+ └── htmx/ createHtmxSync, reflect: action router, typed factory
1706
2175
  ```
1707
2176
 
1708
2177
  ### Tech stack