reflectdb 0.1.3 → 0.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +483 -17
- package/dist/cjs/htmx/index.cjs +1720 -0
- package/dist/cjs/htmx/index.d.cts +760 -0
- package/dist/cjs/server/index.cjs +12 -0
- package/dist/cjs/server/index.d.cts +14 -0
- package/dist/cjs/server/storage/object/index.cjs +2284 -0
- package/dist/cjs/server/storage/object/index.d.cts +578 -0
- package/dist/cjs/transport/sse.cjs +52 -1
- package/dist/cjs/transport/sse.d.cts +34 -0
- package/dist/htmx/index.d.ts +760 -0
- package/dist/htmx/index.js +297 -0
- package/dist/server/drizzle.js +2 -1
- package/dist/server/index.d.ts +14 -0
- package/dist/server/index.js +16 -2
- package/dist/server/storage/object/index.d.ts +578 -0
- package/dist/server/storage/object/index.js +2232 -0
- package/dist/shared/{esm-77ndhmpv.js → esm-5ahpq25j.js} +4 -14
- package/dist/shared/esm-dcs8qa5n.js +263 -0
- package/dist/shared/esm-f11s9zpb.js +15 -0
- package/dist/transport/sse.d.ts +34 -0
- package/dist/transport/sse.js +52 -1
- package/dist/vanilla/index.js +4 -260
- package/package.json +23 -1
package/README.md
CHANGED
|
@@ -18,7 +18,7 @@ You bring your own types and your own database. reflectdb handles the protocol,
|
|
|
18
18
|
└──────────────┘ └──────────────┘ └──────────────┘
|
|
19
19
|
▲ │
|
|
20
20
|
│ offline ▼
|
|
21
|
-
└────── IndexedDB ───── op log (in-memory / SQLite / Postgres)
|
|
21
|
+
└────── IndexedDB ───── op log (in-memory / SQLite / Postgres / S3)
|
|
22
22
|
```
|
|
23
23
|
|
|
24
24
|
## Table of Contents
|
|
@@ -45,8 +45,13 @@ You bring your own types and your own database. reflectdb handles the protocol,
|
|
|
45
45
|
- [Windowed sync and pagination](#windowed-sync-and-pagination)
|
|
46
46
|
- [Auto-generated REST API](#auto-generated-rest-api)
|
|
47
47
|
- [High availability with Postgres](#high-availability-with-postgres)
|
|
48
|
+
- [Sync with no database at all](#sync-with-no-database-at-all)
|
|
49
|
+
- [Serverless sync on Vercel](#serverless-sync-on-vercel)
|
|
50
|
+
- [htmx 4 bindings](#htmx-4-bindings)
|
|
48
51
|
- [Whiteboard + Pictionary example](#whiteboard--pictionary-example)
|
|
49
52
|
- [Infinite Tetris example](#infinite-tetris-example)
|
|
53
|
+
- [Multiplayer kanban example](#multiplayer-kanban-example)
|
|
54
|
+
- [htmx todos example](#htmx-todos-example)
|
|
50
55
|
- [Architecture](#architecture)
|
|
51
56
|
- [Core Concepts](#core-concepts)
|
|
52
57
|
- [Hybrid Logical Clocks](#hybrid-logical-clocks)
|
|
@@ -57,10 +62,12 @@ You bring your own types and your own database. reflectdb handles the protocol,
|
|
|
57
62
|
- [API Reference](#api-reference)
|
|
58
63
|
- [`reflectdb/core`](#reflectdbcore)
|
|
59
64
|
- [`reflectdb/server`](#reflectdbserver)
|
|
65
|
+
- [`reflectdb/server/storage/object`](#reflectdbserverstorageobject)
|
|
60
66
|
- [`reflectdb/client`](#reflectdbclient)
|
|
61
67
|
- [`reflectdb/react`](#reflectdbreact)
|
|
62
68
|
- [`reflectdb/svelte`](#reflectdbsvelte)
|
|
63
69
|
- [`reflectdb/vanilla`](#reflectdbvanilla)
|
|
70
|
+
- [`reflectdb/htmx`](#reflectdbhtmx)
|
|
64
71
|
- [`reflectdb/transport/*`](#reflectdbtransport)
|
|
65
72
|
- [Configuration Reference](#configuration-reference)
|
|
66
73
|
- [Query definition](#query-definition)
|
|
@@ -81,11 +88,16 @@ You bring your own types and your own database. reflectdb handles the protocol,
|
|
|
81
88
|
| Demo | Try it | What it demonstrates |
|
|
82
89
|
|------|--------|----------------------|
|
|
83
90
|
| **Infinite multiplayer Tetris** | [Play live](https://reflectdb-tetris.fly.dev/) · [source](./examples/tetris/) | Optimistic input prediction, server reconciliation and gravity, a live leaderboard, per-player progression, and Bun SQLite persistence in one perpetual game. Open two tabs to add another player. |
|
|
91
|
+
| **Multiplayer kanban** | [Open the board](https://reflectdb-kanban.vercel.app/) · [source](./examples/kanban/) | A board whose entire durable state is an S3 bucket — no Postgres, no SQLite, no volume — running on Vercel functions. Shows leaseless optimistic concurrency and serverless SSE. Open two tabs and drag a card. |
|
|
92
|
+
| **htmx 4 todos** | [Open it](https://reflectdb-htmx-todos.fly.dev/) · [source](./examples/htmx-todos/) | htmx owns the DOM, reflectdb owns the data. Attributes point at `reflect:` actions instead of server routes, so the server renders no HTML at all — every fragment comes from the local store. Open two tabs, or stop typing and go offline. The list resets to its seed rows every minute. |
|
|
84
93
|
| **Collaborative whiteboard** | [Draw live](https://reflectdb-whiteboard.fly.dev/) · [source](./examples/whiteboard/) | Freeform drawing by default, optional Pictionary rounds, guest-authenticated rooms, ephemeral cursors, chat, presence, and per-user query results. Rooms and everything in them are deleted 30 minutes after they are created. Open two tabs to draw with yourself. |
|
|
85
94
|
|
|
86
|
-
|
|
87
|
-
after an idle period may take a
|
|
88
|
-
|
|
95
|
+
Tetris, the whiteboard and the htmx todos each run on one auto-stopping Fly
|
|
96
|
+
Machine with no volume, so the first load after an idle period may take a
|
|
97
|
+
moment. The kanban board has no machine to wake — it is Vercel functions and a
|
|
98
|
+
bucket — but every board resets on a five-minute window, and the htmx todos
|
|
99
|
+
reset every minute. All four keep their data intentionally ephemeral across
|
|
100
|
+
deployments and Machine replacement.
|
|
89
101
|
|
|
90
102
|
## Why reflectdb
|
|
91
103
|
|
|
@@ -109,7 +121,7 @@ No code generation. No glue layer. No second source of truth.
|
|
|
109
121
|
Optional bits (use what you want):
|
|
110
122
|
|
|
111
123
|
- **Drizzle ORM** — if you point `table` at a Drizzle table, row types are auto-inferred.
|
|
112
|
-
- **Server op log storage** — SQLite (
|
|
124
|
+
- **Server op log storage** — SQLite (single-node), Postgres (HA), or an S3-compatible bucket (no database at all). Omit it and the op log is in-memory.
|
|
113
125
|
- **React / Svelte bindings** — use the core client directly if you prefer.
|
|
114
126
|
|
|
115
127
|
## Features
|
|
@@ -119,13 +131,15 @@ Optional bits (use what you want):
|
|
|
119
131
|
- **End-to-end type safety** — schema defines row types, query params, writable fields, and which columns the server owns
|
|
120
132
|
- **Per-row and per-column conflict resolution** — `lww`, `merge`, `server`, or a custom resolver
|
|
121
133
|
- **Causal ordering** via hybrid logical clocks (HLC) — no dependence on synchronized wall clocks
|
|
122
|
-
- **Pluggable storage** — in-memory, SQLite, or
|
|
134
|
+
- **Pluggable storage** — in-memory, SQLite, Postgres, or S3-compatible object storage for the server op log; memory or IndexedDB for the browser
|
|
123
135
|
- **Auto-generated REST** — `server.rest()` turns your schema into CRUD endpoints that broadcast deltas
|
|
124
136
|
- **Room-based access control** — scope clients to `org/:orgId` or arbitrary patterns
|
|
125
137
|
- **Rate limiting** — global and per-table, fail-open
|
|
126
138
|
- **Op log compaction** — configurable retention for old accepted ops
|
|
127
139
|
- **High availability** — shared Postgres + optional cross-instance polling
|
|
128
|
-
- **
|
|
140
|
+
- **No database at all** — `createObjectStorage` runs a room with an S3-compatible bucket as the only durable store, group-committing one object per batch
|
|
141
|
+
- **Runs serverless** — SSE in `serverless` mode answers each POST with the replies it produced, so sync works on Vercel, Lambda or Workers
|
|
142
|
+
- **Framework bindings** — React hooks, Svelte stores, a vanilla-JS helper, and htmx 4 attribute bindings; the core client works anywhere
|
|
129
143
|
- **Ephemeral channels** — presence, cursors, typing indicators that never touch the op log, with a room snapshot on join and a pluggable adapter (Redis included) so presence spans a fleet
|
|
130
144
|
- **Typed presence** — `presence()` in the schema, `usePresence()` in the component, key derived for you
|
|
131
145
|
- **Read-only views** — `view()` entries that recompute on their dependencies and reject writes at both levels
|
|
@@ -141,6 +155,7 @@ Optional bits (use what you want):
|
|
|
141
155
|
- Admin tools that should "just update" when someone else changes a row
|
|
142
156
|
- Field-service or retail apps on spotty networks
|
|
143
157
|
- Games or canvases with presence indicators and live cursors
|
|
158
|
+
- Serverless deployments with no database to attach and no machine to keep warm
|
|
144
159
|
|
|
145
160
|
## Installation
|
|
146
161
|
|
|
@@ -165,13 +180,14 @@ Peer dependencies are all optional:
|
|
|
165
180
|
|
|
166
181
|
```bash
|
|
167
182
|
bun add react # for reflectdb/react
|
|
183
|
+
bun add htmx.org@^4 # for reflectdb/htmx
|
|
168
184
|
bun add drizzle-orm # if you want auto-inferred row types from Drizzle tables
|
|
169
185
|
# Svelte + vanilla have no peer deps
|
|
170
186
|
```
|
|
171
187
|
|
|
172
188
|
## Quick Start
|
|
173
189
|
|
|
174
|
-
A complete sync server in ~30 lines. No ORM, no database — just plain types and an in-memory Map
|
|
190
|
+
A complete sync server in ~30 lines. No ORM, no database — just plain types and an in-memory Map, handed to reflectdb as `db`.
|
|
175
191
|
|
|
176
192
|
### 1. Define your schema
|
|
177
193
|
|
|
@@ -207,7 +223,10 @@ import { queries, type Todo } from "./schema";
|
|
|
207
223
|
const todos = new Map<string, Todo>();
|
|
208
224
|
const transport = createWsServerTransport();
|
|
209
225
|
|
|
210
|
-
|
|
226
|
+
// `db` is whatever holds your data — an ORM handle, a pool, or a plain Map.
|
|
227
|
+
// It is not optional: reflectdb only runs a `query` callback when it has a
|
|
228
|
+
// `db` to pass it, so leaving it out makes every snapshot come back empty.
|
|
229
|
+
const server = createSyncServer({ queries, db: todos, transport, serverId: "s1" });
|
|
211
230
|
|
|
212
231
|
server.auth(async (req) => {
|
|
213
232
|
// validate req.headers.get("authorization")
|
|
@@ -215,7 +234,7 @@ server.auth(async (req) => {
|
|
|
215
234
|
});
|
|
216
235
|
|
|
217
236
|
server.implement("todos", {
|
|
218
|
-
query: () => [...
|
|
237
|
+
query: (_ctx, db) => [...db.values()],
|
|
219
238
|
mutate: async (op) => {
|
|
220
239
|
if (op.type === "delete") todos.delete(op.rowId);
|
|
221
240
|
else todos.set(op.rowId, { id: op.rowId, ...(op.payload as Partial<Todo>) } as Todo);
|
|
@@ -860,6 +879,145 @@ const server = createSyncServer({
|
|
|
860
879
|
|
|
861
880
|
Each poll tick first probes the shared op log's head HLC; an idle tick costs one `MAX(hlc)` query and broadcasts nothing. Only tables that actually changed are re-broadcast. The tick also re-merges the shared clock watermark, so an instance whose wall clock lags its peers stops stamping writes below HLCs clients have already seen.
|
|
862
881
|
|
|
882
|
+
### Sync with no database at all
|
|
883
|
+
|
|
884
|
+
`createObjectStorage` runs a room with an S3-compatible bucket as the only durable store — no Postgres, no SQLite, no volume. Works with AWS S3, Cloudflare R2, Tigris, MinIO and GCS.
|
|
885
|
+
|
|
886
|
+
```ts
|
|
887
|
+
import { createObjectStorage } from "reflectdb/server/storage/object";
|
|
888
|
+
|
|
889
|
+
const storage = createObjectStorage({
|
|
890
|
+
store: {
|
|
891
|
+
provider: "tigris", // or "aws" | "r2" | "minio" | "gcs"
|
|
892
|
+
bucket: process.env.S3_BUCKET!,
|
|
893
|
+
credentials: {
|
|
894
|
+
keyId: process.env.S3_ACCESS_KEY_ID!,
|
|
895
|
+
secret: process.env.S3_SECRET_ACCESS_KEY!,
|
|
896
|
+
},
|
|
897
|
+
},
|
|
898
|
+
roomId: "board-42",
|
|
899
|
+
});
|
|
900
|
+
|
|
901
|
+
const server = createSyncServer({ queries, db, transport, storage });
|
|
902
|
+
|
|
903
|
+
// Flush and release the lease so the next machine takes over immediately.
|
|
904
|
+
process.on("SIGTERM", () => storage.close().then(() => process.exit(0)));
|
|
905
|
+
```
|
|
906
|
+
|
|
907
|
+
Row state is authoritative **in memory** and the bucket is durability only, so reads never touch the network. A write appends to a buffer that group-commits one object per batch, then advances a compare-and-swapped manifest — the log's single linearization point. The flush loop is self-clocking, so there is no flush interval to tune, and an idle room issues no requests at all.
|
|
908
|
+
|
|
909
|
+
The trade is that a room has exactly one writer, elected with a lease. Route each room to one instance: this adapter replaces shared-database HA polling with room affinity rather than layering on top of it. Where you cannot promise that routing, see [Serverless sync on Vercel](#serverless-sync-on-vercel).
|
|
910
|
+
|
|
911
|
+
Every knob and its default is in [Object storage (no database)](#object-storage-no-database); the design, the failure modes and the known limits are in [`docs/object-storage.md`](./docs/object-storage.md).
|
|
912
|
+
|
|
913
|
+
### Serverless sync on Vercel
|
|
914
|
+
|
|
915
|
+
Serverless functions break two assumptions a long-lived server gets for free. Both have a flag, and [`examples/kanban`](./examples/kanban/) is a deployed board that uses them.
|
|
916
|
+
|
|
917
|
+
**Any request can land on any instance,** so there is no single writer to elect. Drop the lease and let instances race on the manifest CAS instead:
|
|
918
|
+
|
|
919
|
+
```ts
|
|
920
|
+
const storage = createObjectStorage({
|
|
921
|
+
store: { /* … */ },
|
|
922
|
+
roomId,
|
|
923
|
+
concurrency: "optimistic", // no lease; the loser of a CAS re-reads and retries
|
|
924
|
+
});
|
|
925
|
+
```
|
|
926
|
+
|
|
927
|
+
That rests on the same guarantee the lease mode does — the CAS is what keeps the data correct, and the lease was only ever an optimization. What it costs is that in-memory state is no longer authoritative, because another invocation may have committed since this one last looked. Call `await storage.refresh()` — one manifest GET, `true` when something moved — before a read that must be current.
|
|
928
|
+
|
|
929
|
+
**A function cannot hold a WebSocket, and SSE is one-way.** The POST and the event stream are two separate invocations, so a reply to a POST can never reach a stream that process does not own. `serverless: true` returns those replies in the POST's own response instead:
|
|
930
|
+
|
|
931
|
+
```ts
|
|
932
|
+
const transport = createSseServerTransport({ serverless: true });
|
|
933
|
+
const handler = new MessageHandler({ transport, serverId, db, allowAnonymous: true });
|
|
934
|
+
handler.setStorage(storage);
|
|
935
|
+
|
|
936
|
+
// POST /api/sync/messages — the replies this message produced
|
|
937
|
+
const messages = await transport.collectReplies(clientId, message, () =>
|
|
938
|
+
handler.whenIdle(clientId),
|
|
939
|
+
);
|
|
940
|
+
return Response.json({ messages });
|
|
941
|
+
|
|
942
|
+
// GET /api/sync/events — the stream carries only OTHER clients' changes
|
|
943
|
+
setInterval(async () => {
|
|
944
|
+
if (await storage.refresh()) await handler.pollRemoteChanges();
|
|
945
|
+
}, 250);
|
|
946
|
+
```
|
|
947
|
+
|
|
948
|
+
`MessageHandler` is exported from `reflectdb/server`; the example drives it directly rather than through `createSyncServer`, because a serverless route needs `whenIdle` and `pollRemoteChanges`. Set the matching `serverless: true` on `createSseClientTransport`. Two more things a serverless deployment owns, both worked through in the kanban example: the client's session and subscriptions are rebuilt per invocation, because the `hello` and `sync_declare` went to a different process, and the stream instance must `bootstrap` so its result cache holds what the client is actually holding — the broadcast engine sends a diff against that cache, and an empty one makes every existing row look new.
|
|
949
|
+
|
|
950
|
+
### htmx 4 bindings
|
|
951
|
+
|
|
952
|
+
`reflectdb/htmx` lets htmx drive the DOM while reflectdb owns the data. Bindings point at a `reflect:` action instead of a server route, so reads and writes resolve against the local store — optimistic, offline-capable, and re-rendered whenever a peer's change arrives. htmx 4 core ships no SSE or WebSocket support of its own; sync stays reflectdb's job.
|
|
953
|
+
|
|
954
|
+
```ts
|
|
955
|
+
import htmx from "htmx.org";
|
|
956
|
+
import { createHtmxSync } from "reflectdb/htmx";
|
|
957
|
+
|
|
958
|
+
const reflect = createHtmxSync({
|
|
959
|
+
htmx,
|
|
960
|
+
url: "ws://localhost:3001/sync",
|
|
961
|
+
token,
|
|
962
|
+
tables: ["todos"],
|
|
963
|
+
});
|
|
964
|
+
|
|
965
|
+
reflect.view<Todo>("todos", ({ rows }) =>
|
|
966
|
+
rows
|
|
967
|
+
.map(
|
|
968
|
+
(todo) => `
|
|
969
|
+
<li>
|
|
970
|
+
<input type="checkbox" ${todo.done ? "checked" : ""}
|
|
971
|
+
hx-put="reflect:todos/${todo.id}"
|
|
972
|
+
hx-vals='{"done": "${!todo.done}"}'>
|
|
973
|
+
${escapeHtml(todo.text)}
|
|
974
|
+
<button hx-delete="reflect:todos/${todo.id}">×</button>
|
|
975
|
+
</li>`,
|
|
976
|
+
)
|
|
977
|
+
.join(""),
|
|
978
|
+
);
|
|
979
|
+
|
|
980
|
+
// Form bodies arrive as strings — coerce before they reach the op log.
|
|
981
|
+
reflect.parse<Todo>("todos", (payload) => ({
|
|
982
|
+
...payload,
|
|
983
|
+
done: payload.done === "true",
|
|
984
|
+
}));
|
|
985
|
+
|
|
986
|
+
await reflect.connect();
|
|
987
|
+
```
|
|
988
|
+
|
|
989
|
+
```html
|
|
990
|
+
<ul hx-get="reflect:todos" hx-trigger="load" hx-swap="innerMorph"></ul>
|
|
991
|
+
|
|
992
|
+
<form hx-post="reflect:todos">
|
|
993
|
+
<input name="text" required>
|
|
994
|
+
<input type="hidden" name="done" value="false">
|
|
995
|
+
<button>Add</button>
|
|
996
|
+
</form>
|
|
997
|
+
```
|
|
998
|
+
|
|
999
|
+
The action grammar is REST-shaped, so the attributes read like ordinary htmx:
|
|
1000
|
+
|
|
1001
|
+
```
|
|
1002
|
+
GET reflect:<table> → render the collection view
|
|
1003
|
+
GET reflect:<table>/:id → render one row
|
|
1004
|
+
POST reflect:<table> → insert (row id from the body's `id`, else generated)
|
|
1005
|
+
PUT reflect:<table>/:id → update
|
|
1006
|
+
PATCH reflect:<table>/:id → update
|
|
1007
|
+
DELETE reflect:<table>/:id → delete
|
|
1008
|
+
```
|
|
1009
|
+
|
|
1010
|
+
Writes answer `204 No Content`, so htmx swaps nothing where the write happened. The store change then re-renders every bound element a beat later — one render path whether the edit came from this tab, another tab, or the server.
|
|
1011
|
+
|
|
1012
|
+
Worth knowing:
|
|
1013
|
+
|
|
1014
|
+
- An element binds by making its first `reflect:` read, so give collection bindings `hx-trigger="load"`.
|
|
1015
|
+
- Use an inner swap (`innerHTML`, `innerMorph`) on collection bindings. `outerHTML` replaces the bound element itself, and a replacement still carrying `hx-trigger="load"` would re-request forever.
|
|
1016
|
+
- `hx-swap="innerMorph"` is usually what you want: htmx 4 morphs in place, so focus and caret position survive a re-render.
|
|
1017
|
+
- Query params reach the view for filtering (`reflect:todos?done=false`); they do not change the server subscription. Declare that with `tables`, or with `reflect.sync.sync(table, { params })`.
|
|
1018
|
+
- A row read whose row is missing answers `204`, leaving existing markup alone instead of blanking it.
|
|
1019
|
+
- A view returns a raw HTML string — escape interpolated values yourself.
|
|
1020
|
+
|
|
863
1021
|
## Whiteboard + Pictionary example
|
|
864
1022
|
|
|
865
1023
|
A complete React + Bun + Drizzle app that exercises most of reflectdb in one
|
|
@@ -931,6 +1089,81 @@ database. The included Fly.io config runs on one auto-stopping 256 MB Machine.
|
|
|
931
1089
|
| Headless game rules and top-out reset tests | [`game.ts`](./examples/tetris/game.ts) / [`game.test.ts`](./examples/tetris/game.test.ts) |
|
|
932
1090
|
| Bun SQLite persistence and restart tests | [`database.ts`](./examples/tetris/database.ts) / [`database.test.ts`](./examples/tetris/database.test.ts) |
|
|
933
1091
|
|
|
1092
|
+
## Multiplayer kanban example
|
|
1093
|
+
|
|
1094
|
+
A shared board whose entire durable state is an S3-compatible bucket, deployed as
|
|
1095
|
+
Vercel functions: [`examples/kanban/`](./examples/kanban/). It is live at
|
|
1096
|
+
[reflectdb-kanban.vercel.app](https://reflectdb-kanban.vercel.app/). No Postgres,
|
|
1097
|
+
no SQLite, no volume, no Redis.
|
|
1098
|
+
|
|
1099
|
+
```bash
|
|
1100
|
+
cd examples/kanban
|
|
1101
|
+
bun install
|
|
1102
|
+
KANBAN_LOCAL_DIR=.data vercel dev
|
|
1103
|
+
# open http://localhost:3000 in two tabs and drag a card
|
|
1104
|
+
```
|
|
1105
|
+
|
|
1106
|
+
`KANBAN_LOCAL_DIR` swaps the bucket for a directory, so the example runs with no
|
|
1107
|
+
credentials — the filesystem driver has the same CAS semantics and the whole
|
|
1108
|
+
conformance suite runs against both. `vite` alone serves the UI but not `/api`,
|
|
1109
|
+
so the board will not connect without `vercel dev`.
|
|
1110
|
+
|
|
1111
|
+
The board is open to anyone with the link and `?board=<slug>` makes a new one.
|
|
1112
|
+
Every board resets to its starting cards on a five-minute window, claimed with a
|
|
1113
|
+
single `If-None-Match: *` write so exactly one of N racing invocations does the
|
|
1114
|
+
work — a cron job would not run on Vercel's Hobby tier, and would leave an idle
|
|
1115
|
+
board costing something.
|
|
1116
|
+
|
|
1117
|
+
| Pattern | Where |
|
|
1118
|
+
|---------|-------|
|
|
1119
|
+
| Object storage as the only durable store | `createObjectStorage` in [`lib/board.ts`](./examples/kanban/lib/board.ts) |
|
|
1120
|
+
| `concurrency: "optimistic"` — no lease, instances race on the manifest CAS | [`lib/board.ts`](./examples/kanban/lib/board.ts) |
|
|
1121
|
+
| Serverless SSE — replies returned from the POST that produced them | [`api/sync/messages.ts`](./examples/kanban/api/sync/messages.ts) |
|
|
1122
|
+
| `storage.refresh()` → `handler.pollRemoteChanges()` as the stream's poll loop | [`api/sync/events.ts`](./examples/kanban/api/sync/events.ts) |
|
|
1123
|
+
| Rebuilding a client's session, subscription and result cache per invocation | `restoreSubscription` in [`api/sync/events.ts`](./examples/kanban/api/sync/events.ts) |
|
|
1124
|
+
| Per-column merge so a rename and a drag on the same card both land | `cards` in [`lib/board.ts`](./examples/kanban/lib/board.ts) |
|
|
1125
|
+
| Payload validation with `MutationError` rather than coercion | `cards.mutate` in [`lib/board.ts`](./examples/kanban/lib/board.ts) |
|
|
1126
|
+
| Lazy periodic reset claimed with create-if-absent, applied through `applyServerOp` | [`lib/reset.ts`](./examples/kanban/lib/reset.ts) |
|
|
1127
|
+
| Fractional positioning for drag-and-drop ordering | [`schema.ts`](./examples/kanban/schema.ts) |
|
|
1128
|
+
| Bundling the API routes so Vercel's Node builder never sees a `.ts` specifier | [`scripts/build-kanban.ts`](./scripts/build-kanban.ts) / [`vercel.json`](./vercel.json) |
|
|
1129
|
+
|
|
1130
|
+
Deploy from the repository root rather than the example directory — the example
|
|
1131
|
+
imports reflectdb from `src/`, and the root `vercel.json` runs a build that
|
|
1132
|
+
bundles the two API routes into `.vercel/output` itself. Pushing to `main` does
|
|
1133
|
+
it; the demo's Vercel project is linked to this repository at the repository
|
|
1134
|
+
root. The variables to set, and the one extra `storage.init()` step MinIO needs,
|
|
1135
|
+
are in [`examples/kanban/README.md`](./examples/kanban/README.md).
|
|
1136
|
+
|
|
1137
|
+
## htmx todos example
|
|
1138
|
+
|
|
1139
|
+
A todo list where htmx 4 owns the DOM and reflectdb owns the data:
|
|
1140
|
+
[`examples/htmx-todos/`](./examples/htmx-todos/). It is live at
|
|
1141
|
+
[reflectdb-htmx-todos.fly.dev](https://reflectdb-htmx-todos.fly.dev/). The
|
|
1142
|
+
server renders no HTML — every fragment is produced in the browser from the
|
|
1143
|
+
local store. The deployed list resets to its seed rows every minute.
|
|
1144
|
+
|
|
1145
|
+
```bash
|
|
1146
|
+
cd examples/htmx-todos
|
|
1147
|
+
bun install
|
|
1148
|
+
bun run dev
|
|
1149
|
+
# open http://localhost:3005 in two tabs
|
|
1150
|
+
```
|
|
1151
|
+
|
|
1152
|
+
The server takes the first free port at or above `PORT` (default 3005), so it
|
|
1153
|
+
does not collide with the other examples.
|
|
1154
|
+
|
|
1155
|
+
| Pattern | Where |
|
|
1156
|
+
|---------|-------|
|
|
1157
|
+
| `reflect:` actions on ordinary htmx attributes | [`index.html`](./examples/htmx-todos/index.html) |
|
|
1158
|
+
| One view function rendering the whole list from local rows | [`client.ts`](./examples/htmx-todos/client.ts) |
|
|
1159
|
+
| `parse` building a patch, so a checkbox toggle keeps the text it never sent | [`client.ts`](./examples/htmx-todos/client.ts) |
|
|
1160
|
+
| A counter riding along as an `hx-swap-oob` element | [`client.ts`](./examples/htmx-todos/client.ts) |
|
|
1161
|
+
| Filters as query params, kept across peers' edits | [`index.html`](./examples/htmx-todos/index.html) |
|
|
1162
|
+
| A periodic reset routed through `applyServerOp`, so it reaches every open tab | [`server.ts`](./examples/htmx-todos/server.ts) |
|
|
1163
|
+
|
|
1164
|
+
Stop the server and keep typing: writes land in the DOM immediately, the badge
|
|
1165
|
+
counts them, and they drain on reconnect.
|
|
1166
|
+
|
|
934
1167
|
## Architecture
|
|
935
1168
|
|
|
936
1169
|
```
|
|
@@ -982,6 +1215,7 @@ database. The included Fly.io config runs on one auto-stopping 256 MB Machine.
|
|
|
982
1215
|
│ • in-memory (default) │ │ │
|
|
983
1216
|
│ • sqlite (bun:sqlite) │ │ │
|
|
984
1217
|
│ • postgres (any pg-compatible) │ │ │
|
|
1218
|
+
│ • object (any S3-compatible) │ │ │
|
|
985
1219
|
└───────────────────────────────────┘ └──────────────────────────────────────┘
|
|
986
1220
|
|
|
987
1221
|
┌─────────────────────────────────────────────────────────────────────────────┐
|
|
@@ -995,14 +1229,60 @@ database. The included Fly.io config runs on one auto-stopping 256 MB Machine.
|
|
|
995
1229
|
└─────────────────────────────────────────────────────────────────────────────┘
|
|
996
1230
|
|
|
997
1231
|
┌─────────────────────────────────────────────────────────────────────────────┐
|
|
998
|
-
│
|
|
1232
|
+
│ FRAMEWORK BINDINGS (react/, svelte/, vanilla/, htmx/) │
|
|
999
1233
|
│ │
|
|
1000
1234
|
│ • createSyncReact(queries) → typed hooks + <SyncProvider> │
|
|
1001
1235
|
│ • createSyncSvelte(queries) → typed Svelte stores │
|
|
1002
1236
|
│ • createSyncVanilla(queries) → typed callback API │
|
|
1237
|
+
│ • createSyncHtmx(queries) → typed views behind reflect: attributes │
|
|
1003
1238
|
└─────────────────────────────────────────────────────────────────────────────┘
|
|
1004
1239
|
```
|
|
1005
1240
|
|
|
1241
|
+
### Object storage as the durable store
|
|
1242
|
+
|
|
1243
|
+
`createObjectStorage` replaces that op-log box with an S3-compatible bucket, and
|
|
1244
|
+
nothing else in the diagram changes. The bucket is durability, never the read
|
|
1245
|
+
path:
|
|
1246
|
+
|
|
1247
|
+
```
|
|
1248
|
+
┌──────────────────────────────────────┐
|
|
1249
|
+
clients ───────▶ │ writer instance (one per room) │
|
|
1250
|
+
│ │
|
|
1251
|
+
│ in-memory authoritative state │ ◀── every read
|
|
1252
|
+
│ rows · per-column HLCs · op ring │ lands here
|
|
1253
|
+
│ reserveOp set · meta │
|
|
1254
|
+
│ │
|
|
1255
|
+
│ write buffer ──▶ group commit │
|
|
1256
|
+
└───────────────────┬──────────────────┘
|
|
1257
|
+
│ one PUT per batch,
|
|
1258
|
+
│ then one CAS
|
|
1259
|
+
▼
|
|
1260
|
+
┌──────────────────────────────────────┐
|
|
1261
|
+
│ object store — S3 · R2 · Tigris · │
|
|
1262
|
+
│ MinIO · GCS │
|
|
1263
|
+
│ │
|
|
1264
|
+
│ _lease writer election │
|
|
1265
|
+
│ _manifest the CAS'd commit point │
|
|
1266
|
+
│ wal/ immutable op batches │
|
|
1267
|
+
│ snap/ materialized rows │
|
|
1268
|
+
└──────────────────────────────────────┘
|
|
1269
|
+
```
|
|
1270
|
+
|
|
1271
|
+
Reads (`getRow`, `getRows`, `getOpsSince`, `reserveOp`) hit memory and never
|
|
1272
|
+
touch the network. A write mutates memory and appends to a buffer; the buffer
|
|
1273
|
+
flushes as one object per batch, and the commit is the compare-and-swap that adds
|
|
1274
|
+
that segment to `_manifest`. Boot is one manifest GET, the newest snapshot, and
|
|
1275
|
+
the segments the manifest still lists.
|
|
1276
|
+
|
|
1277
|
+
Only `_lease` and `_manifest` are ever overwritten, and only via CAS. Everything
|
|
1278
|
+
else is write-once, which is what makes concurrent readers safe — and what lets
|
|
1279
|
+
`concurrency: "optimistic"` drop the lease entirely on a platform that cannot
|
|
1280
|
+
route a room to one instance.
|
|
1281
|
+
|
|
1282
|
+
Full design, provider compatibility, durability model and known limits:
|
|
1283
|
+
[`docs/object-storage.md`](./docs/object-storage.md).
|
|
1284
|
+
|
|
1285
|
+
|
|
1006
1286
|
## Core Concepts
|
|
1007
1287
|
|
|
1008
1288
|
### Hybrid Logical Clocks
|
|
@@ -1073,7 +1353,7 @@ reflectdb keeps its own store alongside yours, and it helps to know which one an
|
|
|
1073
1353
|
|
|
1074
1354
|
| Read | Source |
|
|
1075
1355
|
|------|--------|
|
|
1076
|
-
| Snapshots (`bootstrap`, `resume`) | **Your database**, via the `query` callback |
|
|
1356
|
+
| Snapshots (`bootstrap`, `resume`) | **Your database**, via the `query` callback — which only runs when `db` was passed to `createSyncServer` |
|
|
1077
1357
|
| Broadcast deltas | **Your database**, diffed against a per-client cached result set |
|
|
1078
1358
|
| Conflict resolution (`lww` / `merge` / `server` / custom) | **reflectdb's mirror** — a JSONB row store plus per-column HLCs |
|
|
1079
1359
|
| Which tables changed since an HLC | **reflectdb's op log** |
|
|
@@ -1168,6 +1448,42 @@ See [Server-driven game loops](#server-driven-game-loops) for `interval` / `lock
|
|
|
1168
1448
|
|
|
1169
1449
|
`createServer()` is the lower-level untyped variant — use it only if you need to register queries dynamically or don't have a `defineSyncQueries` map.
|
|
1170
1450
|
|
|
1451
|
+
### `reflectdb/server/storage/object`
|
|
1452
|
+
|
|
1453
|
+
```ts
|
|
1454
|
+
import {
|
|
1455
|
+
createObjectStorage,
|
|
1456
|
+
createS3Driver, createFilesystemDriver, createMemoryDriver,
|
|
1457
|
+
PreconditionFailedError, BackpressureError, NotWriterError, MemoryLimitExceededError,
|
|
1458
|
+
IncompleteStateError, roomPrefix,
|
|
1459
|
+
} from "reflectdb/server/storage/object";
|
|
1460
|
+
```
|
|
1461
|
+
|
|
1462
|
+
**`createObjectStorage(config)`** — a storage adapter backed by an S3-compatible bucket. It satisfies the same `StorageAdapter` contract as the SQLite and Postgres adapters, plus a lifecycle and observability surface the design needs:
|
|
1463
|
+
|
|
1464
|
+
| Member | Purpose |
|
|
1465
|
+
|--------|---------|
|
|
1466
|
+
| `init()` | Boot the room — manifest, snapshot, WAL replay. Idempotent, and implied by the first call to anything else; call it explicitly to surface boot failures at startup rather than on the first query. On MinIO it also seeds `_lease` and `_manifest`, which makes it a **deploy step** rather than something N servers race. |
|
|
1467
|
+
| `refresh()` | Fold in whatever another instance has committed. Returns `true` when the room actually moved. Costs one GET, and nothing further when `commitSeq` has not changed. Only meaningful under `concurrency: "optimistic"`. |
|
|
1468
|
+
| `flush()` | Resolve once everything buffered is durable. |
|
|
1469
|
+
| `close()` | Stop accepting writes, flush, stop the flush loop, release the lease. Every step is bounded by `shutdownFlushMs`, so a hung store cannot hold a `SIGTERM` open. |
|
|
1470
|
+
| `health` | `"healthy"`, `"degraded"` or `"unavailable"`, with `onHealthChange(cb)` to observe transitions. |
|
|
1471
|
+
| `durableHlc` | Highest HLC known to be on the store, with `onDurable(cb)` fired per batch. |
|
|
1472
|
+
|
|
1473
|
+
Errors are typed so a caller can tell "someone else moved first" from a transport failure: `PreconditionFailedError` (a CAS lost), `BackpressureError` (the write buffer hit `batch.maxBufferBytes`), `NotWriterError` (this instance lost or never held the lease), `MemoryLimitExceededError` (room state exceeded `memory.maxRoomBytes`), `IncompleteStateError` (the manifest names an object the store does not have — the room refuses to boot rather than present the loss as an empty room).
|
|
1474
|
+
|
|
1475
|
+
`roomPrefix(roomId)` returns the key prefix a room writes under, for tooling that has to address those keys from outside the adapter — an admin wipe, or a disposable room clearing itself after an `IncompleteStateError`.
|
|
1476
|
+
|
|
1477
|
+
Pass `store` and the S3 driver is built for you; pass `driver` to supply one directly:
|
|
1478
|
+
|
|
1479
|
+
| Driver | Signature | Use |
|
|
1480
|
+
|--------|-----------|-----|
|
|
1481
|
+
| `createS3Driver` | `(config: StoreConfig)` | Any S3-compatible bucket. SigV4 over `fetch` with WebCrypto — no `node:crypto`, so the browser build stays clean. |
|
|
1482
|
+
| `createFilesystemDriver` | `(rootDir: string)` | Local development and tests with no network. Same CAS semantics via atomic rename, but a filesystem cannot make compare-then-rename atomic across processes — single process only. |
|
|
1483
|
+
| `createMemoryDriver` | `(options?)` | Tests, with fault injection (412 storms, 500s, latency) and `casWildcard: false` to reproduce MinIO. |
|
|
1484
|
+
|
|
1485
|
+
See [Object storage (no database)](#object-storage-no-database) for the configuration, and [`docs/object-storage.md`](./docs/object-storage.md) for the design.
|
|
1486
|
+
|
|
1171
1487
|
### `reflectdb/client`
|
|
1172
1488
|
|
|
1173
1489
|
```ts
|
|
@@ -1272,19 +1588,40 @@ store.onError((e) => …);
|
|
|
1272
1588
|
```ts
|
|
1273
1589
|
import { createSync, createSyncVanilla, createBrowserWsTransport } from "reflectdb/vanilla";
|
|
1274
1590
|
|
|
1275
|
-
const sync = createSync({ url, token, tables: ["notes"] });
|
|
1591
|
+
const sync = createSync({ url, token, tables: ["notes"], onError: (e) => … });
|
|
1276
1592
|
const notes = sync.sync<Note>("notes");
|
|
1277
1593
|
|
|
1278
|
-
notes.onChange((
|
|
1594
|
+
notes.onChange(() => render(notes.getRows()));
|
|
1279
1595
|
notes.insert(id, { title: "…" });
|
|
1280
1596
|
|
|
1281
1597
|
sync.onStateChange((s) => …);
|
|
1282
1598
|
sync.onPendingChange((n) => …);
|
|
1283
|
-
sync.onError((e) => …);
|
|
1284
1599
|
sync.connect();
|
|
1285
1600
|
```
|
|
1286
1601
|
|
|
1287
|
-
Also supports ephemeral: `sync.
|
|
1602
|
+
Also supports ephemeral: `sync.ephemeral({ key, userId, ttlMs })` returns a binding with `broadcast(data)`, `getEvents()` and `onChange(listener)`.
|
|
1603
|
+
|
|
1604
|
+
### `reflectdb/htmx`
|
|
1605
|
+
|
|
1606
|
+
```ts
|
|
1607
|
+
import { createHtmxSync, createSyncHtmx } from "reflectdb/htmx";
|
|
1608
|
+
|
|
1609
|
+
const reflect = createHtmxSync({ htmx, url, token, tables: ["todos"] });
|
|
1610
|
+
|
|
1611
|
+
reflect.view<Todo>("todos", ({ rows, rowId, params }) => "<li>…</li>");
|
|
1612
|
+
reflect.parse<Todo>("todos", (payload) => ({ …payload, done: payload.done === "true" }));
|
|
1613
|
+
|
|
1614
|
+
reflect.install(); // attach the reflect: shim (connect() does this too)
|
|
1615
|
+
reflect.uninstall(); // detach it and drop every element binding
|
|
1616
|
+
reflect.refresh("todos"); // re-render bound elements on demand
|
|
1617
|
+
reflect.sync; // the underlying VanillaSync, for anything attributes cannot express
|
|
1618
|
+
|
|
1619
|
+
await reflect.connect();
|
|
1620
|
+
```
|
|
1621
|
+
|
|
1622
|
+
Requires htmx 4 — import the instance yourself and pass it in. The adapter listens for `htmx:config:request` and sets `ctx.fetch` on `reflect:` actions, so htmx still performs the swap, OOB handling, settling and history it would for a server response.
|
|
1623
|
+
|
|
1624
|
+
`createSyncHtmx<typeof queries>()` returns the same factory with `view`, `parse` and `refresh` narrowed to your schema's table names and row types. See [htmx 4 bindings](#htmx-4-bindings) for the action grammar.
|
|
1288
1625
|
|
|
1289
1626
|
### `reflectdb/transport/*`
|
|
1290
1627
|
|
|
@@ -1502,11 +1839,105 @@ At boot the client restores its persisted subscriptions first and hydrates only
|
|
|
1502
1839
|
| _(none)_ | omit `storage` | In-memory op log; ephemeral, single node |
|
|
1503
1840
|
| `createSqliteStorage({ path?, db? })` | `reflectdb/server` | Single server, development, embedded — **Bun only** |
|
|
1504
1841
|
| `createPostgresStorage(poolOrConfig)` | `reflectdb/server` | Multi-server HA, production |
|
|
1842
|
+
| `createObjectStorage({ store, roomId })` | `reflectdb/server/storage/object` | S3-compatible object storage as the only durable store — no database |
|
|
1505
1843
|
|
|
1506
1844
|
`createPostgresStorage` accepts any object with `query(text, values) => { rows }` — `pg.Pool`, `pg.Client`, `@neondatabase/serverless`, etc. Optional config: `{ client, tablePrefix: "_reflectdb" }`.
|
|
1507
1845
|
|
|
1508
1846
|
`createSqliteStorage` is backed by `bun:sqlite` and is resolved lazily, so importing `reflectdb/server` on Node is fine — only calling `createSqliteStorage` there throws. On Node, use `createPostgresStorage` (or omit `storage` for the in-memory op log).
|
|
1509
1847
|
|
|
1848
|
+
#### Object storage (no database)
|
|
1849
|
+
|
|
1850
|
+
`createObjectStorage` runs a room with S3-compatible object storage as the only
|
|
1851
|
+
durable store. Works with AWS S3, Cloudflare R2, Tigris, MinIO and GCS.
|
|
1852
|
+
|
|
1853
|
+
```ts
|
|
1854
|
+
import { createObjectStorage } from "reflectdb/server/storage/object";
|
|
1855
|
+
|
|
1856
|
+
const storage = createObjectStorage({
|
|
1857
|
+
store: {
|
|
1858
|
+
provider: "tigris", // or "aws" | "r2" | "minio" | "gcs"
|
|
1859
|
+
bucket: "my-app",
|
|
1860
|
+
credentials: {
|
|
1861
|
+
keyId: process.env.AWS_ACCESS_KEY_ID!,
|
|
1862
|
+
secret: process.env.AWS_SECRET_ACCESS_KEY!,
|
|
1863
|
+
},
|
|
1864
|
+
},
|
|
1865
|
+
roomId: "board-42",
|
|
1866
|
+
});
|
|
1867
|
+
|
|
1868
|
+
const server = createSyncServer({ queries, db, transport, storage });
|
|
1869
|
+
|
|
1870
|
+
// Flush and release the lease so the next machine takes over immediately.
|
|
1871
|
+
process.on("SIGTERM", () => storage.close().then(() => process.exit(0)));
|
|
1872
|
+
```
|
|
1873
|
+
|
|
1874
|
+
Row state is authoritative **in memory** and the object store is durability
|
|
1875
|
+
only, so reads never touch the network. Writes group-commit: one PUT per batch,
|
|
1876
|
+
self-clocking, with no flush interval to tune. An idle room issues zero requests.
|
|
1877
|
+
|
|
1878
|
+
Because state is in memory and a room has exactly one writer, **route each room
|
|
1879
|
+
to one instance** — this adapter replaces shared-database HA polling with room
|
|
1880
|
+
affinity rather than layering on top of it.
|
|
1881
|
+
|
|
1882
|
+
Where you cannot promise that routing — serverless functions, where any request
|
|
1883
|
+
lands on any instance — set `concurrency: "optimistic"`. It drops the lease and
|
|
1884
|
+
lets instances race on the manifest CAS instead, with the loser re-reading and
|
|
1885
|
+
retrying. That rests on the same guarantee the lease mode does: the CAS is what
|
|
1886
|
+
keeps the data correct, and the lease was only ever an optimization. In exchange,
|
|
1887
|
+
in-memory state is no longer authoritative, so call `await storage.refresh()`
|
|
1888
|
+
(one manifest GET) before a read that must be current.
|
|
1889
|
+
|
|
1890
|
+
Defaults are correct rather than fast: `durability: "durable"` acks a write only
|
|
1891
|
+
once it is on the store. `"buffered"` acks earlier and is lossy on crash until
|
|
1892
|
+
the durable-watermark protocol lands.
|
|
1893
|
+
|
|
1894
|
+
The `SIGTERM` handler above matters — without it a deploy drops whatever is
|
|
1895
|
+
buffered, and the next writer waits out `lease.ttlMs` before taking the room.
|
|
1896
|
+
|
|
1897
|
+
MinIO needs a one-time `await storage.init()` as a **deploy step** (it rejects
|
|
1898
|
+
the `If-None-Match: *` wildcard, so the room has to be seeded); on every other
|
|
1899
|
+
provider `init()` is a no-op.
|
|
1900
|
+
|
|
1901
|
+
`store` describes the bucket. `provider` fills in `endpoint` and `urlStyle`, so most deployments set three fields and nothing else:
|
|
1902
|
+
|
|
1903
|
+
| Field | Required | Notes |
|
|
1904
|
+
|-------|----------|-------|
|
|
1905
|
+
| `bucket` | yes | |
|
|
1906
|
+
| `credentials` | yes | `{ keyId, secret, sessionToken? }`. |
|
|
1907
|
+
| `provider` | no | `"aws"`, `"r2"`, `"tigris"`, `"minio"` or `"gcs"` — a preset for `endpoint`, `region` and `urlStyle`. |
|
|
1908
|
+
| `accountId` | for R2 | Derives `https://<id>.r2.cloudflarestorage.com`. |
|
|
1909
|
+
| `endpoint` / `region` / `urlStyle` | no | Override the preset. MinIO has no default endpoint, and a Fly-provisioned Tigris bucket uses `fly.storage.tigris.dev` rather than the preset's `t3.storage.dev`. |
|
|
1910
|
+
| `prefix` | no | Key prefix, so one bucket holds many apps. Keys are `<prefix>/rooms/<roomId>/…`. |
|
|
1911
|
+
|
|
1912
|
+
Pass `driver` instead of `store` to supply a driver directly — `createFilesystemDriver(dir)` for local development with no credentials, `createMemoryDriver()` for tests.
|
|
1913
|
+
|
|
1914
|
+
Everything else, with its default:
|
|
1915
|
+
|
|
1916
|
+
| Option | Default | Purpose |
|
|
1917
|
+
|--------|---------|---------|
|
|
1918
|
+
| `roomId` | — | Required. One room per adapter; it is the key prefix and the routing key. |
|
|
1919
|
+
| `writerId` | random | Identifies this writer in the lease. Set it to make ownership legible in logs. |
|
|
1920
|
+
| `durability` | `"durable"` | Ack after the manifest CAS. `"buffered"` acks on memory apply and is lossy on crash until the durable-watermark protocol lands. |
|
|
1921
|
+
| `concurrency` | `"single-writer"` | `"optimistic"` drops the lease where a room cannot be routed to one instance. See [Serverless sync on Vercel](#serverless-sync-on-vercel). |
|
|
1922
|
+
| `retentionMs` | `Infinity` | How long accepted ops are kept. |
|
|
1923
|
+
| `batch.maxBytes` | 4 MiB | Largest single WAL segment. |
|
|
1924
|
+
| `batch.minLingerMs` | `5` | Coalesces ops arriving in the same event-loop tick. Not a flush interval — there is none, the flush loop is self-clocking. |
|
|
1925
|
+
| `batch.maxBufferBytes` | 64 MiB | Buffer ceiling before the backpressure policy fires. |
|
|
1926
|
+
| `batch.onBackpressure` | `"reject"` | Throw `BackpressureError` so backpressure reaches the client. `"degrade"` keeps accepting, stops promising durability, and flips `health`. |
|
|
1927
|
+
| `compaction.afterSegments` | `200` | Snapshot once the manifest lists this many segments. Boot costs one GET per listed segment, so lower it for rooms that are read cold more often than they are written. |
|
|
1928
|
+
| `compaction.afterBytes` | 64 MiB | Or this many bytes, whichever comes first. |
|
|
1929
|
+
| `compaction.gcGraceMs` | `3_600_000` | Delay before deleting superseded segments, so a reader holding the old manifest does not 404. |
|
|
1930
|
+
| `lease.ttlMs` / `lease.renewMs` | `300_000` / `120_000` | Long on purpose. `close()` releases the lease, so the TTL bounds only *unclean* failover — and a long one is what keeps an idle room free. |
|
|
1931
|
+
| `lease.mode` | `"on-write"` | A room holding connected clients but taking no writes renews nothing. |
|
|
1932
|
+
| `memory.maxTotalBytes` / `memory.maxRoomBytes` | `Infinity` | Global across rooms, then per room. State is authoritative in memory, so this is a cliff rather than a slope — set it and the ceiling arrives as `MemoryLimitExceededError` instead of an OOM. |
|
|
1933
|
+
| `memory.onExceeded` | `"reject"` | `"evict"` and `"spill"` are accepted as configuration but throw at the limit; neither is implemented yet. |
|
|
1934
|
+
| `memory.idleEvictMs` | `300_000` | Zero-client rooms flush, release the lease and drop their state. |
|
|
1935
|
+
| `shutdownFlushMs` | `5000` | Bounds every step of `close()`, so a hung store cannot hold a `SIGTERM` open. |
|
|
1936
|
+
| `onDurable` / `onHealthChange` | — | Callbacks. The same values are readable as `storage.durableHlc` and `storage.health`. |
|
|
1937
|
+
|
|
1938
|
+
Full design, the configuration reference and the known limits:
|
|
1939
|
+
[`docs/object-storage.md`](./docs/object-storage.md).
|
|
1940
|
+
|
|
1510
1941
|
#### Client
|
|
1511
1942
|
|
|
1512
1943
|
| Adapter | Import | Best for |
|
|
@@ -1592,11 +2023,42 @@ Returns a `ServerTransport` plus `handleOpen`, `handleMessage`, `handleClose`, `
|
|
|
1592
2023
|
```ts
|
|
1593
2024
|
createSseServerTransport({
|
|
1594
2025
|
replayBufferSize: 256, // Last-Event-ID replay window
|
|
2026
|
+
serverless: false, // see below
|
|
1595
2027
|
});
|
|
1596
2028
|
```
|
|
1597
2029
|
|
|
1598
2030
|
Two endpoints to wire: `GET /sync/events/:clientId` (SSE stream) and `POST /sync/messages/:clientId` (client → server).
|
|
1599
2031
|
|
|
2032
|
+
##### Serverless SSE
|
|
2033
|
+
|
|
2034
|
+
SSE is server→client only: normally a client POSTs a message and every reply —
|
|
2035
|
+
`hello_ack`, snapshots, op acks — comes back down the held stream. That works
|
|
2036
|
+
only while the POST and the stream are handled by the **same process**.
|
|
2037
|
+
|
|
2038
|
+
On Vercel, Lambda or Workers they are two separate invocations, so those replies
|
|
2039
|
+
would be enqueued onto a stream the POST's process does not have, and the client
|
|
2040
|
+
hangs at the handshake. Set `serverless: true` on both halves:
|
|
2041
|
+
|
|
2042
|
+
```ts
|
|
2043
|
+
// server: return the replies the POST produced
|
|
2044
|
+
const messages = await transport.collectReplies(clientId, message, () =>
|
|
2045
|
+
handler.whenIdle(clientId),
|
|
2046
|
+
);
|
|
2047
|
+
return Response.json({ messages });
|
|
2048
|
+
|
|
2049
|
+
// client
|
|
2050
|
+
createSseClientTransport({ eventUrl, messageUrl, serverless: true });
|
|
2051
|
+
```
|
|
2052
|
+
|
|
2053
|
+
The stream is then left doing the one thing it can: pushing *other* clients'
|
|
2054
|
+
changes. Leave it off for a single-process server — replies stream normally
|
|
2055
|
+
there, and turning it on would deliver each one twice.
|
|
2056
|
+
|
|
2057
|
+
Two things a serverless deployment must also handle, both shown in
|
|
2058
|
+
[`examples/kanban`](./examples/kanban/): the session is rebuilt per invocation
|
|
2059
|
+
(nothing remembers that this client subscribed), and the stream instance has to
|
|
2060
|
+
poll storage to notice writes made elsewhere.
|
|
2061
|
+
|
|
1600
2062
|
#### HTTP long-polling
|
|
1601
2063
|
|
|
1602
2064
|
```ts
|
|
@@ -1663,6 +2125,7 @@ reference:
|
|
|
1663
2125
|
| `og/index.html` | `public/og.png` | 1200x630 at 2x | reflectdb.dev |
|
|
1664
2126
|
| `og/tetris.html` | `public/og-tetris.png` | 1200x630 at 2x | the Tetris demo — served from reflectdb.dev, since the Fly Machine sleeps |
|
|
1665
2127
|
| `og/whiteboard.html` | `public/og-whiteboard.png` | 1200x630 at 2x | the whiteboard demo, served from reflectdb.dev for the same reason |
|
|
2128
|
+
| `og/kanban.html` | `public/og-kanban.png` | 1200x630 at 2x | the kanban demo, so all four cards regenerate together |
|
|
1666
2129
|
| `og/github.html` | `public/og-github.png` | 1280x640 at 2x | this repository's social preview, uploaded by hand under Settings → Social preview |
|
|
1667
2130
|
|
|
1668
2131
|
Edit the HTML, not the PNGs. 1200x630 is the one ratio X, Facebook, LinkedIn,
|
|
@@ -1697,12 +2160,15 @@ src/
|
|
|
1697
2160
|
│ │ broadcast-engine, result-cache, eager-buffer,
|
|
1698
2161
|
│ │ compaction-manager, replay-detector, ephemeral-manager
|
|
1699
2162
|
│ └── storage/ SQLite + Postgres adapters
|
|
2163
|
+
│ └── object/ S3-compatible object storage — manifest CAS, WAL,
|
|
2164
|
+
│ snapshots, memory / filesystem / S3 drivers
|
|
1700
2165
|
├── client/ sync-client, store, ops, typed-client
|
|
1701
2166
|
│ └── storage/ memory + IndexedDB adapters
|
|
1702
2167
|
├── transport/ WebSocket (runtime-agnostic + Bun.serve), SSE, polling
|
|
1703
2168
|
├── react/ <SyncProvider>, hooks, typed factory
|
|
1704
2169
|
├── svelte/ createSyncStore, typed factory
|
|
1705
|
-
|
|
2170
|
+
├── vanilla/ createSync, typed factory
|
|
2171
|
+
└── htmx/ createHtmxSync, reflect: action router, typed factory
|
|
1706
2172
|
```
|
|
1707
2173
|
|
|
1708
2174
|
### Tech stack
|