@trestleinc/replicate 0.1.0 → 1.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +356 -420
- package/dist/client/collection.d.ts +78 -76
- package/dist/client/errors.d.ts +59 -0
- package/dist/client/index.d.ts +22 -18
- package/dist/client/logger.d.ts +0 -1
- package/dist/client/merge.d.ts +77 -0
- package/dist/client/persistence/adapters/index.d.ts +8 -0
- package/dist/client/persistence/adapters/opsqlite.d.ts +46 -0
- package/dist/client/persistence/adapters/sqljs.d.ts +83 -0
- package/dist/client/persistence/index.d.ts +49 -0
- package/dist/client/persistence/indexeddb.d.ts +17 -0
- package/dist/client/persistence/memory.d.ts +16 -0
- package/dist/client/persistence/sqlite-browser.d.ts +51 -0
- package/dist/client/persistence/sqlite-level.d.ts +63 -0
- package/dist/client/persistence/sqlite-rn.d.ts +36 -0
- package/dist/client/persistence/sqlite.d.ts +47 -0
- package/dist/client/persistence/types.d.ts +42 -0
- package/dist/client/prose.d.ts +56 -0
- package/dist/client/replicate.d.ts +40 -0
- package/dist/client/services/checkpoint.d.ts +18 -0
- package/dist/client/services/reconciliation.d.ts +24 -0
- package/dist/component/_generated/api.d.ts +35 -0
- package/dist/component/_generated/api.js +3 -3
- package/dist/component/_generated/component.d.ts +89 -0
- package/dist/component/_generated/component.js +0 -0
- package/dist/component/_generated/dataModel.d.ts +45 -0
- package/dist/component/_generated/dataModel.js +0 -0
- package/{src → dist}/component/_generated/server.d.ts +9 -38
- package/dist/component/convex.config.d.ts +2 -2
- package/dist/component/convex.config.js +2 -1
- package/dist/component/logger.d.ts +8 -0
- package/dist/component/logger.js +30 -0
- package/dist/component/public.d.ts +36 -61
- package/dist/component/public.js +232 -58
- package/dist/component/schema.d.ts +32 -8
- package/dist/component/schema.js +19 -6
- package/dist/index.js +1553 -308
- package/dist/server/builder.d.ts +94 -0
- package/dist/server/index.d.ts +14 -17
- package/dist/server/schema.d.ts +17 -63
- package/dist/server/storage.d.ts +80 -0
- package/dist/server.js +268 -83
- package/dist/shared/index.d.ts +5 -0
- package/dist/shared/index.js +2 -0
- package/dist/shared/types.d.ts +50 -0
- package/dist/shared/types.js +6 -0
- package/dist/shared.js +6 -0
- package/package.json +59 -49
- package/src/client/collection.ts +877 -450
- package/src/client/errors.ts +45 -0
- package/src/client/index.ts +52 -26
- package/src/client/logger.ts +2 -28
- package/src/client/merge.ts +374 -0
- package/src/client/persistence/adapters/index.ts +8 -0
- package/src/client/persistence/adapters/opsqlite.ts +54 -0
- package/src/client/persistence/adapters/sqljs.ts +128 -0
- package/src/client/persistence/index.ts +54 -0
- package/src/client/persistence/indexeddb.ts +110 -0
- package/src/client/persistence/memory.ts +61 -0
- package/src/client/persistence/sqlite-browser.ts +107 -0
- package/src/client/persistence/sqlite-level.ts +407 -0
- package/src/client/persistence/sqlite-rn.ts +44 -0
- package/src/client/persistence/sqlite.ts +161 -0
- package/src/client/persistence/types.ts +49 -0
- package/src/client/prose.ts +369 -0
- package/src/client/replicate.ts +80 -0
- package/src/client/services/checkpoint.ts +86 -0
- package/src/client/services/reconciliation.ts +108 -0
- package/src/component/_generated/api.ts +52 -0
- package/src/component/_generated/component.ts +103 -0
- package/src/component/_generated/{dataModel.d.ts → dataModel.ts} +1 -1
- package/src/component/_generated/server.ts +161 -0
- package/src/component/convex.config.ts +3 -1
- package/src/component/logger.ts +36 -0
- package/src/component/public.ts +364 -111
- package/src/component/schema.ts +18 -5
- package/src/env.d.ts +31 -0
- package/src/server/builder.ts +85 -0
- package/src/server/index.ts +9 -24
- package/src/server/schema.ts +20 -76
- package/src/server/storage.ts +313 -0
- package/src/shared/index.ts +5 -0
- package/src/shared/types.ts +52 -0
- package/LICENSE.package +0 -201
- package/dist/client/storage.d.ts +0 -143
- package/dist/server/replication.d.ts +0 -122
- package/dist/server/ssr.d.ts +0 -79
- package/dist/ssr.js +0 -19
- package/src/client/storage.ts +0 -206
- package/src/component/_generated/api.d.ts +0 -95
- package/src/component/_generated/api.js +0 -23
- package/src/component/_generated/server.js +0 -90
- package/src/server/replication.ts +0 -244
- package/src/server/ssr.ts +0 -106
package/README.md
CHANGED
|
@@ -1,14 +1,13 @@
|
|
|
1
|
-
#
|
|
1
|
+
# Replicate
|
|
2
2
|
|
|
3
3
|
**Offline-first sync library using Yjs CRDTs and Convex for real-time data synchronization.**
|
|
4
4
|
|
|
5
|
-
|
|
5
|
+
Replicate provides a dual-storage architecture for building offline-capable applications with automatic conflict resolution. It combines Yjs CRDTs (96% smaller than Automerge, no WASM) with TanStack DB's reactive state management and Convex's reactive backend for real-time synchronization and efficient querying.
|
|
6
6
|
|
|
7
7
|
## Features
|
|
8
8
|
|
|
9
9
|
- **Offline-first** - Works without internet, syncs when reconnected
|
|
10
10
|
- **Yjs CRDTs** - Automatic conflict-free replication with Yjs (96% smaller than Automerge, no WASM)
|
|
11
|
-
- **TanStack offline-transactions** - Proven outbox pattern for reliable offline sync
|
|
12
11
|
- **Real-time sync** - Convex WebSocket-based synchronization
|
|
13
12
|
- **TanStack DB integration** - Reactive state management for React and Svelte
|
|
14
13
|
- **Dual-storage pattern** - CRDT layer for conflict resolution + main tables for queries
|
|
@@ -18,7 +17,10 @@ Convex Replicate provides a dual-storage architecture for building offline-capab
|
|
|
18
17
|
- **SSR support** - Server-side rendering with data preloading
|
|
19
18
|
- **Network resilience** - Automatic retry with exponential backoff
|
|
20
19
|
- **Component-based** - Convex component for plug-and-play CRDT storage
|
|
21
|
-
- **
|
|
20
|
+
- **Swappable persistence** - IndexedDB (browser), SQLite (React Native), or in-memory (testing)
|
|
21
|
+
- **React Native compatible** - SQLite persistence with y-op-sqlite and op-sqlite
|
|
22
|
+
- **Version history** - Create, list, restore, and delete document versions
|
|
23
|
+
- **Auto-compaction** - Size-based per-document compaction (no cron jobs needed)
|
|
22
24
|
|
|
23
25
|
## Architecture
|
|
24
26
|
|
|
@@ -42,7 +44,7 @@ sequenceDiagram
|
|
|
42
44
|
|
|
43
45
|
Note over Offline: Automatic retry with backoff
|
|
44
46
|
Offline->>Yjs: Get CRDT delta
|
|
45
|
-
Offline->>Convex:
|
|
47
|
+
Offline->>Convex: insert/update mutation
|
|
46
48
|
Convex->>Component: Append delta to event log
|
|
47
49
|
Convex->>Table: Insert/Update materialized doc
|
|
48
50
|
|
|
@@ -59,7 +61,7 @@ graph LR
|
|
|
59
61
|
Component[Component Storage<br/>Event Log<br/>CRDT Deltas]
|
|
60
62
|
MainTable[Main Application Table<br/>Materialized Docs<br/>Efficient Queries]
|
|
61
63
|
|
|
62
|
-
Client -->|
|
|
64
|
+
Client -->|insert/update/remove| Component
|
|
63
65
|
Component -->|also writes to| MainTable
|
|
64
66
|
MainTable -->|subscription| Client
|
|
65
67
|
```
|
|
@@ -72,14 +74,14 @@ graph LR
|
|
|
72
74
|
## Installation
|
|
73
75
|
|
|
74
76
|
```bash
|
|
75
|
-
# Using
|
|
77
|
+
# Using bun (recommended)
|
|
78
|
+
bun add @trestleinc/replicate
|
|
79
|
+
|
|
80
|
+
# Using pnpm
|
|
76
81
|
pnpm add @trestleinc/replicate
|
|
77
82
|
|
|
78
83
|
# Using npm (v7+)
|
|
79
84
|
npm install @trestleinc/replicate
|
|
80
|
-
|
|
81
|
-
# Using Bun
|
|
82
|
-
bun add @trestleinc/replicate
|
|
83
85
|
```
|
|
84
86
|
|
|
85
87
|
## Quick Start
|
|
@@ -101,31 +103,31 @@ export default app;
|
|
|
101
103
|
|
|
102
104
|
### Step 2: Define Your Schema
|
|
103
105
|
|
|
104
|
-
Use the `
|
|
106
|
+
Use the `schema.table()` helper to automatically inject required fields:
|
|
105
107
|
|
|
106
108
|
```typescript
|
|
107
109
|
// convex/schema.ts
|
|
108
110
|
import { defineSchema } from 'convex/server';
|
|
109
111
|
import { v } from 'convex/values';
|
|
110
|
-
import {
|
|
112
|
+
import { schema } from '@trestleinc/replicate/server';
|
|
111
113
|
|
|
112
114
|
export default defineSchema({
|
|
113
|
-
tasks:
|
|
115
|
+
tasks: schema.table(
|
|
114
116
|
{
|
|
115
117
|
// Your application fields only!
|
|
116
|
-
// version and timestamp are automatically injected by
|
|
118
|
+
// version and timestamp are automatically injected by schema.table()
|
|
117
119
|
id: v.string(),
|
|
118
120
|
text: v.string(),
|
|
119
121
|
isCompleted: v.boolean(),
|
|
120
122
|
},
|
|
121
|
-
(
|
|
123
|
+
(t) => t
|
|
122
124
|
.index('by_user_id', ['id']) // Required for document lookups
|
|
123
125
|
.index('by_timestamp', ['timestamp']) // Required for incremental sync
|
|
124
126
|
),
|
|
125
127
|
});
|
|
126
128
|
```
|
|
127
129
|
|
|
128
|
-
**What `
|
|
130
|
+
**What `schema.table()` does:**
|
|
129
131
|
- Automatically injects `version: v.number()` (for CRDT versioning)
|
|
130
132
|
- Automatically injects `timestamp: v.number()` (for incremental sync)
|
|
131
133
|
- You only define your business logic fields
|
|
@@ -136,88 +138,37 @@ export default defineSchema({
|
|
|
136
138
|
|
|
137
139
|
### Step 3: Create Replication Functions
|
|
138
140
|
|
|
139
|
-
|
|
141
|
+
Use `replicate()` to bind your component and create collection functions:
|
|
140
142
|
|
|
141
143
|
```typescript
|
|
142
144
|
// convex/tasks.ts
|
|
143
|
-
import {
|
|
145
|
+
import { replicate } from '@trestleinc/replicate/server';
|
|
144
146
|
import { components } from './_generated/api';
|
|
145
|
-
import {
|
|
146
|
-
|
|
147
|
-
|
|
148
|
-
|
|
149
|
-
|
|
150
|
-
|
|
151
|
-
|
|
152
|
-
|
|
153
|
-
|
|
154
|
-
|
|
155
|
-
|
|
156
|
-
|
|
157
|
-
|
|
158
|
-
|
|
159
|
-
export const insertDocument = mutation({
|
|
160
|
-
args: {
|
|
161
|
-
collectionName: v.string(),
|
|
162
|
-
documentId: v.string(),
|
|
163
|
-
crdtBytes: v.bytes(),
|
|
164
|
-
materializedDoc: v.any(),
|
|
165
|
-
version: v.number(),
|
|
166
|
-
},
|
|
167
|
-
handler: async (ctx, args) => {
|
|
168
|
-
return await insertDocumentHelper(ctx, components, 'tasks', {
|
|
169
|
-
id: args.documentId,
|
|
170
|
-
crdtBytes: args.crdtBytes,
|
|
171
|
-
materializedDoc: args.materializedDoc,
|
|
172
|
-
version: args.version,
|
|
173
|
-
});
|
|
174
|
-
},
|
|
175
|
-
});
|
|
176
|
-
|
|
177
|
-
export const updateDocument = mutation({
|
|
178
|
-
args: {
|
|
179
|
-
collectionName: v.string(),
|
|
180
|
-
documentId: v.string(),
|
|
181
|
-
crdtBytes: v.bytes(),
|
|
182
|
-
materializedDoc: v.any(),
|
|
183
|
-
version: v.number(),
|
|
184
|
-
},
|
|
185
|
-
handler: async (ctx, args) => {
|
|
186
|
-
return await updateDocumentHelper(ctx, components, 'tasks', {
|
|
187
|
-
id: args.documentId,
|
|
188
|
-
crdtBytes: args.crdtBytes,
|
|
189
|
-
materializedDoc: args.materializedDoc,
|
|
190
|
-
version: args.version,
|
|
191
|
-
});
|
|
192
|
-
},
|
|
147
|
+
import type { Task } from '../src/useTasks';
|
|
148
|
+
|
|
149
|
+
const r = replicate(components.replicate);
|
|
150
|
+
|
|
151
|
+
export const {
|
|
152
|
+
stream,
|
|
153
|
+
material,
|
|
154
|
+
insert,
|
|
155
|
+
update,
|
|
156
|
+
remove,
|
|
157
|
+
versions
|
|
158
|
+
} = r<Task>({
|
|
159
|
+
collection: 'tasks',
|
|
160
|
+
compaction: { threshold: 5_000_000 }, // Optional: size threshold for auto-compaction (default: 5MB)
|
|
193
161
|
});
|
|
162
|
+
```
|
|
194
163
|
|
|
195
|
-
|
|
196
|
-
args: {
|
|
197
|
-
collectionName: v.string(),
|
|
198
|
-
documentId: v.string(),
|
|
199
|
-
crdtBytes: v.bytes(),
|
|
200
|
-
version: v.number(),
|
|
201
|
-
},
|
|
202
|
-
handler: async (ctx, args) => {
|
|
203
|
-
return await deleteDocumentHelper(ctx, components, 'tasks', {
|
|
204
|
-
id: args.documentId,
|
|
205
|
-
crdtBytes: args.crdtBytes,
|
|
206
|
-
version: args.version,
|
|
207
|
-
});
|
|
208
|
-
},
|
|
209
|
-
});
|
|
164
|
+
**What `replicate()` generates:**
|
|
210
165
|
|
|
211
|
-
|
|
212
|
-
|
|
213
|
-
|
|
214
|
-
|
|
215
|
-
|
|
216
|
-
|
|
217
|
-
return await ctx.db.query('tasks').collect();
|
|
218
|
-
},
|
|
219
|
-
});
|
|
220
|
-
```
|
|
166
|
+
- `stream` - Real-time CRDT stream query (for client subscriptions)
|
|
167
|
+
- `material` - SSR-friendly query (for server-side rendering)
|
|
168
|
+
- `insert` - Dual-storage insert mutation (auto-compacts when threshold exceeded)
|
|
169
|
+
- `update` - Dual-storage update mutation (auto-compacts when threshold exceeded)
|
|
170
|
+
- `remove` - Dual-storage delete mutation (auto-compacts when threshold exceeded)
|
|
171
|
+
- `versions` - Version history APIs (create, list, get, restore, remove)
|
|
221
172
|
|
|
222
173
|
### Step 4: Create a Custom Hook
|
|
223
174
|
|
|
@@ -226,11 +177,7 @@ Create a hook that wraps TanStack DB with Convex collection options:
|
|
|
226
177
|
```typescript
|
|
227
178
|
// src/useTasks.ts
|
|
228
179
|
import { createCollection } from '@tanstack/react-db';
|
|
229
|
-
import {
|
|
230
|
-
convexCollectionOptions,
|
|
231
|
-
createConvexCollection,
|
|
232
|
-
type ConvexCollection,
|
|
233
|
-
} from '@trestleinc/replicate/client';
|
|
180
|
+
import { convexCollectionOptions } from '@trestleinc/replicate/client';
|
|
234
181
|
import { api } from '../convex/_generated/api';
|
|
235
182
|
import { convexClient } from './router';
|
|
236
183
|
import { useMemo } from 'react';
|
|
@@ -243,30 +190,22 @@ export interface Task {
|
|
|
243
190
|
|
|
244
191
|
// Module-level singleton to prevent multiple collection instances
|
|
245
192
|
// This ensures only one sync process runs, even across component remounts
|
|
246
|
-
let tasksCollection:
|
|
193
|
+
let tasksCollection: ReturnType<typeof createCollection<Task>>;
|
|
247
194
|
|
|
248
|
-
export function useTasks(
|
|
195
|
+
export function useTasks(
|
|
196
|
+
initialData?: { documents: Task[], checkpoint?: any, count?: number, crdtBytes?: Uint8Array }
|
|
197
|
+
) {
|
|
249
198
|
return useMemo(() => {
|
|
250
199
|
if (!tasksCollection) {
|
|
251
|
-
|
|
252
|
-
const rawCollection = createCollection(
|
|
200
|
+
tasksCollection = createCollection(
|
|
253
201
|
convexCollectionOptions<Task>({
|
|
254
202
|
convexClient,
|
|
255
|
-
api:
|
|
256
|
-
|
|
257
|
-
insertDocument: api.tasks.insertDocument,
|
|
258
|
-
updateDocument: api.tasks.updateDocument,
|
|
259
|
-
deleteDocument: api.tasks.deleteDocument,
|
|
260
|
-
},
|
|
261
|
-
collectionName: 'tasks',
|
|
203
|
+
api: api.tasks,
|
|
204
|
+
collection: 'tasks',
|
|
262
205
|
getKey: (task) => task.id,
|
|
263
|
-
initialData,
|
|
206
|
+
material: initialData,
|
|
264
207
|
})
|
|
265
208
|
);
|
|
266
|
-
|
|
267
|
-
// Step 2: Wrap with Convex offline support (Yjs + TanStack)
|
|
268
|
-
// Config is automatically extracted from rawCollection
|
|
269
|
-
tasksCollection = createConvexCollection(rawCollection);
|
|
270
209
|
}
|
|
271
210
|
return tasksCollection;
|
|
272
211
|
}, [initialData]);
|
|
@@ -331,9 +270,54 @@ export function TaskList() {
|
|
|
331
270
|
}
|
|
332
271
|
```
|
|
333
272
|
|
|
334
|
-
|
|
273
|
+
### Step 6: Server-Side Rendering (Recommended)
|
|
274
|
+
|
|
275
|
+
For frameworks that support SSR (TanStack Start, Next.js, Remix, SvelteKit), preloading data on the server is the recommended approach for instant page loads and better SEO.
|
|
276
|
+
|
|
277
|
+
**Why SSR is recommended:**
|
|
278
|
+
- **Instant page loads** - No loading spinners on first render
|
|
279
|
+
- **Better SEO** - Content visible to search engines
|
|
280
|
+
- **Reduced client work** - Data already available on hydration
|
|
281
|
+
- **Seamless transition** - Real-time sync takes over after hydration
|
|
335
282
|
|
|
336
|
-
|
|
283
|
+
**Step 1: Use the `material` query from replicate()**
|
|
284
|
+
|
|
285
|
+
The `material` query is automatically generated by `replicate()` and returns all documents for SSR hydration.
|
|
286
|
+
|
|
287
|
+
**Step 2: Load data in your route loader**
|
|
288
|
+
|
|
289
|
+
```typescript
|
|
290
|
+
// src/routes/index.tsx
|
|
291
|
+
import { createFileRoute } from '@tanstack/react-router';
|
|
292
|
+
import { ConvexHttpClient } from 'convex/browser';
|
|
293
|
+
import { api } from '../convex/_generated/api';
|
|
294
|
+
import type { Task } from '../useTasks';
|
|
295
|
+
|
|
296
|
+
const httpClient = new ConvexHttpClient(import.meta.env.VITE_CONVEX_URL);
|
|
297
|
+
|
|
298
|
+
export const Route = createFileRoute('/')({
|
|
299
|
+
loader: async () => {
|
|
300
|
+
const tasks = await httpClient.query(api.tasks.material);
|
|
301
|
+
return { tasks };
|
|
302
|
+
},
|
|
303
|
+
});
|
|
304
|
+
|
|
305
|
+
function TasksPage() {
|
|
306
|
+
const { tasks: initialTasks } = Route.useLoaderData();
|
|
307
|
+
|
|
308
|
+
// Pass initialData to your hook - no loading state on first render!
|
|
309
|
+
const collection = useTasks(initialTasks);
|
|
310
|
+
const { data: tasks } = useLiveQuery(collection);
|
|
311
|
+
|
|
312
|
+
return <TaskList tasks={tasks} />;
|
|
313
|
+
}
|
|
314
|
+
```
|
|
315
|
+
|
|
316
|
+
**Note:** If your framework doesn't support SSR, the collection works just fine without `initialData` - it will fetch data on mount and show a loading state.
|
|
317
|
+
|
|
318
|
+
## Delete Pattern: Hard Delete with Event History
|
|
319
|
+
|
|
320
|
+
Replicate uses **hard deletes** where items are physically removed from the main table, while the internal component preserves complete event history.
|
|
337
321
|
|
|
338
322
|
**Why hard delete?**
|
|
339
323
|
- Clean main table (no filtering required)
|
|
@@ -356,7 +340,7 @@ const { data: tasks } = useLiveQuery(collection);
|
|
|
356
340
|
// SSR loader - no filtering needed!
|
|
357
341
|
export const Route = createFileRoute('/')({
|
|
358
342
|
loader: async () => {
|
|
359
|
-
const tasks = await httpClient.query(api.tasks.
|
|
343
|
+
const tasks = await httpClient.query(api.tasks.material);
|
|
360
344
|
return { tasks };
|
|
361
345
|
},
|
|
362
346
|
});
|
|
@@ -364,142 +348,178 @@ export const Route = createFileRoute('/')({
|
|
|
364
348
|
|
|
365
349
|
**How it works:**
|
|
366
350
|
1. Client calls `collection.delete(id)`
|
|
367
|
-
2. `
|
|
351
|
+
2. `onRemove` handler captures Yjs deletion delta
|
|
368
352
|
3. Delta appended to component event log (history preserved)
|
|
369
353
|
4. Main table: document physically removed
|
|
370
354
|
5. Other clients notified and item removed locally
|
|
371
355
|
|
|
372
|
-
|
|
356
|
+
## Advanced Usage
|
|
357
|
+
|
|
358
|
+
### Custom Hooks and Lifecycle Events
|
|
359
|
+
|
|
360
|
+
You can customize the behavior of generated functions using optional hooks:
|
|
373
361
|
|
|
374
362
|
```typescript
|
|
375
363
|
// convex/tasks.ts
|
|
376
|
-
|
|
377
|
-
|
|
378
|
-
|
|
379
|
-
|
|
364
|
+
import { replicate } from '@trestleinc/replicate/server';
|
|
365
|
+
import { components } from './_generated/api';
|
|
366
|
+
import type { Task } from '../src/useTasks';
|
|
367
|
+
|
|
368
|
+
const r = replicate(components.replicate);
|
|
369
|
+
|
|
370
|
+
export const {
|
|
371
|
+
stream,
|
|
372
|
+
material,
|
|
373
|
+
insert,
|
|
374
|
+
update,
|
|
375
|
+
remove,
|
|
376
|
+
versions
|
|
377
|
+
} = r<Task>({
|
|
378
|
+
collection: 'tasks',
|
|
379
|
+
|
|
380
|
+
// Optional hooks for authorization and lifecycle events
|
|
381
|
+
hooks: {
|
|
382
|
+
// Permission checks (eval* hooks validate BEFORE execution, throw to deny)
|
|
383
|
+
evalRead: async (ctx, collection) => {
|
|
384
|
+
const userId = await ctx.auth.getUserIdentity();
|
|
385
|
+
if (!userId) throw new Error('Unauthorized');
|
|
386
|
+
},
|
|
387
|
+
evalWrite: async (ctx, doc) => {
|
|
388
|
+
const userId = await ctx.auth.getUserIdentity();
|
|
389
|
+
if (!userId) throw new Error('Unauthorized');
|
|
390
|
+
},
|
|
391
|
+
evalRemove: async (ctx, documentId) => {
|
|
392
|
+
const userId = await ctx.auth.getUserIdentity();
|
|
393
|
+
if (!userId) throw new Error('Unauthorized');
|
|
394
|
+
},
|
|
395
|
+
evalVersion: async (ctx, collection, documentId) => { /* auth for versioning */ },
|
|
396
|
+
evalRestore: async (ctx, collection, documentId, versionId) => { /* auth for restore */ },
|
|
397
|
+
|
|
398
|
+
// Lifecycle callbacks (on* hooks run AFTER execution)
|
|
399
|
+
onStream: async (ctx, result) => { /* after stream query */ },
|
|
400
|
+
onInsert: async (ctx, doc) => { /* after insert */ },
|
|
401
|
+
onUpdate: async (ctx, doc) => { /* after update */ },
|
|
402
|
+
onRemove: async (ctx, documentId) => { /* after remove */ },
|
|
403
|
+
onVersion: async (ctx, result) => { /* after version created */ },
|
|
404
|
+
onRestore: async (ctx, result) => { /* after restore */ },
|
|
405
|
+
|
|
406
|
+
// Transform hook (modify documents before returning)
|
|
407
|
+
transform: async (docs) => docs.filter(d => d.isPublic),
|
|
408
|
+
}
|
|
380
409
|
});
|
|
381
410
|
```
|
|
382
411
|
|
|
383
|
-
|
|
384
|
-
- **Component Storage**: Append-only event log with complete history (including deletions)
|
|
385
|
-
- **Main Table**: Current state only (deleted items removed)
|
|
412
|
+
### Rich Text / Prose Fields
|
|
386
413
|
|
|
387
|
-
|
|
414
|
+
For collaborative rich text editing, use the `schema.prose()` validator and `prose.extract()` function:
|
|
388
415
|
|
|
389
|
-
|
|
416
|
+
```typescript
|
|
417
|
+
// convex/schema.ts
|
|
418
|
+
import { schema } from '@trestleinc/replicate/server';
|
|
390
419
|
|
|
391
|
-
|
|
420
|
+
export default defineSchema({
|
|
421
|
+
notebooks: schema.table({
|
|
422
|
+
id: v.string(),
|
|
423
|
+
title: v.string(),
|
|
424
|
+
content: schema.prose(), // ProseMirror-compatible JSON
|
|
425
|
+
}),
|
|
426
|
+
});
|
|
392
427
|
|
|
393
|
-
|
|
428
|
+
// Client: Extract plain text for search
|
|
429
|
+
import { prose } from '@trestleinc/replicate/client';
|
|
394
430
|
|
|
395
|
-
|
|
396
|
-
|
|
397
|
-
|
|
398
|
-
|
|
399
|
-
return await ctx.db.query('tasks').collect();
|
|
400
|
-
},
|
|
401
|
-
});
|
|
431
|
+
const plainText = prose.extract(notebook.content);
|
|
432
|
+
|
|
433
|
+
// Client: Get editor binding for ProseMirror/TipTap
|
|
434
|
+
const binding = await collection.utils.prose(notebookId, 'content');
|
|
402
435
|
```
|
|
403
436
|
|
|
404
|
-
|
|
437
|
+
### Version History
|
|
438
|
+
|
|
439
|
+
Create and manage document version history:
|
|
405
440
|
|
|
406
441
|
```typescript
|
|
407
|
-
//
|
|
408
|
-
|
|
409
|
-
|
|
410
|
-
|
|
411
|
-
import type { Task } from '../useTasks';
|
|
442
|
+
// convex/tasks.ts
|
|
443
|
+
export const { versions } = replicate<Task>({
|
|
444
|
+
collection: 'tasks',
|
|
445
|
+
});
|
|
412
446
|
|
|
413
|
-
|
|
447
|
+
// Create a version
|
|
448
|
+
await ctx.runMutation(api.tasks.versions.create, {
|
|
449
|
+
documentId: 'task-123',
|
|
450
|
+
label: 'Before major edit',
|
|
451
|
+
createdBy: 'user-456',
|
|
452
|
+
});
|
|
414
453
|
|
|
415
|
-
|
|
416
|
-
|
|
417
|
-
|
|
418
|
-
|
|
419
|
-
},
|
|
454
|
+
// List versions
|
|
455
|
+
const versionList = await ctx.runQuery(api.tasks.versions.list, {
|
|
456
|
+
documentId: 'task-123',
|
|
457
|
+
limit: 10,
|
|
420
458
|
});
|
|
421
459
|
|
|
422
|
-
|
|
423
|
-
|
|
460
|
+
// Get a specific version
|
|
461
|
+
const version = await ctx.runQuery(api.tasks.versions.get, {
|
|
462
|
+
versionId: 'version-789',
|
|
463
|
+
});
|
|
424
464
|
|
|
425
|
-
|
|
426
|
-
|
|
427
|
-
|
|
465
|
+
// Restore a version
|
|
466
|
+
await ctx.runMutation(api.tasks.versions.restore, {
|
|
467
|
+
documentId: 'task-123',
|
|
468
|
+
versionId: 'version-789',
|
|
469
|
+
createBackup: true, // Optional: create backup before restore
|
|
470
|
+
});
|
|
428
471
|
|
|
429
|
-
|
|
430
|
-
|
|
431
|
-
|
|
472
|
+
// Delete a version
|
|
473
|
+
await ctx.runMutation(api.tasks.versions.remove, {
|
|
474
|
+
versionId: 'version-789',
|
|
475
|
+
});
|
|
432
476
|
```
|
|
433
477
|
|
|
434
|
-
###
|
|
435
|
-
|
|
436
|
-
> **WARNING:** Using `ReplicateStorage` directly only writes to the component CRDT storage layer. It does NOT implement the dual-storage pattern (no writes to main table), which means:
|
|
437
|
-
> - You cannot query this data efficiently in Convex
|
|
438
|
-
> - You lose the benefits of reactive subscriptions on materialized docs
|
|
439
|
-
> - You'll need to manually handle materialization
|
|
440
|
-
>
|
|
441
|
-
> **Recommended:** Use the replication helpers (`insertDocumentHelper`, etc.) shown in Step 3 for the full dual-storage pattern.
|
|
478
|
+
### Persistence Providers
|
|
442
479
|
|
|
443
|
-
|
|
480
|
+
Choose the right storage backend for your platform:
|
|
444
481
|
|
|
445
482
|
```typescript
|
|
446
|
-
|
|
447
|
-
import { ReplicateStorage } from '@trestleinc/replicate/client';
|
|
448
|
-
import { mutation, query } from './_generated/server';
|
|
449
|
-
import { components } from './_generated/api';
|
|
450
|
-
import { v } from 'convex/values';
|
|
451
|
-
|
|
452
|
-
interface Task {
|
|
453
|
-
id: string;
|
|
454
|
-
text: string;
|
|
455
|
-
isCompleted: boolean;
|
|
456
|
-
}
|
|
483
|
+
import { persistence, adapters } from '@trestleinc/replicate/client';
|
|
457
484
|
|
|
458
|
-
|
|
485
|
+
// Browser: IndexedDB (default, no config needed)
|
|
486
|
+
convexCollectionOptions<Task>({
|
|
487
|
+
// ... other options
|
|
488
|
+
persistence: persistence.indexeddb(),
|
|
489
|
+
});
|
|
459
490
|
|
|
460
|
-
|
|
461
|
-
|
|
462
|
-
|
|
463
|
-
|
|
464
|
-
|
|
465
|
-
|
|
466
|
-
|
|
467
|
-
return await tasksStorage.insertDocument(
|
|
468
|
-
ctx,
|
|
469
|
-
args.id,
|
|
470
|
-
args.crdtBytes,
|
|
471
|
-
args.version
|
|
472
|
-
);
|
|
473
|
-
},
|
|
491
|
+
// Browser SQLite: Uses sql.js WASM with OPFS persistence
|
|
492
|
+
// You initialize sql.js and pass the SQL object
|
|
493
|
+
import initSqlJs from 'sql.js';
|
|
494
|
+
const SQL = await initSqlJs({ locateFile: (file) => `/${file}` });
|
|
495
|
+
convexCollectionOptions<Task>({
|
|
496
|
+
// ... other options
|
|
497
|
+
persistence: await persistence.sqlite.browser(SQL, 'my-app-db'),
|
|
474
498
|
});
|
|
475
499
|
|
|
476
|
-
|
|
477
|
-
|
|
478
|
-
|
|
479
|
-
|
|
480
|
-
|
|
481
|
-
|
|
482
|
-
handler: async (ctx, args) => {
|
|
483
|
-
return await tasksStorage.updateDocument(
|
|
484
|
-
ctx,
|
|
485
|
-
args.id,
|
|
486
|
-
args.crdtBytes,
|
|
487
|
-
args.version
|
|
488
|
-
);
|
|
489
|
-
},
|
|
500
|
+
// React Native SQLite: Uses op-sqlite (native SQLite)
|
|
501
|
+
import { open } from '@op-engineering/op-sqlite';
|
|
502
|
+
const db = open({ name: 'my-app-db' });
|
|
503
|
+
convexCollectionOptions<Task>({
|
|
504
|
+
// ... other options
|
|
505
|
+
persistence: await persistence.sqlite.native(db, 'my-app-db'),
|
|
490
506
|
});
|
|
491
507
|
|
|
492
|
-
|
|
493
|
-
|
|
494
|
-
|
|
495
|
-
|
|
496
|
-
},
|
|
497
|
-
handler: async (ctx, args) => {
|
|
498
|
-
return await tasksStorage.stream(ctx, args.checkpoint, args.limit);
|
|
499
|
-
},
|
|
508
|
+
// Testing: In-memory (no persistence)
|
|
509
|
+
convexCollectionOptions<Task>({
|
|
510
|
+
// ... other options
|
|
511
|
+
persistence: persistence.memory(),
|
|
500
512
|
});
|
|
501
513
|
```
|
|
502
514
|
|
|
515
|
+
**IndexedDB** (default) - Uses y-indexeddb for Y.Doc persistence and browser-level for metadata. Browser only.
|
|
516
|
+
|
|
517
|
+
**SQLite Browser** - Uses sql.js (SQLite compiled to WASM) with OPFS persistence. You initialize sql.js yourself and pass the SQL object.
|
|
518
|
+
|
|
519
|
+
**SQLite Native** - Uses op-sqlite for React Native. You create the database and pass it.
|
|
520
|
+
|
|
521
|
+
**Memory** - No persistence, useful for testing without IndexedDB side effects.
|
|
522
|
+
|
|
503
523
|
### Logging Configuration
|
|
504
524
|
|
|
505
525
|
Configure logging for debugging and development using LogTape:
|
|
@@ -520,18 +540,6 @@ await configure({
|
|
|
520
540
|
});
|
|
521
541
|
```
|
|
522
542
|
|
|
523
|
-
Get a logger instance for custom logging:
|
|
524
|
-
|
|
525
|
-
```typescript
|
|
526
|
-
import { getLogger } from '@trestleinc/replicate/client';
|
|
527
|
-
|
|
528
|
-
const logger = getLogger(['my-module']); // Accepts string or string array
|
|
529
|
-
|
|
530
|
-
logger.info('Operation started', { userId: '123' });
|
|
531
|
-
logger.warn('Something unexpected', { reason: 'timeout' });
|
|
532
|
-
logger.error('Operation failed', { error });
|
|
533
|
-
```
|
|
534
|
-
|
|
535
543
|
## API Reference
|
|
536
544
|
|
|
537
545
|
### Client-Side (`@trestleinc/replicate/client`)
|
|
@@ -545,14 +553,17 @@ Creates collection options for TanStack DB with Yjs CRDT integration.
|
|
|
545
553
|
interface ConvexCollectionOptionsConfig<T> {
|
|
546
554
|
convexClient: ConvexClient;
|
|
547
555
|
api: {
|
|
548
|
-
stream: FunctionReference;
|
|
549
|
-
|
|
550
|
-
|
|
551
|
-
|
|
556
|
+
stream: FunctionReference; // Real-time subscription endpoint
|
|
557
|
+
insert: FunctionReference; // Insert mutation
|
|
558
|
+
update: FunctionReference; // Update mutation
|
|
559
|
+
remove: FunctionReference; // Delete mutation
|
|
552
560
|
};
|
|
553
|
-
|
|
561
|
+
collection: string;
|
|
554
562
|
getKey: (item: T) => string | number;
|
|
555
|
-
|
|
563
|
+
persistence?: Persistence; // Optional: defaults to indexeddbPersistence()
|
|
564
|
+
material?: Materialized<T>; // SSR hydration data
|
|
565
|
+
prose?: Array<keyof T>; // Optional: prose fields for rich text
|
|
566
|
+
undoCaptureTimeout?: number; // Undo stack merge window (default: 500ms)
|
|
556
567
|
}
|
|
557
568
|
```
|
|
558
569
|
|
|
@@ -560,177 +571,137 @@ interface ConvexCollectionOptionsConfig<T> {
|
|
|
560
571
|
|
|
561
572
|
**Example:**
|
|
562
573
|
```typescript
|
|
563
|
-
const
|
|
574
|
+
const collection = createCollection(
|
|
564
575
|
convexCollectionOptions<Task>({
|
|
565
576
|
convexClient,
|
|
566
|
-
api:
|
|
567
|
-
|
|
568
|
-
insertDocument: api.tasks.insertDocument,
|
|
569
|
-
updateDocument: api.tasks.updateDocument,
|
|
570
|
-
deleteDocument: api.tasks.deleteDocument,
|
|
571
|
-
},
|
|
572
|
-
collectionName: 'tasks',
|
|
577
|
+
api: api.tasks,
|
|
578
|
+
collection: 'tasks',
|
|
573
579
|
getKey: (task) => task.id,
|
|
574
|
-
initialData,
|
|
580
|
+
material: initialData,
|
|
575
581
|
})
|
|
576
582
|
);
|
|
577
|
-
|
|
578
|
-
const collection = createConvexCollection(rawCollection);
|
|
579
583
|
```
|
|
580
584
|
|
|
581
|
-
#### `
|
|
585
|
+
#### `prose.extract(proseJson)`
|
|
582
586
|
|
|
583
|
-
|
|
587
|
+
Extract plain text from ProseMirror JSON.
|
|
584
588
|
|
|
585
589
|
**Parameters:**
|
|
586
|
-
- `
|
|
590
|
+
- `proseJson` - ProseMirror JSON structure (XmlFragmentJSON)
|
|
587
591
|
|
|
588
|
-
**Returns:** `
|
|
592
|
+
**Returns:** `string` - Plain text content
|
|
589
593
|
|
|
590
594
|
**Example:**
|
|
591
595
|
```typescript
|
|
592
|
-
|
|
596
|
+
import { prose } from '@trestleinc/replicate/client';
|
|
593
597
|
|
|
594
|
-
|
|
595
|
-
collection.insert({ id: '1', text: 'Task 1', isCompleted: false });
|
|
596
|
-
collection.update('1', (draft) => { draft.isCompleted = true });
|
|
597
|
-
collection.delete('1');
|
|
598
|
+
const plainText = prose.extract(task.content);
|
|
598
599
|
```
|
|
599
600
|
|
|
600
|
-
####
|
|
601
|
-
|
|
602
|
-
Type-safe API for direct component access (advanced).
|
|
601
|
+
#### Persistence Providers
|
|
603
602
|
|
|
604
|
-
**Constructor:**
|
|
605
603
|
```typescript
|
|
606
|
-
|
|
607
|
-
```
|
|
608
|
-
|
|
609
|
-
**Methods:**
|
|
610
|
-
|
|
611
|
-
##### `insertDocument(ctx, documentId, crdtBytes, version)`
|
|
612
|
-
Insert a new document with Yjs CRDT bytes.
|
|
613
|
-
|
|
614
|
-
**Parameters:**
|
|
615
|
-
- `ctx` - Convex mutation context
|
|
616
|
-
- `documentId` - Unique document identifier
|
|
617
|
-
- `crdtBytes` - ArrayBuffer containing Yjs CRDT bytes
|
|
618
|
-
- `version` - CRDT version number
|
|
619
|
-
|
|
620
|
-
**Returns:** `Promise<{ success: boolean }>`
|
|
621
|
-
|
|
622
|
-
##### `updateDocument(ctx, documentId, crdtBytes, version)`
|
|
623
|
-
Update an existing document with Yjs CRDT bytes.
|
|
624
|
-
|
|
625
|
-
**Parameters:**
|
|
626
|
-
- `ctx` - Convex mutation context
|
|
627
|
-
- `documentId` - Unique document identifier
|
|
628
|
-
- `crdtBytes` - ArrayBuffer containing Yjs CRDT bytes
|
|
629
|
-
- `version` - CRDT version number
|
|
630
|
-
|
|
631
|
-
**Returns:** `Promise<{ success: boolean }>`
|
|
632
|
-
|
|
633
|
-
##### `deleteDocument(ctx, documentId, crdtBytes, version)`
|
|
634
|
-
Delete a document (appends deletion delta to event log).
|
|
635
|
-
|
|
636
|
-
**Parameters:**
|
|
637
|
-
- `ctx` - Convex mutation context
|
|
638
|
-
- `documentId` - Unique document identifier
|
|
639
|
-
- `crdtBytes` - ArrayBuffer containing Yjs deletion delta
|
|
640
|
-
- `version` - CRDT version number
|
|
641
|
-
|
|
642
|
-
**Returns:** `Promise<{ success: boolean }>`
|
|
643
|
-
|
|
644
|
-
##### `stream(ctx, checkpoint, limit?)`
|
|
645
|
-
Pull document changes for incremental sync.
|
|
604
|
+
import { persistence, adapters } from '@trestleinc/replicate/client';
|
|
646
605
|
|
|
647
|
-
|
|
648
|
-
|
|
649
|
-
|
|
650
|
-
|
|
606
|
+
// Persistence providers
|
|
607
|
+
persistence.indexeddb() // Browser: IndexedDB (default)
|
|
608
|
+
persistence.sqlite.browser(SQL, name) // Browser: sql.js WASM + OPFS
|
|
609
|
+
persistence.sqlite.native(db, name) // React Native: op-sqlite
|
|
610
|
+
persistence.memory() // Testing: in-memory (no persistence)
|
|
651
611
|
|
|
652
|
-
|
|
653
|
-
|
|
654
|
-
|
|
655
|
-
changes: Array<{
|
|
656
|
-
documentId: string;
|
|
657
|
-
crdtBytes: ArrayBuffer;
|
|
658
|
-
version: number;
|
|
659
|
-
timestamp: number;
|
|
660
|
-
}>;
|
|
661
|
-
checkpoint: { lastModified: number };
|
|
662
|
-
hasMore: boolean;
|
|
663
|
-
}>
|
|
612
|
+
// SQLite adapters (for advanced use)
|
|
613
|
+
adapters.sqljs // SqlJsAdapter class for browser
|
|
614
|
+
adapters.opsqlite // OPSqliteAdapter class for React Native
|
|
664
615
|
```
|
|
665
616
|
|
|
666
|
-
|
|
617
|
+
**`persistence.indexeddb()`** - Browser-only, uses y-indexeddb + browser-level.
|
|
667
618
|
|
|
668
|
-
|
|
619
|
+
**`persistence.sqlite.browser(SQL, name)`** - Browser SQLite using sql.js WASM. You initialize sql.js and pass the SQL object.
|
|
669
620
|
|
|
670
|
-
|
|
671
|
-
- `category: string | string[]` - Logger category
|
|
621
|
+
**`persistence.sqlite.native(db, name)`** - React Native SQLite using op-sqlite. You create the database and pass it.
|
|
672
622
|
|
|
673
|
-
|
|
623
|
+
**`persistence.memory()`** - In-memory, no persistence. Useful for testing.
|
|
674
624
|
|
|
675
|
-
|
|
676
|
-
```typescript
|
|
677
|
-
const logger = getLogger('my-module');
|
|
678
|
-
const logger = getLogger(['hooks', 'useTasks']);
|
|
625
|
+
#### Error Classes
|
|
679
626
|
|
|
680
|
-
|
|
627
|
+
```typescript
|
|
628
|
+
import { errors } from '@trestleinc/replicate/client';
|
|
629
|
+
|
|
630
|
+
errors.Network // Network-related failures
|
|
631
|
+
errors.IDB // IndexedDB read errors
|
|
632
|
+
errors.IDBWrite // IndexedDB write errors
|
|
633
|
+
errors.Reconciliation // Phantom document cleanup errors
|
|
634
|
+
errors.Prose // Rich text field errors
|
|
635
|
+
errors.CollectionNotReady// Collection not initialized
|
|
636
|
+
errors.NonRetriable // Errors that should not be retried (auth, validation)
|
|
681
637
|
```
|
|
682
638
|
|
|
683
639
|
### Server-Side (`@trestleinc/replicate/server`)
|
|
684
640
|
|
|
685
|
-
#### `
|
|
686
|
-
|
|
687
|
-
Insert a document into both the CRDT component and the main application table.
|
|
688
|
-
|
|
689
|
-
**Parameters:**
|
|
690
|
-
- `ctx` - Convex mutation context
|
|
691
|
-
- `components` - Generated components from Convex
|
|
692
|
-
- `tableName` - Name of the main application table
|
|
693
|
-
- `args` - `{ id: string; crdtBytes: ArrayBuffer; materializedDoc: any; version: number }`
|
|
694
|
-
|
|
695
|
-
**Returns:** `Promise<{ success: boolean; metadata: {...} }>`
|
|
641
|
+
#### `replicate(component)`
|
|
696
642
|
|
|
697
|
-
|
|
698
|
-
|
|
699
|
-
Update a document in both the CRDT component and the main application table.
|
|
643
|
+
Factory function that creates a bound replicate function for your collections.
|
|
700
644
|
|
|
701
645
|
**Parameters:**
|
|
702
|
-
- `
|
|
703
|
-
- `components` - Generated components from Convex
|
|
704
|
-
- `tableName` - Name of the main application table
|
|
705
|
-
- `args` - `{ id: string; crdtBytes: ArrayBuffer; materializedDoc: any; version: number }`
|
|
646
|
+
- `component` - Your Convex component reference (`components.replicate`)
|
|
706
647
|
|
|
707
|
-
**Returns:**
|
|
648
|
+
**Returns:** A function `<T>(config: ReplicateConfig<T>)` that generates collection operations.
|
|
708
649
|
|
|
709
|
-
|
|
650
|
+
**Example:**
|
|
651
|
+
```typescript
|
|
652
|
+
import { replicate } from '@trestleinc/replicate/server';
|
|
653
|
+
import { components } from './_generated/api';
|
|
710
654
|
|
|
711
|
-
|
|
655
|
+
const r = replicate(components.replicate);
|
|
656
|
+
export const tasks = r<Task>({ collection: 'tasks' });
|
|
657
|
+
```
|
|
712
658
|
|
|
713
|
-
|
|
714
|
-
- `ctx` - Convex mutation context
|
|
715
|
-
- `components` - Generated components from Convex
|
|
716
|
-
- `tableName` - Name of the main application table
|
|
717
|
-
- `args` - `{ id: string; crdtBytes: ArrayBuffer; version: number }`
|
|
659
|
+
#### `ReplicateConfig<T>`
|
|
718
660
|
|
|
719
|
-
|
|
661
|
+
Configuration for the bound replicate function.
|
|
720
662
|
|
|
721
|
-
|
|
663
|
+
**Config:**
|
|
664
|
+
```typescript
|
|
665
|
+
interface ReplicateConfig<T> {
|
|
666
|
+
collection: string; // Collection name (e.g., 'tasks')
|
|
722
667
|
|
|
723
|
-
|
|
668
|
+
// Optional: Auto-compaction settings
|
|
669
|
+
compaction?: {
|
|
670
|
+
threshold?: number; // Size threshold in bytes (default: 5MB / 5_000_000)
|
|
671
|
+
};
|
|
724
672
|
|
|
725
|
-
|
|
726
|
-
|
|
727
|
-
|
|
728
|
-
|
|
729
|
-
|
|
673
|
+
// Optional: Hooks for permissions and lifecycle
|
|
674
|
+
hooks?: {
|
|
675
|
+
// Permission checks (throw to reject)
|
|
676
|
+
evalRead?: (ctx, collection) => Promise<void>;
|
|
677
|
+
evalWrite?: (ctx, doc) => Promise<void>;
|
|
678
|
+
evalRemove?: (ctx, documentId) => Promise<void>;
|
|
679
|
+
evalVersion?: (ctx, collection, documentId) => Promise<void>;
|
|
680
|
+
evalRestore?: (ctx, collection, documentId, versionId) => Promise<void>;
|
|
681
|
+
|
|
682
|
+
// Lifecycle callbacks (run after operation)
|
|
683
|
+
onStream?: (ctx, result) => Promise<void>;
|
|
684
|
+
onInsert?: (ctx, doc) => Promise<void>;
|
|
685
|
+
onUpdate?: (ctx, doc) => Promise<void>;
|
|
686
|
+
onRemove?: (ctx, documentId) => Promise<void>;
|
|
687
|
+
onVersion?: (ctx, result) => Promise<void>;
|
|
688
|
+
onRestore?: (ctx, result) => Promise<void>;
|
|
689
|
+
|
|
690
|
+
// Transform hook (modify documents before returning)
|
|
691
|
+
transform?: (docs) => Promise<T[]>;
|
|
692
|
+
};
|
|
693
|
+
}
|
|
694
|
+
```
|
|
730
695
|
|
|
731
|
-
**Returns:**
|
|
696
|
+
**Returns:** Object with generated functions:
|
|
697
|
+
- `stream` - Real-time CRDT stream query
|
|
698
|
+
- `material` - SSR-friendly query for hydration
|
|
699
|
+
- `insert` - Dual-storage insert mutation (auto-compacts when threshold exceeded)
|
|
700
|
+
- `update` - Dual-storage update mutation (auto-compacts when threshold exceeded)
|
|
701
|
+
- `remove` - Dual-storage delete mutation (auto-compacts when threshold exceeded)
|
|
702
|
+
- `versions` - Version history APIs (create, list, get, restore, remove)
|
|
732
703
|
|
|
733
|
-
#### `
|
|
704
|
+
#### `schema.table(userFields, applyIndexes?)`
|
|
734
705
|
|
|
735
706
|
Automatically inject replication metadata fields (`version`, `timestamp`).
|
|
736
707
|
|
|
@@ -742,38 +713,37 @@ Automatically inject replication metadata fields (`version`, `timestamp`).
|
|
|
742
713
|
|
|
743
714
|
**Example:**
|
|
744
715
|
```typescript
|
|
745
|
-
|
|
716
|
+
import { schema } from '@trestleinc/replicate/server';
|
|
717
|
+
|
|
718
|
+
tasks: schema.table(
|
|
746
719
|
{
|
|
747
720
|
id: v.string(),
|
|
748
721
|
text: v.string(),
|
|
749
722
|
},
|
|
750
|
-
(
|
|
723
|
+
(t) => t
|
|
751
724
|
.index('by_user_id', ['id'])
|
|
752
725
|
.index('by_timestamp', ['timestamp'])
|
|
753
726
|
)
|
|
754
727
|
```
|
|
755
728
|
|
|
756
|
-
|
|
757
|
-
|
|
758
|
-
#### `loadCollection<T>(httpClient, config)`
|
|
729
|
+
#### `schema.prose()`
|
|
759
730
|
|
|
760
|
-
|
|
731
|
+
Validator for ProseMirror-compatible JSON fields.
|
|
761
732
|
|
|
762
|
-
**
|
|
733
|
+
**Returns:** Convex validator for prose fields
|
|
763
734
|
|
|
764
|
-
**
|
|
765
|
-
|
|
766
|
-
|
|
767
|
-
|
|
768
|
-
**Returns:** `Promise<ReadonlyArray<T>>`
|
|
735
|
+
**Example:**
|
|
736
|
+
```typescript
|
|
737
|
+
content: schema.prose() // Validates ProseMirror JSON structure
|
|
738
|
+
```
|
|
769
739
|
|
|
770
740
|
## Performance
|
|
771
741
|
|
|
772
742
|
### Storage Performance
|
|
773
743
|
|
|
774
|
-
- **
|
|
744
|
+
- **Swappable persistence** - IndexedDB (browser), SQLite (React Native), or in-memory (testing)
|
|
775
745
|
- **Yjs** CRDT operations are extremely fast (96% smaller than Automerge)
|
|
776
|
-
- **TanStack
|
|
746
|
+
- **TanStack DB** provides optimistic updates and reactive state management
|
|
777
747
|
- **Indexed queries** in Convex for fast incremental sync
|
|
778
748
|
|
|
779
749
|
### Sync Performance
|
|
@@ -787,13 +757,13 @@ Load collection data during SSR for instant page loads.
|
|
|
787
757
|
|
|
788
758
|
- **TanStack coordination** - Built-in multi-tab sync via BroadcastChannel
|
|
789
759
|
- **Yjs shared state** - Single source of truth per browser
|
|
790
|
-
- **
|
|
760
|
+
- **Leader election** - Only one tab runs sync operations
|
|
791
761
|
|
|
792
762
|
## Offline Behavior
|
|
793
763
|
|
|
794
764
|
### How It Works
|
|
795
765
|
|
|
796
|
-
- **Writes** - Queue locally in Yjs CRDT, sync when online
|
|
766
|
+
- **Writes** - Queue locally in Yjs CRDT, sync when online
|
|
797
767
|
- **Reads** - Always work from local TanStack DB cache (instant!)
|
|
798
768
|
- **UI** - Fully functional with optimistic updates
|
|
799
769
|
- **Conflicts** - Auto-resolved by Yjs CRDTs (conflict-free!)
|
|
@@ -807,63 +777,29 @@ Load collection data during SSR for instant page loads.
|
|
|
807
777
|
|
|
808
778
|
## Examples
|
|
809
779
|
|
|
810
|
-
|
|
780
|
+
### Interval - Linear-style Issue Tracker
|
|
811
781
|
|
|
812
|
-
|
|
813
|
-
- `src/useTasks.ts` - Hook with TanStack DB integration
|
|
814
|
-
- `src/routes/index.tsx` - Component usage with SSR
|
|
815
|
-
- `src/routes/__root.tsx` - Logging configuration
|
|
816
|
-
- `convex/tasks.ts` - Replication functions using dual-storage helpers
|
|
817
|
-
- `convex/schema.ts` - Schema with `replicatedTable` helper
|
|
782
|
+
A full-featured offline-first issue tracker built with Replicate, demonstrating real-world usage patterns.
|
|
818
783
|
|
|
819
|
-
**
|
|
820
|
-
```bash
|
|
821
|
-
cd examples/tanstack-start
|
|
822
|
-
pnpm install
|
|
823
|
-
pnpm run dev
|
|
824
|
-
```
|
|
784
|
+
🔗 **Live Demo:** [interval.robelest.com](https://interval.robelest.com)
|
|
825
785
|
|
|
826
|
-
|
|
786
|
+
📦 **Source Code:** [github.com/robelest/interval](https://github.com/robelest/interval)
|
|
827
787
|
|
|
828
|
-
|
|
788
|
+
**Features demonstrated:**
|
|
789
|
+
- Offline-first with SQLite persistence (sql.js + OPFS)
|
|
790
|
+
- Rich text editing with TipTap + Yjs collaboration
|
|
791
|
+
- PWA with custom service worker
|
|
792
|
+
- Real-time sync across devices
|
|
793
|
+
- Search with client-side text extraction (`prose.extract()`)
|
|
829
794
|
|
|
830
|
-
|
|
831
|
-
pnpm run build # Build package using Rslib
|
|
832
|
-
pnpm run clean # Remove build artifacts
|
|
833
|
-
pnpm run typecheck # Type check
|
|
834
|
-
```
|
|
835
|
-
|
|
836
|
-
### Code Quality
|
|
837
|
-
|
|
838
|
-
```bash
|
|
839
|
-
pnpm run check # Lint + format check (dry run)
|
|
840
|
-
pnpm run check:fix # Auto-fix all issues (run before committing)
|
|
841
|
-
pnpm run lint # Lint only
|
|
842
|
-
pnpm run lint:fix # Auto-fix lint issues
|
|
843
|
-
pnpm run format # Format only
|
|
844
|
-
pnpm run format:check # Check formatting
|
|
845
|
-
```
|
|
846
|
-
|
|
847
|
-
### Running Example
|
|
795
|
+
## Development
|
|
848
796
|
|
|
849
797
|
```bash
|
|
850
|
-
|
|
798
|
+
bun run build # Build with Rslib (includes ESLint + TypeScript checking)
|
|
799
|
+
bun run dev # Watch mode
|
|
800
|
+
bun run clean # Remove build artifacts
|
|
851
801
|
```
|
|
852
802
|
|
|
853
|
-
## Roadmap
|
|
854
|
-
|
|
855
|
-
- [ ] Partial sync (sync subset of collection)
|
|
856
|
-
- [ ] Delta sync (only sync changed fields)
|
|
857
|
-
- [ ] Encryption at rest
|
|
858
|
-
- [ ] Attachment support (files, images)
|
|
859
|
-
- [x] React Native support (works with Yjs v0.3.0+)
|
|
860
|
-
- [ ] Advanced Yjs features (rich text editing, shared cursors)
|
|
861
|
-
- [ ] Recovery features (restore deleted items from event log)
|
|
862
|
-
|
|
863
|
-
## Contributing
|
|
864
|
-
|
|
865
|
-
Contributions welcome! Please see `CLAUDE.md` for coding standards.
|
|
866
|
-
|
|
867
803
|
## License
|
|
868
804
|
|
|
869
805
|
Apache-2.0 License - see [LICENSE](./LICENSE) file for details.
|