foldkit-sync 0.7.0 → 0.8.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +424 -898
- package/package.json +8 -8
package/README.md
CHANGED
|
@@ -2,103 +2,74 @@
|
|
|
2
2
|
|
|
3
3
|
Local-first replication for part of a Foldkit application.
|
|
4
4
|
|
|
5
|
-
|
|
6
|
-
|
|
7
|
-
|
|
8
|
-
|
|
9
|
-
|
|
10
|
-
|
|
11
|
-
`Message`, and `update` remain the state machine. Sync does not introduce a
|
|
12
|
-
second reducer. You declare which Model slice is shared and which existing
|
|
13
|
-
Messages are durable; Sync derives replay, persistence, protocol codecs, and the
|
|
14
|
-
server journal contract from that declaration.
|
|
15
|
-
|
|
16
|
-
> **Sync turns a subset of your existing Foldkit Messages into durable
|
|
17
|
-
> operations. They still run through your normal `update`; Sync adds persistence,
|
|
18
|
-
> optimistic replay, server ordering, and reconciliation around them.**
|
|
19
|
-
|
|
20
|
-
**Use it when** a user should keep editing offline, the edit must survive a
|
|
21
|
-
reload or network failure, and several replicas must eventually converge under
|
|
22
|
-
one authoritative server order. **Do not use it for** server-owned disposable
|
|
23
|
-
facts — use [`foldkit-remote`](../remote) for those — or for peer-to-peer / CRDT
|
|
24
|
-
replication where every peer independently accepts writes.
|
|
5
|
+
You name a slice of the Model that several devices share, and the existing
|
|
6
|
+
Messages that change it. From then on an edit applies at once, is saved in a
|
|
7
|
+
local outbox, reaches the server when the network allows, and lands in the same
|
|
8
|
+
order on every device. Nothing about the application changes shape: `Model`,
|
|
9
|
+
`Message`, and `update` stay the one state machine. Sync wraps them; it never
|
|
10
|
+
adds a second reducer.
|
|
25
11
|
|
|
26
|
-
|
|
27
|
-
|
|
12
|
+
> **A durable Message is one of your existing Messages, recorded.** It still
|
|
13
|
+
> runs through your `update`. Sync adds the outbox, the optimistic view, the
|
|
14
|
+
> exchange with the server, and the rebase when the server's answer arrives.
|
|
28
15
|
|
|
29
|
-
|
|
16
|
+
It is the client half of [replicated Foldkit state](../../docs/replication.md).
|
|
17
|
+
[`foldkit-durable`](../durable) is the server half, an ordered journal on
|
|
18
|
+
SQLite. Either works without the other.
|
|
30
19
|
|
|
31
|
-
|
|
20
|
+
## Is this the right package?
|
|
32
21
|
|
|
33
|
-
|
|
|
34
|
-
| --- | --- |
|
|
35
|
-
|
|
|
36
|
-
|
|
|
37
|
-
|
|
|
38
|
-
|
|
|
22
|
+
| The state | Owner | Use |
|
|
23
|
+
| --- | --- | --- |
|
|
24
|
+
| A route, a selection, a transient error | the local Model | plain Foldkit |
|
|
25
|
+
| Facts the server owns and the client may refetch | the server | [`foldkit-remote`](../remote) |
|
|
26
|
+
| Edits the user makes that must survive offline and converge | the durable log | **`foldkit-sync`** |
|
|
27
|
+
| The authoritative order and snapshot on the server | the server journal | [`foldkit-durable`](../durable) |
|
|
39
28
|
|
|
40
|
-
The
|
|
29
|
+
The Remote and Sync line is the one worth memorizing. Remote caches a fact the
|
|
30
|
+
server owns, so refetching is recovery. Sync records an edit the user authored,
|
|
31
|
+
so the outbox is user data and replay is recovery.
|
|
41
32
|
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
server owns the fact
|
|
45
|
-
client caches it
|
|
46
|
-
refetch is recovery
|
|
47
|
-
|
|
48
|
-
Sync
|
|
49
|
-
client creates the durable operation
|
|
50
|
-
operation must survive offline
|
|
51
|
-
replay + reconciliation is recovery
|
|
52
|
-
```
|
|
33
|
+
Sync assumes one server order per document. It is not peer-to-peer and not a
|
|
34
|
+
CRDT; every peer accepting writes without a server is a different design.
|
|
53
35
|
|
|
54
|
-
|
|
55
|
-
time. Those observation boundaries do not change who owns each datum.
|
|
56
|
-
|
|
57
|
-
## The replica mental model
|
|
58
|
-
|
|
59
|
-
A replica has two pieces of state:
|
|
36
|
+
## The model in one equation
|
|
60
37
|
|
|
61
38
|
```text
|
|
62
|
-
|
|
63
|
-
+
|
|
64
|
-
pending local operations
|
|
65
|
-
=
|
|
66
|
-
optimistic shared state the UI sees
|
|
39
|
+
what the user sees = committed snapshot + pending local edits, replayed
|
|
67
40
|
```
|
|
68
41
|
|
|
69
|
-
|
|
42
|
+
The server has committed `A B C`; this device has typed `D E` since:
|
|
70
43
|
|
|
71
44
|
```text
|
|
72
|
-
server
|
|
73
|
-
|
|
74
|
-
|
|
75
|
-
visible state = replay(A, B, C, D, E)
|
|
45
|
+
server: A B C
|
|
46
|
+
pending: D E
|
|
47
|
+
visible: replay(A, B, C, D, E)
|
|
76
48
|
```
|
|
77
49
|
|
|
78
|
-
|
|
79
|
-
|
|
50
|
+
Another device gets `X` in before `D` reaches the server. After the next
|
|
51
|
+
exchange:
|
|
80
52
|
|
|
81
53
|
```text
|
|
82
|
-
|
|
83
|
-
|
|
84
|
-
|
|
85
|
-
new visible state = replay(A, B, C, X, D, E)
|
|
54
|
+
server: A B C X
|
|
55
|
+
pending: D E
|
|
56
|
+
visible: replay(A, B, C, X, D, E)
|
|
86
57
|
```
|
|
87
58
|
|
|
88
|
-
|
|
89
|
-
|
|
59
|
+
`D` and `E` never waited. The base moved underneath them and they were replayed
|
|
60
|
+
on top. That one move, **rebase**, is also how a rejection rolls back: drop the
|
|
61
|
+
rejected edit from `pending` and replay the rest.
|
|
90
62
|
|
|
91
|
-
|
|
63
|
+
An edit passes three milestones, and each means something different:
|
|
92
64
|
|
|
93
|
-
|
|
94
|
-
|
|
95
|
-
|
|
96
|
-
|
|
97
|
-
|
|
65
|
+
| Milestone | Where | What it proves |
|
|
66
|
+
| --- | --- | --- |
|
|
67
|
+
| **visible** | the Model, at once | nothing yet: the user saw it |
|
|
68
|
+
| **saved** | the local outbox | a reload or a dropped connection cannot lose it |
|
|
69
|
+
| **committed** | the server journal | every other device will see it in this order |
|
|
98
70
|
|
|
99
|
-
|
|
100
|
-
|
|
101
|
-
replica status when deciding what the UI may call saved.
|
|
71
|
+
A UI that says "saved" should read the second, and one that says "shared"
|
|
72
|
+
the third. Both are observable; see [what to show the user](#what-to-show-the-user).
|
|
102
73
|
|
|
103
74
|
## Install
|
|
104
75
|
|
|
@@ -107,23 +78,25 @@ pnpm add foldkit-sync
|
|
|
107
78
|
```
|
|
108
79
|
|
|
109
80
|
`foldkit` and `effect` are peer dependencies; `foldkit-surface` comes with the
|
|
110
|
-
package.
|
|
81
|
+
package. The browser storage is IndexedDB. The server needs
|
|
82
|
+
[`foldkit-durable`](../durable) or your own implementation of
|
|
83
|
+
[the exchange](#the-exchange).
|
|
111
84
|
|
|
112
|
-
## Sixty seconds:
|
|
85
|
+
## Sixty seconds: say what is shared
|
|
113
86
|
|
|
114
|
-
Start
|
|
87
|
+
Start from an ordinary application. The example keeps a todo list and a
|
|
88
|
+
selection; only the list is shared.
|
|
115
89
|
|
|
116
90
|
```ts
|
|
117
91
|
import { Option, Schema } from 'effect'
|
|
118
92
|
import { defineMessageUnion } from 'foldkit/message'
|
|
93
|
+
import { modifyFields } from 'foldkit/struct'
|
|
119
94
|
import type * as Update from 'foldkit/update'
|
|
120
95
|
import { MessageSet, Projection, Surface } from 'foldkit-surface'
|
|
121
96
|
import { DocumentId, Sync } from 'foldkit-sync'
|
|
122
97
|
|
|
123
98
|
const Model = Schema.Struct({
|
|
124
|
-
todos: Schema.Array(
|
|
125
|
-
Schema.Struct({ id: Schema.String, title: Schema.String }),
|
|
126
|
-
),
|
|
99
|
+
todos: Schema.Array(Schema.Struct({ id: Schema.String, title: Schema.String })),
|
|
127
100
|
selectedTodoId: Schema.Option(Schema.String),
|
|
128
101
|
lastError: Schema.Option(Schema.String),
|
|
129
102
|
})
|
|
@@ -136,30 +109,22 @@ const Message = defineMessageUnion({
|
|
|
136
109
|
})
|
|
137
110
|
type Message = typeof Message.Type
|
|
138
111
|
|
|
139
|
-
const initial: Model = {
|
|
140
|
-
todos: [],
|
|
141
|
-
selectedTodoId: Option.none(),
|
|
142
|
-
lastError: Option.none(),
|
|
143
|
-
}
|
|
112
|
+
const initial: Model = { todos: [], selectedTodoId: Option.none(), lastError: Option.none() }
|
|
144
113
|
|
|
145
114
|
type Return = Update.Return<Model, Message>
|
|
146
115
|
|
|
147
|
-
// This is still the application's one transition function.
|
|
148
116
|
const update = (model: Model, message: Message): Return =>
|
|
149
117
|
Message.match<Return>(message, {
|
|
150
118
|
CreatedTodo: ({ id, title }) => ({
|
|
151
|
-
model:
|
|
119
|
+
model: modifyFields(model, { todos: () => [...model.todos, { id, title }] }),
|
|
152
120
|
}),
|
|
153
121
|
RenamedTodo: ({ id, title }) => ({
|
|
154
|
-
model: {
|
|
155
|
-
...
|
|
156
|
-
|
|
157
|
-
todo.id === id ? { ...todo, title } : todo,
|
|
158
|
-
),
|
|
159
|
-
},
|
|
122
|
+
model: modifyFields(model, {
|
|
123
|
+
todos: () => model.todos.map(todo => (todo.id === id ? { ...todo, title } : todo)),
|
|
124
|
+
}),
|
|
160
125
|
}),
|
|
161
126
|
SelectedTodo: ({ id }) => ({
|
|
162
|
-
model:
|
|
127
|
+
model: modifyFields(model, { selectedTodoId: () => Option.some(id) }),
|
|
163
128
|
}),
|
|
164
129
|
})
|
|
165
130
|
|
|
@@ -167,493 +132,190 @@ const App = Surface.application({ Model, Message, initial, update })
|
|
|
167
132
|
|
|
168
133
|
const TodoSync = Sync.forApplication(App).make({
|
|
169
134
|
documentId: DocumentId.make('todos'),
|
|
170
|
-
|
|
171
|
-
// This slice of Model is replicated.
|
|
172
|
-
// The writable Projection lets Sync install checkpoints/reconciled state back
|
|
173
|
-
// into the application Model without owning a second copy of the schema.
|
|
174
135
|
shared: Projection.pick(App.model.todos),
|
|
175
|
-
|
|
176
|
-
// Only these existing Messages become durable operations.
|
|
177
|
-
durable: MessageSet.make(App, [
|
|
178
|
-
Message.CreatedTodo,
|
|
179
|
-
Message.RenamedTodo,
|
|
180
|
-
]),
|
|
136
|
+
durable: MessageSet.make(App, [Message.CreatedTodo, Message.RenamedTodo]),
|
|
181
137
|
})
|
|
182
138
|
```
|
|
183
139
|
|
|
184
|
-
Read
|
|
140
|
+
Read the declaration as a table:
|
|
185
141
|
|
|
186
142
|
```text
|
|
187
|
-
|
|
188
|
-
|
|
189
|
-
|
|
190
|
-
lastError -> local only
|
|
191
|
-
|
|
192
|
-
Messages
|
|
193
|
-
CreatedTodo -> durable
|
|
194
|
-
RenamedTodo -> durable
|
|
195
|
-
SelectedTodo -> local only
|
|
196
|
-
```
|
|
197
|
-
|
|
198
|
-
`Sync.forApplication(App)` derives the shared schema, initial shared snapshot,
|
|
199
|
-
replay function, operation codecs, and server journal contract from the same
|
|
200
|
-
application declaration. There is no parallel sync-specific version of
|
|
201
|
-
`update` to keep aligned.
|
|
202
|
-
|
|
203
|
-
### Declaration is not a running replica
|
|
204
|
-
|
|
205
|
-
The example above defines codecs and replay rules. It has not opened storage,
|
|
206
|
-
created an outbox, connected to a server, or mounted a view. For the application
|
|
207
|
-
path, continue with [mounting](#mounting-the-full-foldkit-application). Use
|
|
208
|
-
[the replica API](#running-a-replica) when you need to control that lifecycle.
|
|
209
|
-
|
|
210
|
-
`SelectedTodo` stays local because it is outside the durable MessageSet.
|
|
211
|
-
Adding a Message to that set is a semantic decision: its update must be safe
|
|
212
|
-
to replay and confined to the shared projection. An intent that starts a
|
|
213
|
-
Command should instead produce a deterministic fact, as described next.
|
|
214
|
-
|
|
215
|
-
## Wiring: the contract joins an assembly
|
|
143
|
+
todos shared replicated, persisted, journaled
|
|
144
|
+
selectedTodoId local this device only
|
|
145
|
+
lastError local
|
|
216
146
|
|
|
217
|
-
|
|
218
|
-
|
|
219
|
-
|
|
220
|
-
```ts
|
|
221
|
-
const wiring = TodoSync.wiring()
|
|
147
|
+
CreatedTodo durable recorded in the outbox, committed by the server
|
|
148
|
+
RenamedTodo durable
|
|
149
|
+
SelectedTodo local runs through update and goes nowhere
|
|
222
150
|
```
|
|
223
151
|
|
|
224
|
-
|
|
225
|
-
|
|
226
|
-
|
|
227
|
-
|
|
228
|
-
## What makes a Message durable?
|
|
152
|
+
From that, `make` derives the shared codec, the initial shared value, the
|
|
153
|
+
operation envelope, replay (your `update` on the shared slice), and the
|
|
154
|
+
server's [journal contract](#the-server). You write none of them.
|
|
229
155
|
|
|
230
|
-
|
|
231
|
-
|
|
156
|
+
`TodoSync` is a value. It has opened no storage, connected to nothing, and
|
|
157
|
+
rendered nothing. The next section does that.
|
|
232
158
|
|
|
233
|
-
|
|
234
|
-
shared projection.
|
|
159
|
+
## Run it in the browser
|
|
235
160
|
|
|
236
|
-
|
|
237
|
-
|
|
238
|
-
|
|
239
|
-
- returns a Command — a live external effect cannot be replayed later; or
|
|
240
|
-
- changes a field outside the shared projection — that change would disappear
|
|
241
|
-
when only the shared slice is persisted/replayed.
|
|
242
|
-
|
|
243
|
-
For example:
|
|
244
|
-
|
|
245
|
-
```text
|
|
246
|
-
RequestedChargeCard
|
|
247
|
-
-> runs a Command / talks to a provider
|
|
248
|
-
-> local intent, NOT durable
|
|
249
|
-
|
|
250
|
-
CardCharged
|
|
251
|
-
-> pure fact about state
|
|
252
|
-
-> durable and replayable
|
|
253
|
-
```
|
|
254
|
-
|
|
255
|
-
A common pattern is therefore:
|
|
256
|
-
|
|
257
|
-
```text
|
|
258
|
-
local intent Message
|
|
259
|
-
|
|
|
260
|
-
v
|
|
261
|
-
Command / nondeterministic work
|
|
262
|
-
|
|
|
263
|
-
v
|
|
264
|
-
durable fact Message
|
|
265
|
-
```
|
|
266
|
-
|
|
267
|
-
Replay runs the durable fact through the same application `update` on every
|
|
268
|
-
replica. If an application genuinely needs different replay semantics, `make`
|
|
269
|
-
accepts a custom `replay`, but that custom replay does not get the derived
|
|
270
|
-
guardrails.
|
|
271
|
-
|
|
272
|
-
When the work between intent and fact needs nothing but the Model — an id minted
|
|
273
|
-
from a local counter, say — return the fact as `Sync.fact(message)` instead of an
|
|
274
|
-
ordinary Command. `Sync.mount` applies it straight after the intent's `update`,
|
|
275
|
-
in the same transition, and persists it if it is durable. An ordinary Command's
|
|
276
|
-
Message arrives later, so a second intent dispatched in the meantime would read
|
|
277
|
-
the Model from before the fact and could mint the same id. Outside `mount`, or
|
|
278
|
-
once a parent has mapped it, `Sync.fact` is an ordinary Command that yields the
|
|
279
|
-
Message.
|
|
280
|
-
|
|
281
|
-
## Running a replica
|
|
282
|
-
|
|
283
|
-
The low-level runtime is a `Replica`. It owns the local outbox, committed shared
|
|
284
|
-
snapshot, cursor, optimistic projection, and synchronization state.
|
|
161
|
+
Three steps: open storage and a replica, mount the application over it, start
|
|
162
|
+
the exchange loop.
|
|
285
163
|
|
|
286
164
|
```ts
|
|
287
|
-
import { Effect } from 'effect'
|
|
165
|
+
import { Effect, Scope } from 'effect'
|
|
288
166
|
import { ReplicaId, Sync } from 'foldkit-sync'
|
|
289
167
|
|
|
290
|
-
const
|
|
291
|
-
// One durable storage identity per document + replica/writer.
|
|
292
|
-
const storage = yield* Sync.indexedDb('todos/tab-1')
|
|
293
|
-
|
|
294
|
-
const replica = yield* TodoSync.openReplica(
|
|
295
|
-
ReplicaId.make('tab-1'),
|
|
296
|
-
storage,
|
|
297
|
-
)
|
|
298
|
-
|
|
299
|
-
// submit validates/replays first, then persists the operation in the outbox.
|
|
300
|
-
yield* replica.submit(
|
|
301
|
-
Message.CreatedTodo({ id: 't1', title: 'Milk' }),
|
|
302
|
-
)
|
|
303
|
-
|
|
304
|
-
// This already includes the pending operation. No server round trip required.
|
|
305
|
-
const optimistic = yield* replica.shared
|
|
306
|
-
|
|
307
|
-
return optimistic
|
|
308
|
-
}).pipe(Effect.scoped)
|
|
309
|
-
```
|
|
310
|
-
|
|
311
|
-
`replica.submit(message)` does not mean “send this now.” It means:
|
|
312
|
-
|
|
313
|
-
1. verify the Message is valid and durable;
|
|
314
|
-
2. replay it against the current optimistic shared state;
|
|
315
|
-
3. assign a stable local operation identity;
|
|
316
|
-
4. persist it to the local outbox;
|
|
317
|
-
5. make the new optimistic projection observable; and
|
|
318
|
-
6. wake the exchange loop if one is running.
|
|
319
|
-
|
|
320
|
-
A Message replay rejects never enters the outbox.
|
|
321
|
-
|
|
322
|
-
## Optimistic state and rebasing
|
|
323
|
-
|
|
324
|
-
Internally the replica keeps:
|
|
325
|
-
|
|
326
|
-
```text
|
|
327
|
-
committed
|
|
328
|
-
the authoritative shared snapshot up to cursor N
|
|
329
|
-
|
|
330
|
-
pending
|
|
331
|
-
locally-authored operations not yet resolved by the server
|
|
332
|
-
```
|
|
333
|
-
|
|
334
|
-
The value returned by `replica.shared` is:
|
|
335
|
-
|
|
336
|
-
```text
|
|
337
|
-
pending.reduce(replay, committed)
|
|
338
|
-
```
|
|
339
|
-
|
|
340
|
-
That simple rule gives optimistic UI and reconciliation the same semantics.
|
|
341
|
-
There is no special “optimistic reducer.”
|
|
342
|
-
|
|
343
|
-
When synchronization advances the committed base, Sync removes pending
|
|
344
|
-
operations that became committed, were explicitly acknowledged, or were
|
|
345
|
-
rejected, then the remaining outbox is replayed over the new committed state.
|
|
346
|
-
|
|
347
|
-
This also explains why durable Messages have to be replay-safe: a pending
|
|
348
|
-
operation may run many times as the authoritative base changes beneath it.
|
|
349
|
-
|
|
350
|
-
## Synchronization
|
|
351
|
-
|
|
352
|
-
A replica exchanges these with the server:
|
|
353
|
-
|
|
354
|
-
```text
|
|
355
|
-
request
|
|
356
|
-
current cursor
|
|
357
|
-
pending operations
|
|
358
|
-
the server's epoch, once the replica has heard one
|
|
359
|
-
|
|
360
|
-
response
|
|
361
|
-
newly committed operations
|
|
362
|
-
acknowledgements
|
|
363
|
-
rejections
|
|
364
|
-
optional checkpoint
|
|
365
|
-
optional more (the replica asks again at once)
|
|
366
|
-
optional epoch
|
|
367
|
-
```
|
|
368
|
-
|
|
369
|
-
`replica.synchronize` performs one exchange through the `Transport` Effect
|
|
370
|
-
service:
|
|
168
|
+
const container = document.getElementById('app')!
|
|
371
169
|
|
|
372
|
-
|
|
373
|
-
const
|
|
374
|
-
|
|
170
|
+
// One IndexedDB database per document and tab; its connection lives for the page.
|
|
171
|
+
const scope = Effect.runSync(Scope.make())
|
|
172
|
+
const storage = Effect.runSync(
|
|
173
|
+
Effect.provideService(Sync.indexedDb('todos/tab-1'), Scope.Scope, scope),
|
|
375
174
|
)
|
|
376
|
-
|
|
175
|
+
const replica = Effect.runSync(TodoSync.openReplica(ReplicaId.make('tab-1'), storage))
|
|
377
176
|
|
|
378
|
-
|
|
379
|
-
|
|
380
|
-
|
|
381
|
-
|
|
382
|
-
|
|
383
|
-
|
|
384
|
-
|
|
177
|
+
const mounted = Sync.mount(App, TodoSync, {
|
|
178
|
+
replica,
|
|
179
|
+
container,
|
|
180
|
+
view: (model, h) => ({
|
|
181
|
+
title: 'Todos',
|
|
182
|
+
body: h.ul([], model.todos.map(todo => h.li([], [todo.title]))),
|
|
183
|
+
}),
|
|
184
|
+
onPersistenceFailure: (model, error) =>
|
|
185
|
+
modifyFields(model, { lastError: () => Option.some(error.message) }),
|
|
186
|
+
})
|
|
385
187
|
|
|
386
|
-
```ts
|
|
387
188
|
Effect.runFork(
|
|
388
|
-
Effect.provide(
|
|
389
|
-
replica.start,
|
|
390
|
-
Sync.transport.socket({ url: 'wss://example.com/sync' }),
|
|
391
|
-
),
|
|
189
|
+
Effect.provide(replica.start, Sync.transport.socket({ url: 'wss://example.com/sync' })),
|
|
392
190
|
)
|
|
393
|
-
```
|
|
394
|
-
|
|
395
|
-
The transport does not decide reconciliation. It only carries the exchange;
|
|
396
|
-
the replica validates the response and applies the protocol rules.
|
|
397
|
-
|
|
398
|
-
### Acknowledgements
|
|
399
191
|
|
|
400
|
-
|
|
401
|
-
it may leave the outbox even if its committed payload is not repeated in this
|
|
402
|
-
response. While a paged response says there is `more`, an acknowledged operation
|
|
403
|
-
stays pending, because its committed copy may be on a later page; it leaves once
|
|
404
|
-
that copy arrives or the last page has been read.
|
|
405
|
-
|
|
406
|
-
### Rejections
|
|
407
|
-
|
|
408
|
-
A rejection says the optimistic operation did not commit. Sync removes it from
|
|
409
|
-
the pending set and recomputes the optimistic state from the authoritative base
|
|
410
|
-
plus whatever pending operations remain.
|
|
411
|
-
|
|
412
|
-
```text
|
|
413
|
-
before response
|
|
414
|
-
committed: A B C
|
|
415
|
-
pending: D E
|
|
416
|
-
visible: A B C D E
|
|
417
|
-
|
|
418
|
-
server rejects D
|
|
419
|
-
|
|
420
|
-
committed: A B C
|
|
421
|
-
pending: E
|
|
422
|
-
visible: A B C E
|
|
423
|
-
```
|
|
424
|
-
|
|
425
|
-
`replica.status.rejected` keeps recent rejected operation ids so the UI can
|
|
426
|
-
explain that an optimistic edit was refused.
|
|
427
|
-
|
|
428
|
-
### Checkpoints
|
|
429
|
-
|
|
430
|
-
A server may compact old history. If the replica is too far behind to replay the
|
|
431
|
-
missing prefix, the response can include a checkpoint:
|
|
432
|
-
|
|
433
|
-
```text
|
|
434
|
-
checkpoint = authoritative shared snapshot + cursor
|
|
192
|
+
mounted.dispatch(Message.CreatedTodo({ id: crypto.randomUUID(), title: 'Milk' }))
|
|
435
193
|
```
|
|
436
194
|
|
|
437
|
-
|
|
438
|
-
pending operations on top. A checkpoint behind the replica's current cursor is
|
|
439
|
-
rejected with `CheckpointRegressionError` rather than silently moving history
|
|
440
|
-
backwards.
|
|
195
|
+
What each call does, and does not do:
|
|
441
196
|
|
|
442
|
-
|
|
197
|
+
- **`Sync.indexedDb(name)`** opens one database. Name it by document and
|
|
198
|
+
writer: two tabs on one name race, and the loser fails its write rather than
|
|
199
|
+
merging two histories.
|
|
200
|
+
- **`openReplica(id, storage)`** reads back the committed snapshot, cursor, and
|
|
201
|
+
outbox saved last time, or starts empty. It does no network I/O.
|
|
202
|
+
- **`Sync.mount(App, TodoSync, …)`** runs the ordinary Foldkit runtime with
|
|
203
|
+
your `view` and `update`. A durable Message applies through `update` at once,
|
|
204
|
+
then a Command persists it; if the persist fails, the edit is reverted and
|
|
205
|
+
`onPersistenceFailure` says so in the Model. A local Message is untouched.
|
|
206
|
+
- **`replica.start`** is the exchange loop: once now, then after every submit
|
|
207
|
+
and every notice from the server. A failed exchange is retried on a backoff
|
|
208
|
+
from 0.5 s to 30 s, or at once on the next edit. The loop never fails; it
|
|
209
|
+
reports through `status`.
|
|
210
|
+
- **`mounted.dispatch`** sends a Message the way a view does. The container
|
|
211
|
+
must have an `id`, or `mount` throws.
|
|
443
212
|
|
|
444
|
-
|
|
445
|
-
|
|
446
|
-
`more` or a round moves the cursor nowhere. A replica that was offline for a
|
|
447
|
-
long time therefore catches up in bounded steps within one `synchronize`. The
|
|
448
|
-
example servers page with Durable's `read(key, cursor, { limit })`.
|
|
213
|
+
Reload the page: the todos come back from the outbox and the committed
|
|
214
|
+
snapshot, the selection does not. That is the shared/local line, visible.
|
|
449
215
|
|
|
450
|
-
|
|
216
|
+
`mounted` also exposes `model()`, `subscribe`, `observe`, `committed`, and
|
|
217
|
+
`dispose()`, which waits for in-flight persists and leaves the replica open.
|
|
218
|
+
Close the replica with its own scope.
|
|
451
219
|
|
|
452
|
-
|
|
453
|
-
the server has not seen them. `make({ ..., coalesce: (last, next) => merged })`
|
|
454
|
-
is offered each submitted Message together with the last one in the outbox. If
|
|
455
|
-
it returns a Message, that Message replaces the last operation under the last
|
|
456
|
-
operation's id; if it returns `undefined`, the new Message gets an operation of
|
|
457
|
-
its own. Replaying the merged Message must give the same result as replaying
|
|
458
|
-
`last` then `next`.
|
|
220
|
+
## The server
|
|
459
221
|
|
|
460
|
-
|
|
461
|
-
|
|
462
|
-
|
|
463
|
-
|
|
464
|
-
and a single keystroke goes out as quickly as it did before. The local sequence
|
|
465
|
-
still advances on every submit.
|
|
466
|
-
|
|
467
|
-
## Replica status and UI state
|
|
468
|
-
|
|
469
|
-
The public replica surface deliberately exposes enough state for UI and recovery
|
|
470
|
-
without exposing the application Messages themselves:
|
|
222
|
+
The server answers the exchange: it takes a cursor and pending operations,
|
|
223
|
+
commits what it accepts, and returns what the replica is missing. With
|
|
224
|
+
`foldkit-durable`, the same contract supplies the journal's codecs, empty
|
|
225
|
+
snapshot, reducer, and any [authorization](#authorize-on-the-server):
|
|
471
226
|
|
|
472
227
|
```ts
|
|
473
|
-
const status = yield* replica.status
|
|
474
|
-
// {
|
|
475
|
-
// pending: number,
|
|
476
|
-
// cursor: Sequence,
|
|
477
|
-
// lastError: string | undefined,
|
|
478
|
-
// rejected: ReadonlyArray<OpId>
|
|
479
|
-
// }
|
|
480
|
-
```
|
|
481
|
-
|
|
482
|
-
`replica.statusChanges` emits that status initially and after every submit and
|
|
483
|
-
exchange. `replica.changes` emits the status and optimistic `shared` value
|
|
484
|
-
together from one replica snapshot, so a UI can subscribe once without racing
|
|
485
|
-
two separate reads. `replica.snapshot` is the same read, once: the status,
|
|
486
|
-
`shared`, `committed`, and `nextLocalSequence`, the local sequence the next
|
|
487
|
-
submitted operation takes.
|
|
488
|
-
|
|
489
|
-
## Sync + Durable
|
|
490
|
-
|
|
491
|
-
Sync owns **client replication semantics**. Durable can own **server ordering and
|
|
492
|
-
persistence**.
|
|
493
|
-
|
|
494
|
-
```text
|
|
495
|
-
client
|
|
496
|
-
-----------------------------------------------
|
|
497
|
-
Foldkit update
|
|
498
|
-
|
|
|
499
|
-
| durable Message
|
|
500
|
-
v
|
|
501
|
-
foldkit-sync Replica
|
|
502
|
-
optimistic shared state
|
|
503
|
-
persistent outbox
|
|
504
|
-
|
|
|
505
|
-
| exchange
|
|
506
|
-
v
|
|
507
|
-
-----------------------------------------------
|
|
508
|
-
server
|
|
509
|
-
foldkit-durable Journal
|
|
510
|
-
authoritative order
|
|
511
|
-
snapshot + cursor
|
|
512
|
-
|
|
|
513
|
-
v
|
|
514
|
-
committed operations / checkpoint / rejection
|
|
515
|
-
```
|
|
516
|
-
|
|
517
|
-
The same Sync contract produces the server journal contract, so the two halves
|
|
518
|
-
do not duplicate the Message codec, shared schema, initial snapshot, or replay
|
|
519
|
-
reducer:
|
|
520
|
-
|
|
521
|
-
```ts
|
|
522
|
-
import { Effect } from 'effect'
|
|
523
228
|
import { ActorId, DocumentId, Journal, OpId } from 'foldkit-durable'
|
|
524
229
|
|
|
525
230
|
type Principal = { readonly actorId: string }
|
|
526
231
|
|
|
527
|
-
const
|
|
528
|
-
|
|
529
|
-
|
|
530
|
-
|
|
531
|
-
|
|
532
|
-
|
|
232
|
+
const journal = yield* Journal.make({
|
|
233
|
+
...TodoSync.journalContract(),
|
|
234
|
+
file: 'todos.sqlite',
|
|
235
|
+
opId: operation => OpId.make(operation.opId),
|
|
236
|
+
actorId: (principal: Principal) => ActorId.make(principal.actorId),
|
|
237
|
+
})
|
|
238
|
+
```
|
|
239
|
+
|
|
240
|
+
The exchange handler itself is yours: it calls `journal.append` for each
|
|
241
|
+
pending operation, `journal.read` for what follows the cursor, and serves the
|
|
242
|
+
result on a socket with `Sync.transport.serve`. [The replicated state
|
|
243
|
+
guide](../../docs/replication.md#3-the-server-an-exchange-over-the-journal) shows
|
|
244
|
+
the whole handler in one page, and [`examples/sync`](../../examples/sync) runs
|
|
245
|
+
it over `ws`.
|
|
533
246
|
|
|
534
|
-
|
|
535
|
-
opId: operation => OpId.make(operation.opId),
|
|
536
|
-
actorId: (principal: Principal) => ActorId.make(principal.actorId),
|
|
537
|
-
})
|
|
247
|
+
A server that is not Durable implements the same [exchange](#the-exchange).
|
|
538
248
|
|
|
539
|
-
|
|
540
|
-
}).pipe(Effect.scoped)
|
|
541
|
-
```
|
|
249
|
+
## What makes a Message durable
|
|
542
250
|
|
|
543
|
-
|
|
544
|
-
|
|
545
|
-
|
|
251
|
+
Replay runs a durable Message through `update` on another machine, later,
|
|
252
|
+
possibly many times as the base beneath it moves. So it has to be a
|
|
253
|
+
deterministic, state-only transition of the shared slice. `make` derives replay
|
|
254
|
+
from `update` and refuses a durable Message whose transition:
|
|
546
255
|
|
|
547
|
-
|
|
256
|
+
- returns a Command, because an effect cannot be replayed; or
|
|
257
|
+
- changes a field outside `shared`, because that change would be lost.
|
|
548
258
|
|
|
549
|
-
|
|
550
|
-
|
|
551
|
-
Model.
|
|
259
|
+
A refused Message fails `submit` with `ReplayError` and writes nothing, so a
|
|
260
|
+
mistake shows on the first edit instead of as diverging replicas later.
|
|
552
261
|
|
|
553
|
-
|
|
262
|
+
The consequence is a pattern you will use everywhere: **an intent mints, a fact
|
|
263
|
+
records**.
|
|
554
264
|
|
|
555
265
|
```text
|
|
556
|
-
|
|
557
|
-
|
|
|
266
|
+
RequestedTodo { title } local: its update returns a Command
|
|
267
|
+
| that reads the clock and mints an id
|
|
558
268
|
v
|
|
559
|
-
|
|
560
|
-
|
|
|
561
|
-
+--> local fields / render
|
|
562
|
-
|
|
|
563
|
-
v
|
|
564
|
-
persist durable Message through replica
|
|
565
|
-
|
|
|
566
|
-
exchange changes shared state later
|
|
567
|
-
v
|
|
568
|
-
install reconciled shared slice back into Model
|
|
269
|
+
CreatedTodo { id, title, at } durable: pure over the shared slice
|
|
569
270
|
```
|
|
570
271
|
|
|
571
|
-
|
|
572
|
-
|
|
573
|
-
|
|
574
|
-
```ts
|
|
575
|
-
import { Effect, Scope } from 'effect'
|
|
576
|
-
import { ReplicaId, Sync } from 'foldkit-sync'
|
|
272
|
+
Put every nondeterministic input (ids, timestamps, the result of a provider
|
|
273
|
+
call) *inside* the fact. `update` for the fact only applies it.
|
|
577
274
|
|
|
578
|
-
|
|
275
|
+
When the intent needs nothing outside the Model, say an id from a counter in
|
|
276
|
+
the shared slice, return the fact as `Sync.fact(message)` instead of an
|
|
277
|
+
ordinary Command. The mount applies it in the intent's own transition, before
|
|
278
|
+
any later Message, so two intents dispatched together cannot read the same
|
|
279
|
+
counter. Outside the mount it is an ordinary Command.
|
|
579
280
|
|
|
580
|
-
|
|
581
|
-
|
|
582
|
-
Effect.provideService(
|
|
583
|
-
Sync.indexedDb('todos/tab-1'),
|
|
584
|
-
Scope.Scope,
|
|
585
|
-
scope,
|
|
586
|
-
),
|
|
587
|
-
)
|
|
588
|
-
const replica = Effect.runSync(
|
|
589
|
-
TodoSync.openReplica(ReplicaId.make('tab-1'), storage),
|
|
590
|
-
)
|
|
591
|
-
|
|
592
|
-
const mounted = Sync.mount(App, TodoSync, {
|
|
593
|
-
replica,
|
|
594
|
-
container,
|
|
595
|
-
view: (model, h) => ({
|
|
596
|
-
title: 'Todos',
|
|
597
|
-
body: h.ul(
|
|
598
|
-
[],
|
|
599
|
-
model.todos.map(todo => h.li([], [todo.title])),
|
|
600
|
-
),
|
|
601
|
-
}),
|
|
602
|
-
onPersistenceFailure: (model, error) => ({
|
|
603
|
-
...model,
|
|
604
|
-
lastError: Option.some(error.message),
|
|
605
|
-
}),
|
|
606
|
-
})
|
|
607
|
-
|
|
608
|
-
Effect.runFork(
|
|
609
|
-
Effect.provide(
|
|
610
|
-
replica.start,
|
|
611
|
-
Sync.transport.socket({ url: 'wss://example.com/sync' }),
|
|
612
|
-
),
|
|
613
|
-
)
|
|
614
|
-
|
|
615
|
-
mounted.dispatch(
|
|
616
|
-
Message.CreatedTodo({ id: crypto.randomUUID(), title: 'Milk' }),
|
|
617
|
-
)
|
|
281
|
+
A durable `update` should also be idempotent where a retry could deliver one
|
|
282
|
+
operation twice, as in `CreatedTodo` ignoring an id it already holds.
|
|
618
283
|
|
|
619
|
-
|
|
620
|
-
await mounted.dispose()
|
|
621
|
-
```
|
|
284
|
+
## What to show the user
|
|
622
285
|
|
|
623
|
-
|
|
624
|
-
remains separately owned and must be closed with its scope/lifecycle.
|
|
286
|
+
The three milestones each have a reader.
|
|
625
287
|
|
|
626
|
-
|
|
627
|
-
exchange changes what the replica holds (it commits, acknowledges or rejects
|
|
628
|
-
something), and when a failed persist reverts its edit. A replica status that
|
|
629
|
-
changes nothing the Model shows, such as the one a mount starts with, replaces
|
|
630
|
-
nothing. `onReinstall(next, previous)` sees both cases and returns the
|
|
631
|
-
transition, so local state that points into the shared slice (a selection) can
|
|
632
|
-
be carried across the change, and a Command can take it where it has to go (a
|
|
633
|
-
DOM that is patched rather than re-rendered). Omitted, the Model is simply
|
|
634
|
-
`next`. A durable `Sync.fact` it returns is a new edit, so return one only for
|
|
635
|
-
what changed: returned every time, it adds an edit per exchange, and while
|
|
636
|
-
storage keeps failing, one per failure.
|
|
288
|
+
**Visible** is the Model. Nothing to do.
|
|
637
289
|
|
|
638
|
-
|
|
290
|
+
**Saved** is the persist. `onPersistenceFailure` runs when it fails, after the
|
|
291
|
+
edit has been reverted; the Model it returns is what the user sees next. The
|
|
292
|
+
error names why: `ReplayError` for a Message that cannot be durable,
|
|
293
|
+
`StorageError` for IndexedDB.
|
|
639
294
|
|
|
640
|
-
|
|
295
|
+
**Committed** is the replica's status:
|
|
641
296
|
|
|
642
|
-
|
|
643
|
-
|
|
644
|
-
|
|
297
|
+
```ts
|
|
298
|
+
const status = yield* replica.status
|
|
299
|
+
// { pending: number, cursor: Sequence, lastError: string | undefined, rejected: ReadonlyArray<OpId> }
|
|
300
|
+
```
|
|
645
301
|
|
|
646
|
-
|
|
647
|
-
|
|
302
|
+
`pending` above zero means edits the server has not confirmed; `lastError` is
|
|
303
|
+
the last failed exchange, cleared by the next success; `rejected` lists recent
|
|
304
|
+
edits the server refused. `replica.statusChanges` is a Stream of the same,
|
|
305
|
+
emitted after every submit and exchange; `replica.changes` pairs it with the
|
|
306
|
+
optimistic `shared` value so one subscription sees both.
|
|
648
307
|
|
|
649
|
-
|
|
650
|
-
|
|
651
|
-
|
|
652
|
-
|
|
308
|
+
**A refused edit** arrives as a rejection on the next exchange, not as an
|
|
309
|
+
error. The replica drops it and replays the rest; the mount re-installs the
|
|
310
|
+
shared slice, and the field the user edited reverts. `onPersistenceFailure`
|
|
311
|
+
does not run for this, since the local save succeeded. Watch `rejected` to
|
|
312
|
+
explain the revert.
|
|
653
313
|
|
|
654
|
-
The Model shows
|
|
655
|
-
must not report success until the
|
|
656
|
-
`mounted.committed
|
|
314
|
+
**Waiting for the server.** The Model shows an edit before the server has it.
|
|
315
|
+
Something that must not report success until the edit is committed, an agent
|
|
316
|
+
tool for instance, waits on `mounted.committed`: the shared slice as the server
|
|
317
|
+
confirmed it, with no pending edit applied. It is a source, `{ get, subscribe }`,
|
|
318
|
+
told after every exchange:
|
|
657
319
|
|
|
658
320
|
```ts
|
|
659
321
|
completion: Agent.when({
|
|
@@ -662,29 +324,22 @@ completion: Agent.when({
|
|
|
662
324
|
})
|
|
663
325
|
```
|
|
664
326
|
|
|
665
|
-
|
|
666
|
-
server rejects.
|
|
667
|
-
`mounted.committed` is a source (`{ get, subscribe }`), not a Projection: it reads
|
|
668
|
-
the replica, not the Model, and tells subscribers after every exchange.
|
|
327
|
+
That completes after the exchange that commits the todo and never for one the
|
|
328
|
+
server rejects.
|
|
669
329
|
|
|
670
|
-
|
|
671
|
-
different mechanism: see [what a reader sees while a change is in
|
|
672
|
-
flight](../../docs/state-model.md#what-a-reader-sees-while-a-change-is-in-flight).
|
|
330
|
+
## Growing the application
|
|
673
331
|
|
|
674
|
-
|
|
332
|
+
### Compose a document from features
|
|
675
333
|
|
|
676
|
-
|
|
677
|
-
|
|
334
|
+
Each feature declares the fields it shares and the Messages that change them;
|
|
335
|
+
`compose` merges them into one document:
|
|
678
336
|
|
|
679
337
|
```ts
|
|
680
338
|
const AppSync = Sync.forApplication(App)
|
|
681
339
|
|
|
682
340
|
const Todos = AppSync.fragment({
|
|
683
341
|
shared: Projection.pick(App.model.todos),
|
|
684
|
-
durable: MessageSet.make(App, [
|
|
685
|
-
Message.CreatedTodo,
|
|
686
|
-
Message.RenamedTodo,
|
|
687
|
-
]),
|
|
342
|
+
durable: MessageSet.make(App, [Message.CreatedTodo, Message.RenamedTodo]),
|
|
688
343
|
})
|
|
689
344
|
|
|
690
345
|
const Members = AppSync.fragment({
|
|
@@ -698,424 +353,295 @@ const Board = AppSync.make({
|
|
|
698
353
|
})
|
|
699
354
|
```
|
|
700
355
|
|
|
701
|
-
`compose`
|
|
702
|
-
|
|
703
|
-
|
|
704
|
-
architectural mistake of two contracts claiming the same field ownership.
|
|
356
|
+
`compose` throws on a field declared twice with different codecs, a Message
|
|
357
|
+
declared durable twice, or a fragment from another application. The result is
|
|
358
|
+
still one document: one outbox, one cursor, one server order.
|
|
705
359
|
|
|
706
|
-
|
|
360
|
+
### Authorize on the server
|
|
707
361
|
|
|
708
|
-
|
|
709
|
-
|
|
710
|
-
`journalContract()` and is enforced by the server journal — not trusted to the
|
|
711
|
-
client.
|
|
362
|
+
Policy belongs on the contract, beside the Messages it governs, and runs on the
|
|
363
|
+
server inside the journal's commit. The client never runs it.
|
|
712
364
|
|
|
713
365
|
```ts
|
|
714
|
-
const Authorized = Sync.forApplication(App)
|
|
715
|
-
.withPrincipal<{ readonly role: 'admin' | 'guest' }>()
|
|
366
|
+
const Authorized = Sync.forApplication(App).withPrincipal<{ readonly role: 'admin' | 'guest' }>()
|
|
716
367
|
|
|
717
368
|
const Board = Authorized.make({
|
|
718
369
|
documentId: DocumentId.make('board'),
|
|
719
370
|
...Authorized.compose(Todos, Members),
|
|
720
371
|
authorize: {
|
|
721
372
|
RenamedTodo: ({ principal, message, shared }) =>
|
|
722
|
-
principal.role === 'admin' &&
|
|
723
|
-
shared.todos.some(todo => todo.id === message.id),
|
|
373
|
+
principal.role === 'admin' && shared.todos.some(todo => todo.id === message.id),
|
|
724
374
|
},
|
|
725
375
|
})
|
|
726
376
|
```
|
|
727
377
|
|
|
728
|
-
|
|
729
|
-
|
|
378
|
+
`message` is exactly that variant, `shared` is the authoritative snapshot, and
|
|
379
|
+
`principal` is whatever the server's transport established for the connection;
|
|
380
|
+
it is never read from the operation. A variant without a rule is allowed. A key
|
|
381
|
+
that is not a durable variant is a type error. The rules ride in
|
|
382
|
+
`journalContract()`, so the server enforces them by spreading the contract.
|
|
730
383
|
|
|
731
|
-
|
|
732
|
-
|
|
733
|
-
rebases the remaining ones. This is why rejection is a normal exchange result,
|
|
734
|
-
not a transport failure.
|
|
384
|
+
An edit the policy refuses still shows at once on the client, then comes back
|
|
385
|
+
as a rejection, as [above](#what-to-show-the-user).
|
|
735
386
|
|
|
736
|
-
|
|
387
|
+
### Coalesce a burst of typing
|
|
737
388
|
|
|
738
|
-
|
|
739
|
-
|
|
740
|
-
|
|
741
|
-
The built-in browser storage adapter is IndexedDB:
|
|
389
|
+
Typing makes one durable Message per keystroke. `coalesce(last, next)` is
|
|
390
|
+
offered each submit together with the last operation in the outbox; return a
|
|
391
|
+
merged Message to replace it, or `undefined` to keep them apart:
|
|
742
392
|
|
|
743
393
|
```ts
|
|
744
|
-
|
|
394
|
+
coalesce: (last, next) =>
|
|
395
|
+
last._tag === 'RenamedTodo' && next._tag === 'RenamedTodo' && last.id === next.id
|
|
396
|
+
? next
|
|
397
|
+
: undefined
|
|
745
398
|
```
|
|
746
399
|
|
|
747
|
-
|
|
748
|
-
|
|
749
|
-
|
|
750
|
-
|
|
751
|
-
Message Schema.
|
|
752
|
-
|
|
753
|
-
A submit writes only its operation. A `Storage` may implement the optional
|
|
754
|
-
`append(entry, expectedRevision)`, which records one operation with the
|
|
755
|
-
replica's next revision and local sequence. `load` then returns the saved state
|
|
756
|
-
with those operations added to its outbox. The IndexedDB adapter keeps them in an
|
|
757
|
-
`outbox` store beside the state, so a keystroke costs the size of the operation,
|
|
758
|
-
not of the document. An exchange changes the committed snapshot, so it still
|
|
759
|
-
saves the whole state, which clears the appended rows. A storage without
|
|
760
|
-
`append` saves the whole state on every submit, as before.
|
|
761
|
-
|
|
762
|
-
Important recovery cases:
|
|
763
|
-
|
|
764
|
-
- **Storage absent or evicted.** The replica starts at cursor 0 and can catch up
|
|
765
|
-
from the server, but edits that existed only in the lost outbox are gone.
|
|
766
|
-
- **Two writers use one replica storage.** Compare-and-swap detects the stale
|
|
767
|
-
writer. Give each tab/writer its own replica/storage identity.
|
|
768
|
-
- **Replay refuses a Message.** `submit` fails and writes nothing, so invalid
|
|
769
|
-
durable work never enters the outbox.
|
|
770
|
-
- **Malformed stored data.** Opening fails rather than silently replacing bytes;
|
|
771
|
-
the application can offer an explicit reset/recovery path.
|
|
772
|
-
- **Unsupported storage version.** `UnsupportedReplicaVersionError` names the
|
|
773
|
-
persisted and supported versions so migration/reset can be explicit.
|
|
774
|
-
- **Rolling back past the outbox store.** The IndexedDB adapter upgrades its
|
|
775
|
-
database to version 2, adding the `outbox` store. A release from before that
|
|
776
|
-
opens it as version 1, which the browser refuses with a `VersionError`, so a
|
|
777
|
-
rollback that far needs a new database name or a reset.
|
|
778
|
-
- **Another tab blocks the upgrade.** Opening fails with a `StorageError`
|
|
779
|
-
rather than waiting; a connection the upgrade opens later is closed.
|
|
780
|
-
- **Server compaction.** A checkpoint replaces history the replica can no longer
|
|
781
|
-
replay and pending local work is rebased on it.
|
|
782
|
-
- **Server reset.** A server that answers with an `epoch` (Durable's
|
|
783
|
-
`journal.epoch(key)`) names its history, and the replica stores it and sends
|
|
784
|
-
it back as `exchange`'s third argument. A server that finds another epoch
|
|
785
|
-
answers from sequence 0 of its own history, and should skip its
|
|
786
|
-
cursor-ahead check for that request. The replica then rebuilds its committed
|
|
787
|
-
state from that answer and keeps its outbox, which the same exchange delivers.
|
|
788
|
-
Everything the old server committed is gone, on every replica; only what was
|
|
789
|
-
still pending survives. The replica trusts a server that names a new epoch to
|
|
790
|
-
have answered from its start; it does not check. A replica that has never
|
|
791
|
-
heard an epoch takes the first one as its own, so a reset before that is not
|
|
792
|
-
recognized: such a replica fails every exchange (its cursor is ahead of the
|
|
793
|
-
server's) until its storage is cleared.
|
|
794
|
-
|
|
795
|
-
Application Message/shared-state migrations remain application policy; Sync
|
|
796
|
-
versioning protects its own persisted envelope.
|
|
797
|
-
|
|
798
|
-
## Transport
|
|
799
|
-
|
|
800
|
-
`Transport` is an Effect service for the one exchange operation. Sync ships
|
|
801
|
-
helpers under `Sync.transport`:
|
|
400
|
+
Replaying the merged Message must equal replaying `last` then `next`. An
|
|
401
|
+
operation is only merged into while no exchange has carried it, so there is no
|
|
402
|
+
timer to tune: keystrokes typed while a request is out become one operation, and
|
|
403
|
+
a lone keystroke goes out as fast as before.
|
|
802
404
|
|
|
803
|
-
|
|
804
|
-
Sync.transport.socket(...)
|
|
805
|
-
Sync.transport.loopback(...)
|
|
806
|
-
Sync.transport.fromPromise(...)
|
|
807
|
-
Sync.transport.toPromise(...)
|
|
808
|
-
Sync.transport.serve(...)
|
|
809
|
-
Sync.transport.nativeSocket(...)
|
|
810
|
-
```
|
|
405
|
+
### The URL, mirrors, and agents
|
|
811
406
|
|
|
812
|
-
|
|
813
|
-
|
|
814
|
-
|
|
815
|
-
|
|
816
|
-
|
|
817
|
-
only when a connection proves healthy, by answering a frame or by staying open
|
|
818
|
-
for `maxRetryDelay`, so a server that accepts each socket and drops it is backed
|
|
819
|
-
off like one that refuses it. The queue is bounded by `maxQueue`. A server rejection is protocol data; only a wire failure is
|
|
820
|
-
a `TransportError`.
|
|
821
|
-
|
|
822
|
-
The server can also tell a client that something changed. Pass `serve` a
|
|
823
|
-
`changes` subscription, a function that takes a listener and returns its
|
|
824
|
-
unsubscribe, and it sends the socket a `{ notify: true }` frame after each
|
|
825
|
-
commit. The client's socket transport exposes these notices, and every
|
|
826
|
-
(re)connection, as the `Transport`'s optional `changes` stream, which
|
|
827
|
-
`replica.start` wakes on as it does on a submit. A notice carries no data: the
|
|
828
|
-
exchange that follows reads from the replica's own cursor, so a lost or repeated
|
|
829
|
-
notice is harmless, and a replica that is only reading still sees others' edits
|
|
830
|
-
without polling. With Durable, the subscription is `journal.subscribe` filtered
|
|
831
|
-
to the document's key; `examples/sync/src/server.ts` wires it.
|
|
832
|
-
|
|
833
|
-
Transport is deliberately below reconciliation. A custom transport can carry
|
|
834
|
-
the same exchange without changing replica semantics.
|
|
835
|
-
|
|
836
|
-
## Presence: ephemeral collaboration state
|
|
837
|
-
|
|
838
|
-
Presence is intentionally **not** a durable Message log. “Alice is viewing this
|
|
839
|
-
page” or a cursor position should expire when the peer disappears, not replay
|
|
840
|
-
next week.
|
|
841
|
-
|
|
842
|
-
`Sync.presence.make` creates a TTL'd peer registry. Values are decoded before
|
|
843
|
-
they enter the registry; a peer that stops refreshing is removed. Changes are
|
|
844
|
-
available both as a callback subscription and as a Stream. A caret sets a new
|
|
845
|
-
value on nearly every keystroke; with `throttle: '50 millis'` a value set after
|
|
846
|
-
a quiet interval is sent at once, and those set within the interval wait, only
|
|
847
|
-
the latest of them sent when it ends. The peer's own entry changes at once. A departure (`leave`) is sent immediately
|
|
848
|
-
and cancels a value still waiting.
|
|
849
|
-
|
|
850
|
-
Presence can share the socket transport's connection. `transport.socket` is a
|
|
851
|
-
stable handle to it: a send goes to the socket that is open now and is dropped
|
|
852
|
-
while none is, and messages from every socket the transport opens arrive
|
|
853
|
-
through it. So `Sync.presence.socketChannel(transport.socket)` survives
|
|
854
|
-
reconnects without being bound again. After a reconnect the server has
|
|
855
|
-
forgotten the peer's value, so announce it again within the time to live.
|
|
856
|
-
|
|
857
|
-
Presence can travel through:
|
|
407
|
+
`mount` takes the application's own `subscriptions` and the `resources` Layer
|
|
408
|
+
they need, and `url: { init, onUrlChange, onUrlRequest? }` to route navigation
|
|
409
|
+
through `update`: `init(model, url)` reduces the first URL before the first
|
|
410
|
+
render and `onUrlChange(url)` names the Message for every navigation. A
|
|
411
|
+
[`foldkit-mirror`](../mirror) URL mirror plugs into exactly that seam.
|
|
858
412
|
|
|
859
|
-
|
|
860
|
-
|
|
861
|
-
|
|
862
|
-
```
|
|
413
|
+
`Mounted` is also the host an agent binds to: `model`, `dispatch`, `subscribe`,
|
|
414
|
+
and `observe`, which reports every application Message the runtime applies.
|
|
415
|
+
See [`examples/todo-app`](../../examples/todo-app) for both.
|
|
863
416
|
|
|
864
|
-
|
|
417
|
+
### Carry local state across a reinstall
|
|
865
418
|
|
|
866
|
-
|
|
867
|
-
|
|
868
|
-
|
|
869
|
-
|
|
419
|
+
When an exchange or a failed persist replaces the shared slice outside
|
|
420
|
+
`update`, `onReinstall(next, previous)` returns the transition instead. Use it
|
|
421
|
+
to keep a selection that points into the slice, or to return a Command that
|
|
422
|
+
patches a DOM the change has to reach. Omitted, the Model is simply `next`. It
|
|
423
|
+
is not called for a status that changed nothing the Model shows.
|
|
870
424
|
|
|
871
|
-
|
|
872
|
-
deterministic in tests.
|
|
425
|
+
## Testing without a browser or a server
|
|
873
426
|
|
|
874
|
-
|
|
875
|
-
|
|
876
|
-
```text
|
|
877
|
-
shared application fact that must replay later -> durable Message / Sync
|
|
878
|
-
peer status that should disappear when stale -> Presence
|
|
879
|
-
```
|
|
880
|
-
|
|
881
|
-
## Last-writer-wins fields (experimental)
|
|
882
|
-
|
|
883
|
-
Server arrival order is the default conflict rule: whichever operation the
|
|
884
|
-
server commits first is replayed first. Sometimes one field needs a different
|
|
885
|
-
rule — for example, the newest logical title write should win even if an offline
|
|
886
|
-
device reconnects later.
|
|
887
|
-
|
|
888
|
-
`Sync.lww.register` supplies a schema and pure merge rule for that specialized
|
|
889
|
-
case. It is a field-level helper, **not a general CRDT mode for arbitrary
|
|
890
|
-
Messages**.
|
|
427
|
+
A `Storage` needs three members, and the loopback transport takes a handler in
|
|
428
|
+
place of a socket:
|
|
891
429
|
|
|
892
430
|
```ts
|
|
893
|
-
import {
|
|
894
|
-
import {
|
|
895
|
-
|
|
896
|
-
const
|
|
897
|
-
|
|
898
|
-
|
|
899
|
-
|
|
900
|
-
|
|
901
|
-
|
|
902
|
-
|
|
903
|
-
const update = (
|
|
904
|
-
model: typeof Shared.Type,
|
|
905
|
-
message: typeof Renamed.Type,
|
|
906
|
-
) => ({
|
|
907
|
-
...model,
|
|
908
|
-
title: Title.merge(model.title, message.title),
|
|
909
|
-
})
|
|
910
|
-
|
|
911
|
-
const message: typeof Renamed.Type = {
|
|
912
|
-
_tag: 'Renamed',
|
|
913
|
-
title: {
|
|
914
|
-
stamp: { counter: 1, replicaId: ReplicaId.make('tab-a') },
|
|
915
|
-
value: 'Milk',
|
|
916
|
-
},
|
|
431
|
+
import { Effect } from 'effect'
|
|
432
|
+
import type { Storage } from 'foldkit-sync'
|
|
433
|
+
|
|
434
|
+
const memoryStorage = (): Storage => {
|
|
435
|
+
let state: unknown
|
|
436
|
+
return {
|
|
437
|
+
load: () => Effect.sync(() => state),
|
|
438
|
+
save: next => Effect.sync(() => void (state = structuredClone(next))),
|
|
439
|
+
close: Effect.void,
|
|
440
|
+
}
|
|
917
441
|
}
|
|
918
|
-
```
|
|
919
442
|
|
|
920
|
-
|
|
921
|
-
|
|
922
|
-
|
|
923
|
-
|
|
443
|
+
const program = Effect.gen(function* () {
|
|
444
|
+
const replica = yield* TodoSync.openReplica(ReplicaId.make('test'), memoryStorage())
|
|
445
|
+
yield* replica.submit(Message.CreatedTodo({ id: 't1', title: 'Milk' }))
|
|
446
|
+
yield* replica.synchronize
|
|
447
|
+
return yield* replica.status
|
|
448
|
+
}).pipe(
|
|
449
|
+
Effect.provide(
|
|
450
|
+
Sync.transport.loopback((_cursor, pending) => ({
|
|
451
|
+
operations: [],
|
|
452
|
+
acknowledged: pending.map(operation => operation.opId),
|
|
453
|
+
rejected: [],
|
|
454
|
+
})),
|
|
455
|
+
),
|
|
456
|
+
)
|
|
457
|
+
```
|
|
924
458
|
|
|
925
|
-
|
|
926
|
-
|
|
927
|
-
|
|
459
|
+
In Node, `Sync.indexedDb(name, new IDBFactory())` from `fake-indexeddb` tests
|
|
460
|
+
the real storage adapter. [`examples/sync`](../../examples/sync) tests every
|
|
461
|
+
recovery case below against a real journal.
|
|
462
|
+
|
|
463
|
+
## When things go wrong
|
|
464
|
+
|
|
465
|
+
Each case is designed, not incidental. The ones that change how you build the
|
|
466
|
+
application come first.
|
|
467
|
+
|
|
468
|
+
- **The network is gone.** Edits keep applying and keep being saved. `pending`
|
|
469
|
+
grows; the loop retries. Nothing to handle.
|
|
470
|
+
- **The server rejects an edit.** Dropped from `pending`, the rest replayed,
|
|
471
|
+
the Model re-installed. Explain it from `status.rejected`.
|
|
472
|
+
- **The persist fails.** The edit is reverted and `onPersistenceFailure` runs.
|
|
473
|
+
The user has to redo it.
|
|
474
|
+
- **Local storage was evicted.** The replica starts at cursor 0 and catches up
|
|
475
|
+
from the server, but edits that lived only in the outbox are gone. The
|
|
476
|
+
outbox is user data, not a cache.
|
|
477
|
+
- **Two tabs share one storage name.** Compare-and-swap fails the stale writer
|
|
478
|
+
with `StorageError`. Give each tab its own name and `ReplicaId`.
|
|
479
|
+
- **The replica is far behind.** The server may answer in pages (`more`), and
|
|
480
|
+
`synchronize` asks again at once until caught up. If the server compacted
|
|
481
|
+
the history it needs, it sends a checkpoint instead: a snapshot and cursor
|
|
482
|
+
the replica adopts as its base before replaying its pending edits. A
|
|
483
|
+
checkpoint behind the replica's cursor is refused
|
|
484
|
+
(`CheckpointRegressionError`), never applied backwards.
|
|
485
|
+
- **The server lost its history.** A server names its history with an `epoch`;
|
|
486
|
+
a replica that hears a new one rebuilds its committed state from that answer
|
|
487
|
+
and resends its outbox. Everything the old server committed is gone on every
|
|
488
|
+
replica. A replica that never heard an epoch cannot tell, and fails every
|
|
489
|
+
exchange until its storage is cleared.
|
|
490
|
+
- **Stored state is from another version.** Opening fails with
|
|
491
|
+
`UnsupportedReplicaVersionError` naming both versions; the bytes are left for
|
|
492
|
+
a deliberate migration or reset. Sync versions its own envelope only; your
|
|
493
|
+
Message and snapshot migrations stay yours.
|
|
494
|
+
- **The server answers nonsense.** Malformed exchanges, gaps in the committed
|
|
495
|
+
order, acknowledgements or rejections for operations the replica never sent:
|
|
496
|
+
each fails the exchange rather than touch local work.
|
|
497
|
+
|
|
498
|
+
## Advanced: the replica
|
|
499
|
+
|
|
500
|
+
`Sync.mount` is the normal path. Underneath, a `Replica` works on the shared
|
|
501
|
+
slice alone, and an application that is not a Foldkit runtime (a worker, a
|
|
502
|
+
server-side actor, a test) uses it directly.
|
|
928
503
|
|
|
929
504
|
```ts
|
|
930
|
-
|
|
931
|
-
|
|
932
|
-
|
|
933
|
-
type Shared = typeof Shared.Type
|
|
934
|
-
type Renamed = typeof Renamed.Type
|
|
935
|
-
|
|
936
|
-
const rename = (replica: Replica<Renamed, Shared>, title: string) =>
|
|
937
|
-
Effect.gen(function* () {
|
|
938
|
-
const storage = yield* Sync.indexedDb('todos-tab-a-clock')
|
|
939
|
-
const clock = yield* Sync.lww.openClock({
|
|
940
|
-
documentId: DocumentId.make('todos'),
|
|
941
|
-
replicaId: ReplicaId.make('tab-a'),
|
|
942
|
-
storage,
|
|
943
|
-
})
|
|
944
|
-
|
|
945
|
-
const shared = yield* replica.shared
|
|
946
|
-
const stamp = yield* clock.next(shared.title.stamp.counter)
|
|
947
|
-
|
|
948
|
-
yield* replica.submit({
|
|
949
|
-
_tag: 'Renamed',
|
|
950
|
-
title: { stamp, value: title },
|
|
951
|
-
})
|
|
952
|
-
|
|
953
|
-
yield* clock.close
|
|
954
|
-
}).pipe(Effect.scoped)
|
|
955
|
-
```
|
|
956
|
-
|
|
957
|
-
`next(observedCounter)` persists a counter greater than both its previous saved
|
|
958
|
-
value and the supplied observation before returning the stamp. Pass the latest
|
|
959
|
-
observed counter for the field being edited; if causality spans several
|
|
960
|
-
registers, pass their maximum. Calling `next()` without an observation advances
|
|
961
|
-
only the saved local counter.
|
|
505
|
+
const program = Effect.gen(function* () {
|
|
506
|
+
const storage = yield* Sync.indexedDb('todos/tab-1')
|
|
507
|
+
const replica = yield* TodoSync.openReplica(ReplicaId.make('tab-1'), storage)
|
|
962
508
|
|
|
963
|
-
|
|
964
|
-
|
|
965
|
-
Storage failures do not return a stamp. Counter gaps after a crash or rejected
|
|
966
|
-
submission are harmless.
|
|
509
|
+
yield* replica.submit(Message.CreatedTodo({ id: 't1', title: 'Milk' }))
|
|
510
|
+
const optimistic = yield* replica.shared // includes the pending edit
|
|
967
511
|
|
|
968
|
-
|
|
969
|
-
|
|
512
|
+
yield* replica.synchronize // one exchange
|
|
513
|
+
return optimistic
|
|
514
|
+
}).pipe(Effect.scoped, Effect.provide(Sync.transport.socket({ url: 'wss://example.com/sync' })))
|
|
515
|
+
```
|
|
970
516
|
|
|
971
|
-
|
|
972
|
-
|
|
973
|
-
|
|
974
|
-
|
|
975
|
-
offline write cannot resurrect deleted state.
|
|
517
|
+
`submit(message)` checks that the Message is durable, replays it against the
|
|
518
|
+
current optimistic state, gives it a stable operation id, saves it to the
|
|
519
|
+
outbox, publishes the new optimistic value, and wakes the loop. It sends
|
|
520
|
+
nothing. A Message replay refuses never enters the outbox.
|
|
976
521
|
|
|
977
|
-
|
|
978
|
-
|
|
979
|
-
|
|
980
|
-
admission when needed.
|
|
522
|
+
`synchronize` takes one snapshot of the cursor and outbox, performs one
|
|
523
|
+
exchange, validates the answer, and reconciles it with the replica's *current*
|
|
524
|
+
state, so an edit submitted while the request was in flight survives.
|
|
981
525
|
|
|
982
|
-
|
|
983
|
-
|
|
526
|
+
The rest of the surface: `committed` (the base alone), `pending`, `cursor`,
|
|
527
|
+
`status`, `statusChanges`, `changes` (status and `shared` from one snapshot),
|
|
528
|
+
`snapshot` (the same plus `nextLocalSequence`), `start`, and `close`.
|
|
984
529
|
|
|
985
|
-
|
|
530
|
+
### The exchange
|
|
986
531
|
|
|
987
|
-
|
|
988
|
-
|
|
532
|
+
One request, one answer. A custom server implements this; a custom transport
|
|
533
|
+
carries it.
|
|
989
534
|
|
|
990
|
-
|
|
991
|
-
|
|
992
|
-
|
|
535
|
+
```text
|
|
536
|
+
request cursor
|
|
537
|
+
pending operations
|
|
538
|
+
epoch, once the replica has heard one
|
|
539
|
+
|
|
540
|
+
answer operations committed after the cursor, contiguous
|
|
541
|
+
acknowledged ids the server accepted without repeating them
|
|
542
|
+
rejected ids the server refused
|
|
543
|
+
checkpoint? { cursor, model } when the history is gone
|
|
544
|
+
more? true when another page follows
|
|
545
|
+
epoch? the server's history
|
|
546
|
+
```
|
|
547
|
+
|
|
548
|
+
An acknowledged operation leaves the outbox once its committed copy has
|
|
549
|
+
arrived or the last page has been read. A rejected one leaves at once. The
|
|
550
|
+
replica validates every field; see [when things go wrong](#when-things-go-wrong).
|
|
551
|
+
|
|
552
|
+
### Transports
|
|
553
|
+
|
|
554
|
+
`Transport` is an Effect service with one operation. Sync ships:
|
|
555
|
+
|
|
556
|
+
- `Sync.transport.socket({ url, … })`: a reconnecting WebSocket client. It
|
|
557
|
+
queues exchanges while connecting, retries on a jittered backoff capped at
|
|
558
|
+
`maxRetryDelay` (5 s), keeps request ids stable across resends, and after
|
|
559
|
+
`maxRetries` (5) consecutive failed connections fails queued work and fails
|
|
560
|
+
new exchanges fast until a socket is healthy again. `maxQueue` (64) bounds
|
|
561
|
+
the queue.
|
|
562
|
+
- `Sync.transport.serve(socket, { exchange, changes? })`: the server side of one
|
|
563
|
+
accepted socket. With `changes`, a subscription to the document's commits, it
|
|
564
|
+
sends a notice after each; the client's loop exchanges on every notice, so a
|
|
565
|
+
tab that is only reading still sees others' edits without polling.
|
|
566
|
+
- `Sync.transport.loopback(handler)`, `fromPromise(client)`, `toPromise(shape)`,
|
|
567
|
+
and `nativeSocket` for the platform `WebSocket`.
|
|
568
|
+
|
|
569
|
+
A refusal is data in the answer; only a wire failure is a `TransportError`.
|
|
570
|
+
|
|
571
|
+
### Presence
|
|
572
|
+
|
|
573
|
+
"Who is viewing this" and "where is their caret" should vanish when the peer
|
|
574
|
+
does, not replay next week. `Sync.presence.make({ id, ttl, decodeValue, channel?,
|
|
575
|
+
throttle? })` is a TTL'd peer registry outside the log: values are decoded before
|
|
576
|
+
they enter it, a peer that stops refreshing is pruned, and `throttle` sends at
|
|
577
|
+
most one value per interval, the latest. `Sync.presence.socketChannel(transport.socket)`
|
|
578
|
+
shares the sync socket across reconnects; `Sync.presence.hub` and `.serve` are
|
|
579
|
+
the server side. [`examples/pages`](../../examples/pages) shows remote carets.
|
|
580
|
+
|
|
581
|
+
### Last-writer-wins fields (experimental)
|
|
582
|
+
|
|
583
|
+
Server order decides concurrent edits. When one field should instead keep the
|
|
584
|
+
newest logical write even from a device that reconnects late, `Sync.lww.register(schema)`
|
|
585
|
+
gives it a stamped schema and a pure `merge`; `Sync.lww.openClock` allocates
|
|
586
|
+
stamps from a persisted counter, before dispatch and never in `update`. It is a
|
|
587
|
+
per-field rule, not a CRDT mode: sets, counters, and text stay outside.
|
|
993
588
|
|
|
994
589
|
```ts
|
|
995
|
-
|
|
996
|
-
|
|
997
|
-
|
|
998
|
-
|
|
999
|
-
Sync,
|
|
1000
|
-
type TransportClient,
|
|
1001
|
-
} from 'foldkit-sync'
|
|
1002
|
-
|
|
1003
|
-
const Shared = Schema.Struct({ todos: Schema.Array(Schema.String) })
|
|
1004
|
-
const Renamed = Schema.Struct({
|
|
1005
|
-
_tag: Schema.Literal('Renamed'),
|
|
1006
|
-
title: Schema.String,
|
|
1007
|
-
})
|
|
590
|
+
const Title = Sync.lww.register(Schema.String)
|
|
591
|
+
const Shared = Schema.Struct({ title: Title.schema })
|
|
592
|
+
// in update: title: Title.merge(model.title, message.title)
|
|
593
|
+
```
|
|
1008
594
|
|
|
1009
|
-
|
|
595
|
+
### Without a Foldkit application
|
|
1010
596
|
|
|
1011
|
-
|
|
1012
|
-
|
|
1013
|
-
|
|
1014
|
-
|
|
1015
|
-
empty: { todos: [] },
|
|
1016
|
-
durable: () => true,
|
|
1017
|
-
replay: (shared, message) => ({
|
|
1018
|
-
todos: [...shared.todos, message.title],
|
|
1019
|
-
}),
|
|
1020
|
-
})
|
|
597
|
+
`Sync.define({ documentId, message, shared, empty, durable, replay })` is the
|
|
598
|
+
protocol primitive `forApplication` compiles to, for a client with no
|
|
599
|
+
application to derive from. Its `replay` is unguarded; the author owns its
|
|
600
|
+
agreement with whatever applies the Message elsewhere.
|
|
1021
601
|
|
|
1022
|
-
|
|
1023
|
-
const storage = yield* Sync.indexedDb('todos-tab-1')
|
|
1024
|
-
const replica = yield* Protocol.openReplica(
|
|
1025
|
-
ReplicaId.make('tab-1'),
|
|
1026
|
-
storage,
|
|
1027
|
-
)
|
|
602
|
+
### Joining an assembly
|
|
1028
603
|
|
|
1029
|
-
|
|
604
|
+
`TodoSync.wiring()` adds the contract to a `foldkit-bundle` assembly so
|
|
605
|
+
`Module.validate` can report two owners of one field. It routes nothing and
|
|
606
|
+
runs nothing; Sync owns its runtime through `mount`.
|
|
1030
607
|
|
|
1031
|
-
|
|
1032
|
-
replica.synchronize,
|
|
1033
|
-
Sync.transport.fromPromise(transport),
|
|
1034
|
-
)
|
|
608
|
+
## What Sync owns, and does not
|
|
1035
609
|
|
|
1036
|
-
|
|
1037
|
-
|
|
1038
|
-
|
|
610
|
+
Owns the replication contract (shared slice, durable set, replay, codecs,
|
|
611
|
+
`journalContract`), the replica (outbox, base, cursor, optimistic view, status),
|
|
612
|
+
reconciliation, the transport seam, and the optional presence and LWW helpers.
|
|
1039
613
|
|
|
1040
|
-
|
|
1041
|
-
|
|
1042
|
-
|
|
1043
|
-
|
|
1044
|
-
## What Sync owns
|
|
1045
|
-
|
|
1046
|
-
- **The replication contract.** Shared projection, durable Message subset,
|
|
1047
|
-
initial shared value, replay, operation codecs, and `journalContract()`.
|
|
1048
|
-
- **The replica.** Persisted outbox, committed base, cursor, optimistic shared
|
|
1049
|
-
projection, local operation sequence, synchronization, status, and recovery
|
|
1050
|
-
validation.
|
|
1051
|
-
- **Reconciliation.** Adopt committed operations/checkpoints, drop
|
|
1052
|
-
acknowledged/rejected pending operations, and replay what remains.
|
|
1053
|
-
- **The transport seam.** Effect `Transport` plus loopback/promise/WebSocket
|
|
1054
|
-
adapters; transport carries exchanges but does not own conflict semantics.
|
|
1055
|
-
- **Optional collaboration primitives.** Ephemeral Presence and the specialized
|
|
1056
|
-
LWW register/clock helpers.
|
|
1057
|
-
|
|
1058
|
-
Sync does **not** own:
|
|
1059
|
-
|
|
1060
|
-
- unrelated local Model fields;
|
|
1061
|
-
- server-derived disposable caches (`foldkit-remote`);
|
|
1062
|
-
- authoritative server storage/order (`foldkit-durable` or your own server);
|
|
1063
|
-
- peer-to-peer CRDT convergence;
|
|
1064
|
-
- application Message/shared-state migrations.
|
|
1065
|
-
|
|
1066
|
-
## How state changes here
|
|
1067
|
-
|
|
1068
|
-
Durable Messages replay the normal application `update` — which is where
|
|
1069
|
-
`modifyFields` lives — so deterministic replay and optimistic application are the
|
|
1070
|
-
same code path, not two reducers. Checkpoints and adopted server state
|
|
1071
|
-
arrive the other way: structurally installed through the declared writable
|
|
1072
|
-
projection. Sync already verifies that durable Messages touch only the
|
|
1073
|
-
declared projection; treat any other write as a bug.
|
|
614
|
+
Does not own the rest of the Model, server-derived caches (`foldkit-remote`),
|
|
615
|
+
the authoritative order and storage (`foldkit-durable` or your server),
|
|
616
|
+
peer-to-peer merging, or migration of your own Messages and snapshots.
|
|
1074
617
|
|
|
1075
618
|
## Limits
|
|
1076
619
|
|
|
1077
|
-
- IndexedDB is the built-in storage
|
|
1078
|
-
|
|
1079
|
-
|
|
1080
|
-
-
|
|
1081
|
-
|
|
1082
|
-
Messages commutative.
|
|
1083
|
-
- There is no operational transform / collaborative text algorithm.
|
|
1084
|
-
- Binding the socket transport and presence server to a platform WebSocket
|
|
1085
|
-
server remains application/platform glue. The sync example demonstrates a
|
|
1086
|
-
`ws` transport binding. `examples/pages` serves presence over the same socket,
|
|
1087
|
-
and shows remote carets.
|
|
1088
|
-
- Application schema migration remains application policy. Sync versions and
|
|
1089
|
-
validates its own persisted/wire envelope.
|
|
620
|
+
- IndexedDB is the only built-in storage; `Storage` is three members to
|
|
621
|
+
implement.
|
|
622
|
+
- One server order per document; no CRDT and no collaborative text algorithm.
|
|
623
|
+
- Binding the socket transport to a platform WebSocket server is glue the
|
|
624
|
+
examples show with `ws`.
|
|
1090
625
|
|
|
1091
626
|
## Older spellings
|
|
1092
627
|
|
|
1093
|
-
`Sync` groups the
|
|
1094
|
-
|
|
1095
|
-
|
|
1096
|
-
|
|
1097
|
-
|
|
1098
|
-
|
|
1099
|
-
|
|
1100
|
-
|
|
1101
|
-
- `createPresence`, `createPresenceHub`, `servePresence`,
|
|
1102
|
-
`socketPresenceChannel`, `loopbackPresenceChannel` -> `Sync.presence.make`,
|
|
1103
|
-
`.hub`, `.serve`, `.socketChannel`, `.loopbackChannel`;
|
|
1104
|
-
- `lwwRegister`, `openLwwClock` -> `Sync.lww.register`, `Sync.lww.openClock`;
|
|
1105
|
-
- `documentId('todos')` and `DocumentId.make('todos')` produce the same branded
|
|
1106
|
-
value, and likewise for `replicaId`, `opId`, `sequence`, and
|
|
1107
|
-
`localSequence`.
|
|
628
|
+
`Sync` groups the exports and wraps nothing; every name is also exported on its
|
|
629
|
+
own with the same signature: `defineSync` is `Sync.define`, `syncMetrics` is
|
|
630
|
+
`Sync.metrics`, `layerSocket`/`layerLoopback`/`layerFromPromise`/`toPromise`/
|
|
631
|
+
`serveSocket`/`nativeSocket` are `Sync.transport.*`, `createPresence`/
|
|
632
|
+
`createPresenceHub`/`servePresence`/`socketPresenceChannel`/
|
|
633
|
+
`loopbackPresenceChannel` are `Sync.presence.*`, and `lwwRegister`/`openLwwClock`
|
|
634
|
+
are `Sync.lww.*`. `documentId('todos')` and `DocumentId.make('todos')` make the
|
|
635
|
+
same branded value, as do the other id constructors.
|
|
1108
636
|
|
|
1109
637
|
## See also
|
|
1110
638
|
|
|
1111
|
-
- [Replicated state](../../docs/replication.md)
|
|
1112
|
-
|
|
1113
|
-
- [
|
|
1114
|
-
|
|
1115
|
-
|
|
1116
|
-
|
|
1117
|
-
|
|
1118
|
-
|
|
1119
|
-
|
|
1120
|
-
authorization, Mirror, and agent integration.
|
|
1121
|
-
- [`examples/sync`](../../examples/sync) — focused replica/server/transport trace.
|
|
639
|
+
- [Replicated state](../../docs/replication.md): both halves end to end, with
|
|
640
|
+
the server's exchange handler.
|
|
641
|
+
- [`foldkit-durable`](../durable): the journal and what it guarantees.
|
|
642
|
+
- [Runtime binding](../../docs/sync-runtime-binding.md): what `Sync.mount` does
|
|
643
|
+
inside the Foldkit runtime, for the curious.
|
|
644
|
+
- [`examples/todo-app`](../../examples/todo-app): a whole application over
|
|
645
|
+
`Sync.mount`, with fragments, authorization, mirrors, and an agent.
|
|
646
|
+
- [`examples/sync`](../../examples/sync): the replica, the journal, the socket,
|
|
647
|
+
and a test for every failure above.
|