foldkit-sync 0.3.0 → 0.3.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (2) hide show
  1. package/README.md +203 -114
  2. package/package.json +2 -2
package/README.md CHANGED
@@ -1,60 +1,97 @@
1
1
  # `foldkit-sync`
2
2
 
3
- A local-first replica of a shared Foldkit projection. The application's Message
4
- union and `update` stay authoritative; sync wraps them rather than introducing a
5
- second reducer.
3
+ Keeps part of a Foldkit application's Model working offline and converging with
4
+ a server. An edit applies to the local Model immediately, is written to a
5
+ persistent outbox, and is sent to the server when the network allows. The
6
+ server decides the order; every replica replays that order and ends up with the
7
+ same state.
8
+
9
+ The trick is that it is still your application. You declare which Model fields
10
+ replicate and which Messages are durable; the shared codec, the initial
11
+ snapshot, the replay reducer, and the server journal's contract are all derived
12
+ from the `update` you already wrote. There is no second reducer to keep in sync
13
+ with the first.
14
+
15
+ **Use it when** clients must keep working offline, several devices or tabs edit
16
+ the same document, and you have (or can run) a server that orders operations —
17
+ [`foldkit-durable`](https://github.com/doeixd/foldkit-plus/tree/main/packages/durable)
18
+ is one. **Not for** peer-to-peer collaboration: this is server-ordered and
19
+ single-writer-per-document, not a CRDT mesh. Concurrent edits to the same field
20
+ resolve by server arrival order unless you opt that field into the
21
+ [last-writer-wins register](#last-writer-wins-fields-experimental) below, and
22
+ there is no operational transform for collaborative text.
23
+
24
+ It is the **client half** of [replicated Foldkit state](https://github.com/doeixd/foldkit-plus/blob/main/docs/replication.md);
25
+ the [guide](https://github.com/doeixd/foldkit-plus/blob/main/docs/replication.md)
26
+ covers the mental model and when not to use it.
27
+
28
+ ## Install
29
+
30
+ ```bash
31
+ pnpm add foldkit-sync
32
+ ```
6
33
 
7
- It is the **client half** of [replicated Foldkit state](https://github.com/doeixd/foldkit-plus/blob/main/docs/replication.md).
8
- Reach for it when clients must keep working offline and converge later, with
9
- `foldkit-durable` (or any server that orders operations) as the authority. The
10
- [guide](https://github.com/doeixd/foldkit-plus/blob/main/docs/replication.md) covers the mental model and when not to use
11
- it.
34
+ `foldkit` and `effect` are peer dependencies; `foldkit-surface` comes with it.
35
+ `foldkit-durable` is the server half.
12
36
 
13
37
  ## Quick start
14
38
 
15
- `Sync.forApplication` is the Foldkit-facing layer. From one
16
- `Surface.application` and a writable projection it derives the shared schema, the
17
- durable Message subset, the initial snapshot, and replay, and exposes a read-only
18
- Surface over the same projection. The application's `update` stays the only
19
- reducer.
39
+ `forApplication` is the Foldkit-facing layer. From one `Surface.application`
40
+ and a writable projection it derives the shared schema, the durable Message
41
+ subset, the initial snapshot, and replay, and exposes a read-only Surface over
42
+ the same projection.
20
43
 
21
44
  ```ts
22
45
  import { Schema } from 'effect'
23
46
  import { defineMessageUnion } from 'foldkit/message'
24
- import { Surface } from 'foldkit-surface'
47
+ import type * as Update from 'foldkit/update'
48
+ import { MessageSet, Projection, Surface } from 'foldkit-surface'
25
49
  import { documentId, forApplication } from 'foldkit-sync'
26
50
 
27
51
  const Model = Schema.Struct({
28
52
  todos: Schema.Array(Schema.Struct({ id: Schema.String, title: Schema.String })),
29
53
  selectedTodoId: Schema.NullOr(Schema.String),
54
+ lastError: Schema.NullOr(Schema.String),
30
55
  })
56
+ type Model = typeof Model.Type
57
+
31
58
  const Message = defineMessageUnion({
32
59
  CreatedTodo: { id: Schema.String, title: Schema.String },
33
60
  RenamedTodo: { id: Schema.String, title: Schema.String },
34
61
  SelectedTodo: { id: Schema.String },
35
62
  })
36
-
37
- // `update` is the application's ordinary transition.
38
- const App = Surface.application({
39
- Model,
40
- Message,
41
- initial: { todos: [], selectedTodoId: null },
42
- update,
43
- })
44
-
45
- const Todos = Projection.pick(App.fields.todos)
46
- const TodoChanges = MessageSet.make(App, [Message.CreatedTodo, Message.RenamedTodo])
63
+ type Message = typeof Message.Type
64
+
65
+ const initial: Model = { todos: [], selectedTodoId: null, lastError: null }
66
+
67
+ type Return = Update.Return<Model, Message>
68
+ // The application's ordinary transition. Sync does not add a second one.
69
+ const update = (model: Model, message: Message): Return =>
70
+ Message.match<Return>(message, {
71
+ CreatedTodo: ({ id, title }) => ({
72
+ model: { ...model, todos: [...model.todos, { id, title }] },
73
+ }),
74
+ RenamedTodo: ({ id, title }) => ({
75
+ model: {
76
+ ...model,
77
+ todos: model.todos.map(todo => (todo.id === id ? { ...todo, title } : todo)),
78
+ },
79
+ }),
80
+ SelectedTodo: ({ id }) => ({ model: { ...model, selectedTodoId: id } }),
81
+ })
82
+
83
+ const App = Surface.application({ Model, Message, initial, update })
47
84
 
48
85
  const TodoSync = forApplication(App).make({
49
86
  documentId: documentId('todos'),
50
- shared: Todos, // the shared codec, read, and write
51
- durable: TodoChanges, // the durable subset
87
+ shared: Projection.pick(App.fields.todos), // the codec, read, and write
88
+ durable: MessageSet.make(App, [Message.CreatedTodo, Message.RenamedTodo]),
52
89
  })
53
90
  ```
54
91
 
55
- Only `todos` is replicated; `selectedTodoId` stays local. The derived `replay`
56
- installs the shared slice into the application's initial Model, applies the
57
- durable Message through `update`, and reads the shared slice back.
92
+ Only `todos` is replicated; `selectedTodoId` and `lastError` stay local. The
93
+ derived `replay` installs the shared slice into the application's initial Model,
94
+ applies the durable Message through `update`, and reads the shared slice back.
58
95
 
59
96
  A durable Message must therefore be a deterministic, state-only transition of
60
97
  the shared projection. Replay refuses one whose `update` returns a Command (a
@@ -69,53 +106,82 @@ On the server, the same contract produces the journal's codecs and reducer, so
69
106
  neither is written twice:
70
107
 
71
108
  ```ts
72
- import { makeJournal } from 'foldkit-durable'
73
-
74
- const journal = yield* makeJournal({
75
- ...TodoSync.journalContract(), // operation/snapshot codecs, empty, reduce
76
- file: 'todos.sqlite',
77
- opId: operation => operation.opId,
78
- actorId: principal => principal.actorId,
79
- })
109
+ import { Effect } from 'effect'
110
+ import { actorId, documentId, makeJournal, opId } from 'foldkit-durable'
111
+
112
+ type Principal = { readonly actorId: string }
113
+
114
+ const server = Effect.gen(function* () {
115
+ const journal = yield* makeJournal({
116
+ // Operation and snapshot codecs, `empty`, `reduce`, and `authorize` if declared.
117
+ ...TodoSync.journalContract(),
118
+ file: 'todos.sqlite',
119
+ // The two packages brand their ids separately, so re-brand at the seam.
120
+ opId: operation => opId(operation.opId),
121
+ actorId: (principal: Principal) => actorId(principal.actorId),
122
+ })
123
+ return yield* journal.load(documentId('todos'))
124
+ }).pipe(Effect.scoped)
80
125
  ```
81
126
 
82
127
  ### Mounting
83
128
 
84
- `Sync.mount` runs the application over an open replica with one reducer. A
85
- durable Message is applied at once through the application's `update` and
86
- persisted afterwards in a Command; the shared slice is re-installed from the
87
- replica when an exchange or a rejection changes it, or when a persist fails.
129
+ `mount` runs the application over an open replica with one reducer. A durable
130
+ Message is applied at once through the application's `update` and persisted
131
+ afterwards in a Command; the shared slice is re-installed from the replica when
132
+ an exchange or a rejection changes it, or when a persist fails.
88
133
 
89
134
  ```ts
90
- const replica = yield* TodoSync.openReplica(replicaId('tab-1'), yield* indexedDb('todos'))
135
+ import { Effect, Scope } from 'effect'
136
+ import { indexedDb, layerSocket, mount, replicaId } from 'foldkit-sync'
137
+
138
+ const container = document.getElementById('app')! // Foldkit needs the element to have an id
139
+
140
+ const scope = Effect.runSync(Scope.make())
141
+ const storage = Effect.runSync(Effect.provideService(indexedDb('todos/tab-1'), Scope.Scope, scope))
142
+ const replica = Effect.runSync(TodoSync.openReplica(replicaId('tab-1'), storage))
143
+
91
144
  const mounted = mount(App, TodoSync, {
92
145
  replica,
93
- container: document.getElementById('app')!, // Foldkit needs the id
94
- view: (model, h) => ({ title: 'Todos', body: h.ul([], model.todos.map(todo => h.li([], [todo.title]))) }),
146
+ container,
147
+ view: (model, h) => ({
148
+ title: 'Todos',
149
+ body: h.ul(
150
+ [],
151
+ model.todos.map(todo => h.li([], [todo.title])),
152
+ ),
153
+ }),
95
154
  onPersistenceFailure: (model, error) => ({ ...model, lastError: error.message }),
96
155
  })
97
- mounted.dispatch(Message.CreatedTodo({ id, title: 'Milk' }))
156
+
157
+ // The exchange loop: once, then after every submit, retrying a failed exchange.
158
+ Effect.runFork(Effect.provide(replica.start, layerSocket({ url: 'wss://example.com/sync' })))
159
+
160
+ mounted.dispatch(Message.CreatedTodo({ id: crypto.randomUUID(), title: 'Milk' }))
98
161
  mounted.model() // the Model after the last transition
99
162
  await mounted.dispose() // waits for in-flight persists; the replica stays open
100
163
  ```
101
164
 
102
- `url` routes the URL through the application: `init(model, url)` reduces it
103
- into the Model before the first render, `onUrlChange(url)` names the Message
104
- for every navigation, and `onUrlRequest` the Message for a link click (omitted,
105
- the mount follows the link: an internal one pushed, an external one loaded).
106
- A `foldkit-mirror` URL mirror plugs in as `url: { init: (model, url) =>
107
- Filters.reduce(model, url), onUrlChange: url => Message.UrlChanged({ url }) }`;
165
+ The optional `url` option routes the URL through the application:
166
+ `init(model, url)` reduces it into the Model before the first render,
167
+ `onUrlChange(url)` names the Message for every navigation, and `onUrlRequest`
168
+ the Message for a link click (omitted, the mount follows the link itself: an
169
+ internal one pushed, an external one loaded). A `foldkit-mirror` URL mirror
170
+ plugs in there as
171
+ `url: { init: (model, url) => Filters.reduce(model, url), onUrlChange: url => Message.UrlChanged({ url }) }`;
108
172
  `subscriptions` and `resources` carry its write entry and the store layer.
109
173
 
110
174
  `mounted.model`, `mounted.dispatch`, `mounted.subscribe`, and `mounted.observe`
111
175
  are the host an agent binds to; `observe` reports every application Message the
112
176
  runtime applies, which a capability with a `completion` contract needs. See
113
- [docs/sync-runtime-binding.md](../../docs/sync-runtime-binding.md) for what the
177
+ [docs/sync-runtime-binding.md](https://github.com/doeixd/foldkit-plus/blob/main/docs/sync-runtime-binding.md) for what the
114
178
  mount guarantees and why no Foldkit change is required.
115
179
 
116
180
  ### Fragments
117
181
 
118
- A large application declares one fragment per feature and composes them:
182
+ A large application declares one fragment per feature and composes them. `App`
183
+ below is a wider application than the quick start's: it also has a `members`
184
+ field and an `Invited` Message.
119
185
 
120
186
  ```ts
121
187
  const Sync = forApplication(App)
@@ -138,66 +204,79 @@ over the same field are a different mistake, which `Module.validate` reports.
138
204
  ### Authorization
139
205
 
140
206
  Policy attaches to the contract per durable variant and compiles into the
141
- journal contract, so the server applies it with no glue:
207
+ journal contract, so the server applies it with no glue. This is the same
208
+ `Board` as above, with a principal fixed and one rule added:
142
209
 
143
210
  ```ts
144
- const Sync = forApplication(App).withPrincipal<{ readonly role: 'admin' | 'guest' }>()
145
- const Board = Sync.make({
211
+ const Authorized = forApplication(App).withPrincipal<{ readonly role: 'admin' | 'guest' }>()
212
+ const Board = Authorized.make({
146
213
  documentId: documentId('board'),
147
- ...Sync.compose(Todos, Members),
214
+ ...Authorized.compose(Todos, Members),
148
215
  authorize: {
149
216
  // `message` is exactly `RenamedTodo`; `shared` is the authoritative snapshot.
150
217
  RenamedTodo: ({ principal, message, shared }) =>
151
218
  principal.role === 'admin' && shared.todos.some(todo => todo.id === message.id),
152
219
  },
153
220
  })
154
-
155
- const journal = yield* makeJournal({ ...Board.journalContract(), file, opId, actorId })
156
221
  ```
157
222
 
158
- A durable variant without a rule is allowed; a key that is not a durable tag is
159
- a compile error. The rule runs inside the append transaction against the
160
- snapshot, so keep it synchronous and local. The client does not run it: a
161
- replica has no principal, and a refused operation comes back as a rejection.
223
+ `Board.journalContract()` then carries the compiled rules in the shape
224
+ `makeJournal` takes. A durable variant without a rule is allowed; a key that is
225
+ not a durable tag is a compile error. The rule runs inside the append
226
+ transaction against the snapshot, so keep it synchronous and local. The client
227
+ does not run it: a replica has no principal, and a refused operation comes back
228
+ as a rejection.
162
229
 
163
230
  ### Lower level
164
231
 
165
- `Sync.forApplication(App).make` compiles down to `defineSync`, the protocol
232
+ `forApplication(App).make` compiles down to `defineSync`, the protocol
166
233
  primitive. Use `defineSync` directly when there is no Foldkit application to
167
234
  derive the contract from — a non-Foldkit client, or a hand-written projection.
168
235
 
169
236
  ```ts
170
- import { Effect } from 'effect'
171
- import { defineSync, indexedDb, layerFromPromise, replicaId } from 'foldkit-sync'
172
-
173
- const Sync = defineSync({ ... })
174
-
175
- const replica = await Effect.runPromise(
176
- Effect.gen(function* () {
177
- const storage = yield* indexedDb('todos-tab-1')
178
- return yield* Sync.openReplica(replicaId('tab-1'), storage)
179
- }),
180
- )
181
- await Effect.runPromise(
182
- replica.submit(Message.CreatedTodo({ id: crypto.randomUUID(), title: 'Milk' })),
183
- )
184
- await Effect.runPromise(Effect.provide(replica.synchronize, layerFromPromise(transport)))
237
+ import { Effect, Schema } from 'effect'
238
+ import {
239
+ defineSync,
240
+ documentId,
241
+ indexedDb,
242
+ layerFromPromise,
243
+ replicaId,
244
+ type TransportClient,
245
+ } from 'foldkit-sync'
246
+
247
+ const Shared = Schema.Struct({ todos: Schema.Array(Schema.String) })
248
+ const Renamed = Schema.Struct({ _tag: Schema.Literal('Renamed'), title: Schema.String })
249
+ declare const transport: TransportClient // the application's own server client
250
+
251
+ const Protocol = defineSync({
252
+ documentId: documentId('todos'),
253
+ message: Renamed,
254
+ shared: Shared,
255
+ empty: { todos: [] },
256
+ durable: () => true,
257
+ replay: (shared, message) => ({ todos: [...shared.todos, message.title] }),
258
+ })
185
259
 
186
- const shared = Effect.runSync(replica.shared)
260
+ const program = Effect.gen(function* () {
261
+ const storage = yield* indexedDb('todos-tab-1')
262
+ const replica = yield* Protocol.openReplica(replicaId('tab-1'), storage)
263
+ yield* replica.submit({ _tag: 'Renamed', title: 'Milk' })
264
+ yield* Effect.provide(replica.synchronize, layerFromPromise(transport))
265
+ return yield* replica.shared
266
+ }).pipe(Effect.scoped)
187
267
  ```
188
268
 
189
269
  ## What it owns
190
270
 
191
- - The Foldkit-facing contract (`Sync.forApplication(App).make`, optionally with
271
+ - The Foldkit-facing contract (`forApplication(App).make`, optionally with
192
272
  a custom `replay`): derives the shared projection, the durable Message subset, the
193
273
  initial snapshot, and the journal contract from the application, so none is
194
274
  declared twice. `Projection.pick`/`Projection.compose` build the writable projection;
195
275
  `TodoSync.journalContract()` builds the durable codecs and reducer.
196
276
  - The operation envelope: `replicaId:localSequence` identity, `baseCursor`, and
197
- protocol/schema versions. The wire codecs live under `Sync.codec`
198
- (`normalizeOperation`, `operationFrom`, `committedFrom`, `decodeExchange`) for a
199
- custom transport or server; a server-committed operation is a
200
- `CommittedOperation`.
277
+ protocol/schema versions. The contract's `codec` (`normalizeOperation`,
278
+ `operationFrom`, `committedFrom`, `decodeExchange`) is there for a custom
279
+ transport or server; a server-committed operation is a `CommittedOperation`.
201
280
  - A persisted outbox and optimistic projection: a durable Message is visible
202
281
  immediately and rebased onto the authoritative order.
203
282
  - Reconciliation: committed operations are replayed in order, acknowledged or
@@ -246,7 +325,7 @@ const shared = Effect.runSync(replica.shared)
246
325
  server is left to the application; the sync example shows a `ws` one for the
247
326
  transport (presence over `ws` is not wired there yet).
248
327
 
249
- ## Last-writer-wins fields (M8, experimental)
328
+ ## Last-writer-wins fields (experimental)
250
329
 
251
330
  Use `lwwRegister` when a field's winner should depend on a write's logical time
252
331
  instead of the order offline clients reconnect. It returns a schema and a pure
@@ -254,7 +333,7 @@ instead of the order offline clients reconnect. It returns a schema and a pure
254
333
 
255
334
  ```ts
256
335
  import { Schema } from 'effect'
257
- import { lwwRegister } from 'foldkit-sync'
336
+ import { lwwRegister, replicaId } from 'foldkit-sync'
258
337
 
259
338
  const Title = lwwRegister(Schema.String)
260
339
  const Shared = Schema.Struct({ title: Title.schema })
@@ -267,7 +346,7 @@ const update = (model: typeof Shared.Type, message: typeof Renamed.Type) => ({
267
346
 
268
347
  const message: typeof Renamed.Type = {
269
348
  _tag: 'Renamed',
270
- title: { stamp: { counter: 1, replicaId: 'tab-a' }, value: 'Milk' },
349
+ title: { stamp: { counter: 1, replicaId: replicaId('tab-a') }, value: 'Milk' },
271
350
  }
272
351
  ```
273
352
 
@@ -282,38 +361,38 @@ write cannot cause timestamp reuse after reload:
282
361
 
283
362
  ```ts
284
363
  import { Effect } from 'effect'
285
- import { indexedDb, openLwwClock } from 'foldkit-sync'
364
+ import { documentId, indexedDb, openLwwClock, replicaId, type Replica } from 'foldkit-sync'
286
365
 
287
- const clock = await Effect.runPromise(
366
+ type Shared = typeof Shared.Type
367
+ type Renamed = typeof Renamed.Type
368
+
369
+ const rename = (replica: Replica<Renamed, Shared>, title: string) =>
288
370
  Effect.gen(function* () {
371
+ // A separate database from the replica's outbox; one clock per document/writer.
289
372
  const storage = yield* indexedDb('todos-tab-a-clock')
290
- return yield* openLwwClock({ documentId: 'todos', replicaId: 'tab-a', storage })
291
- }),
292
- )
293
- try {
294
- const stamp = await Effect.runPromise(
295
- clock.next(Effect.runSync(replica.shared).title.stamp.counter),
296
- )
297
- await Effect.runPromise(
298
- replica.submit({ _tag: 'Renamed', title: { stamp, value: 'Milk' } }),
299
- )
300
- } finally {
301
- await Effect.runPromise(clock.close)
302
- }
373
+ const clock = yield* openLwwClock({
374
+ documentId: documentId('todos'),
375
+ replicaId: replicaId('tab-a'),
376
+ storage,
377
+ })
378
+ const shared = yield* replica.shared
379
+ const stamp = yield* clock.next(shared.title.stamp.counter)
380
+ yield* replica.submit({ _tag: 'Renamed', title: { stamp, value: title } })
381
+ yield* clock.close
382
+ }).pipe(Effect.scoped)
303
383
  ```
304
384
 
305
- Use a **separate database** from the replica's outbox, with one clock per
306
- document/writer. `next(observedCounter)` returns only after persisting a counter
307
- greater than both its saved value and the supplied observation. Pass the latest
308
- observed counter for the field being edited; when causality spans several
309
- registers, pass their maximum. Calling `next()` without an observation advances
310
- only the saved local counter. The clock does not inspect application state.
385
+ `next(observedCounter)` returns only after persisting a counter greater than
386
+ both its saved value and the supplied observation. Pass the latest observed
387
+ counter for the field being edited; when causality spans several registers, pass
388
+ their maximum. Calling `next()` without an observation advances only the saved
389
+ local counter. The clock does not inspect application state.
311
390
 
312
391
  Allocations on one handle are serialized. Separate handles use storage's
313
392
  compare-and-swap check; a stale handle fails and must be closed and reopened.
314
393
  Storage failures reject allocation without returning a stamp. Counter gaps after
315
394
  a crash or failed submission are harmless. Counters are nonnegative safe
316
- integers; exhaustion fails before writing. `close()` waits for accepted
395
+ integers; exhaustion fails before writing. `close` waits for accepted
317
396
  allocations and refuses new ones. Opening takes ownership of storage and closes
318
397
  it on initialization failure too.
319
398
 
@@ -340,6 +419,16 @@ identity. Applications must enforce writer ownership and clock policy at admissi
340
419
  where needed. A losing write can still trigger a Command if the application's
341
420
  update produces one; this helper only resolves state.
342
421
 
343
- These M8 helpers do not change the todo example's persisted schema or claim
344
- arbitrary Messages commute. Specialized sets, counters and collaborative text
345
- remain future work.
422
+ These helpers do not claim arbitrary Messages commute. Specialized sets,
423
+ counters, and collaborative text remain future work.
424
+
425
+ ## See also
426
+
427
+ - [Replicated state](https://github.com/doeixd/foldkit-plus/blob/main/docs/replication.md) — the mental model, and
428
+ when not to use it.
429
+ - [Runtime binding](https://github.com/doeixd/foldkit-plus/blob/main/docs/sync-runtime-binding.md) — what
430
+ `mount` guarantees, and why no Foldkit change is required.
431
+ - [`foldkit-durable`](https://github.com/doeixd/foldkit-plus/tree/main/packages/durable) — the server half; [`foldkit-mirror`](https://github.com/doeixd/foldkit-plus/tree/main/packages/mirror) plugs into the mount's
432
+ `url` option.
433
+ - [`examples/todo-app`](https://github.com/doeixd/foldkit-plus/tree/main/examples/todo-app) — two fragments, owner-only rules, and a browser mount;
434
+ [`examples/sync`](https://github.com/doeixd/foldkit-plus/tree/main/examples/sync) is the in-process trace.
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "foldkit-sync",
3
- "version": "0.3.0",
3
+ "version": "0.3.1",
4
4
  "description": "A local-first replica of a shared Foldkit projection: offline outbox, optimistic updates, and reconciliation.",
5
5
  "keywords": [
6
6
  "foldkit",
@@ -38,7 +38,7 @@
38
38
  "LICENSE"
39
39
  ],
40
40
  "dependencies": {
41
- "foldkit-surface": "0.1.0"
41
+ "foldkit-surface": "0.1.1"
42
42
  },
43
43
  "peerDependencies": {
44
44
  "effect": "^4.0.0-rc.112",