foldkit-sync 0.1.0 → 0.3.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -4,14 +4,179 @@ A local-first replica of a shared Foldkit projection. The application's Message
4
4
  union and `update` stay authoritative; sync wraps them rather than introducing a
5
5
  second reducer.
6
6
 
7
+ It is the **client half** of [replicated Foldkit state](https://github.com/doeixd/foldkit-plus/blob/main/docs/replication.md).
8
+ Reach for it when clients must keep working offline and converge later, with
9
+ `foldkit-durable` (or any server that orders operations) as the authority. The
10
+ [guide](https://github.com/doeixd/foldkit-plus/blob/main/docs/replication.md) covers the mental model and when not to use
11
+ it.
12
+
13
+ ## Quick start
14
+
15
+ `Sync.forApplication` is the Foldkit-facing layer. From one
16
+ `Surface.application` and a writable projection it derives the shared schema, the
17
+ durable Message subset, the initial snapshot, and replay, and exposes a read-only
18
+ Surface over the same projection. The application's `update` stays the only
19
+ reducer.
20
+
21
+ ```ts
22
+ import { Schema } from 'effect'
23
+ import { defineMessageUnion } from 'foldkit/message'
24
+ import { Surface } from 'foldkit-surface'
25
+ import { documentId, forApplication } from 'foldkit-sync'
26
+
27
+ const Model = Schema.Struct({
28
+ todos: Schema.Array(Schema.Struct({ id: Schema.String, title: Schema.String })),
29
+ selectedTodoId: Schema.NullOr(Schema.String),
30
+ })
31
+ const Message = defineMessageUnion({
32
+ CreatedTodo: { id: Schema.String, title: Schema.String },
33
+ RenamedTodo: { id: Schema.String, title: Schema.String },
34
+ SelectedTodo: { id: Schema.String },
35
+ })
36
+
37
+ // `update` is the application's ordinary transition.
38
+ const App = Surface.application({
39
+ Model,
40
+ Message,
41
+ initial: { todos: [], selectedTodoId: null },
42
+ update,
43
+ })
44
+
45
+ const Todos = Projection.pick(App.fields.todos)
46
+ const TodoChanges = MessageSet.make(App, [Message.CreatedTodo, Message.RenamedTodo])
47
+
48
+ const TodoSync = forApplication(App).make({
49
+ documentId: documentId('todos'),
50
+ shared: Todos, // the shared codec, read, and write
51
+ durable: TodoChanges, // the durable subset
52
+ })
53
+ ```
54
+
55
+ Only `todos` is replicated; `selectedTodoId` stays local. The derived `replay`
56
+ installs the shared slice into the application's initial Model, applies the
57
+ durable Message through `update`, and reads the shared slice back.
58
+
59
+ A durable Message must therefore be a deterministic, state-only transition of
60
+ the shared projection. Replay refuses one whose `update` returns a Command (a
61
+ live effect cannot be replayed) or changes a field outside the projection (the
62
+ change would be silently lost), naming the Message and the fields. A Message that
63
+ needs an effect stays local and emits a durable fact once the effect settles:
64
+ `RequestedChargeCard` runs the Command; `CardCharged` is what replicates.
65
+ Pass `replay` to `make` when an application needs a custom reducer over the
66
+ shared slice; a custom replay is not guarded.
67
+
68
+ On the server, the same contract produces the journal's codecs and reducer, so
69
+ neither is written twice:
70
+
71
+ ```ts
72
+ import { makeJournal } from 'foldkit-durable'
73
+
74
+ const journal = yield* makeJournal({
75
+ ...TodoSync.journalContract(), // operation/snapshot codecs, empty, reduce
76
+ file: 'todos.sqlite',
77
+ opId: operation => operation.opId,
78
+ actorId: principal => principal.actorId,
79
+ })
80
+ ```
81
+
82
+ ### Mounting
83
+
84
+ `Sync.mount` runs the application over an open replica with one reducer. A
85
+ durable Message is applied at once through the application's `update` and
86
+ persisted afterwards in a Command; the shared slice is re-installed from the
87
+ replica when an exchange or a rejection changes it, or when a persist fails.
88
+
89
+ ```ts
90
+ const replica = yield* TodoSync.openReplica(replicaId('tab-1'), yield* indexedDb('todos'))
91
+ const mounted = mount(App, TodoSync, {
92
+ replica,
93
+ container: document.getElementById('app')!, // Foldkit needs the id
94
+ view: (model, h) => ({ title: 'Todos', body: h.ul([], model.todos.map(todo => h.li([], [todo.title]))) }),
95
+ onPersistenceFailure: (model, error) => ({ ...model, lastError: error.message }),
96
+ })
97
+ mounted.dispatch(Message.CreatedTodo({ id, title: 'Milk' }))
98
+ mounted.model() // the Model after the last transition
99
+ await mounted.dispose() // waits for in-flight persists; the replica stays open
100
+ ```
101
+
102
+ `url` routes the URL through the application: `init(model, url)` reduces it
103
+ into the Model before the first render, `onUrlChange(url)` names the Message
104
+ for every navigation, and `onUrlRequest` the Message for a link click (omitted,
105
+ the mount follows the link: an internal one pushed, an external one loaded).
106
+ A `foldkit-mirror` URL mirror plugs in as `url: { init: (model, url) =>
107
+ Filters.reduce(model, url), onUrlChange: url => Message.UrlChanged({ url }) }`;
108
+ `subscriptions` and `resources` carry its write entry and the store layer.
109
+
110
+ `mounted.model`, `mounted.dispatch`, `mounted.subscribe`, and `mounted.observe`
111
+ are the host an agent binds to; `observe` reports every application Message the
112
+ runtime applies, which a capability with a `completion` contract needs. See
113
+ [docs/sync-runtime-binding.md](../../docs/sync-runtime-binding.md) for what the
114
+ mount guarantees and why no Foldkit change is required.
115
+
116
+ ### Fragments
117
+
118
+ A large application declares one fragment per feature and composes them:
119
+
120
+ ```ts
121
+ const Sync = forApplication(App)
122
+ const Todos = Sync.fragment({
123
+ shared: Projection.pick(App.fields.todos),
124
+ durable: MessageSet.make(App, [Message.CreatedTodo, Message.RenamedTodo]),
125
+ })
126
+ const Members = Sync.fragment({
127
+ shared: Projection.pick(App.fields.members),
128
+ durable: MessageSet.make(App, [Message.Invited]),
129
+ })
130
+ const Board = Sync.make({ documentId: documentId('board'), ...Sync.compose(Todos, Members) })
131
+ ```
132
+
133
+ `compose` merges the shared projections and the durable subsets and infers the
134
+ merged shape. A field declared twice with a different codec, a Message declared
135
+ durable twice, or a fragment from another application throws. Two contracts
136
+ over the same field are a different mistake, which `Module.validate` reports.
137
+
138
+ ### Authorization
139
+
140
+ Policy attaches to the contract per durable variant and compiles into the
141
+ journal contract, so the server applies it with no glue:
142
+
143
+ ```ts
144
+ const Sync = forApplication(App).withPrincipal<{ readonly role: 'admin' | 'guest' }>()
145
+ const Board = Sync.make({
146
+ documentId: documentId('board'),
147
+ ...Sync.compose(Todos, Members),
148
+ authorize: {
149
+ // `message` is exactly `RenamedTodo`; `shared` is the authoritative snapshot.
150
+ RenamedTodo: ({ principal, message, shared }) =>
151
+ principal.role === 'admin' && shared.todos.some(todo => todo.id === message.id),
152
+ },
153
+ })
154
+
155
+ const journal = yield* makeJournal({ ...Board.journalContract(), file, opId, actorId })
156
+ ```
157
+
158
+ A durable variant without a rule is allowed; a key that is not a durable tag is
159
+ a compile error. The rule runs inside the append transaction against the
160
+ snapshot, so keep it synchronous and local. The client does not run it: a
161
+ replica has no principal, and a refused operation comes back as a rejection.
162
+
163
+ ### Lower level
164
+
165
+ `Sync.forApplication(App).make` compiles down to `defineSync`, the protocol
166
+ primitive. Use `defineSync` directly when there is no Foldkit application to
167
+ derive the contract from — a non-Foldkit client, or a hand-written projection.
168
+
7
169
  ```ts
8
170
  import { Effect } from 'effect'
9
- import { defineSync, indexedDb, layerFromPromise } from 'foldkit-sync'
171
+ import { defineSync, indexedDb, layerFromPromise, replicaId } from 'foldkit-sync'
10
172
 
11
173
  const Sync = defineSync({ ... })
12
174
 
13
175
  const replica = await Effect.runPromise(
14
- Sync.openReplica('tab-1', await indexedDb('todos-tab-1')),
176
+ Effect.gen(function* () {
177
+ const storage = yield* indexedDb('todos-tab-1')
178
+ return yield* Sync.openReplica(replicaId('tab-1'), storage)
179
+ }),
15
180
  )
16
181
  await Effect.runPromise(
17
182
  replica.submit(Message.CreatedTodo({ id: crypto.randomUUID(), title: 'Milk' })),
@@ -23,22 +188,45 @@ const shared = Effect.runSync(replica.shared)
23
188
 
24
189
  ## What it owns
25
190
 
191
+ - The Foldkit-facing contract (`Sync.forApplication(App).make`, optionally with
192
+ a custom `replay`): derives the shared projection, the durable Message subset, the
193
+ initial snapshot, and the journal contract from the application, so none is
194
+ declared twice. `Projection.pick`/`Projection.compose` build the writable projection;
195
+ `TodoSync.journalContract()` builds the durable codecs and reducer.
26
196
  - The operation envelope: `replicaId:localSequence` identity, `baseCursor`, and
27
- protocol/schema versions.
197
+ protocol/schema versions. The wire codecs live under `Sync.codec`
198
+ (`normalizeOperation`, `operationFrom`, `committedFrom`, `decodeExchange`) for a
199
+ custom transport or server; a server-committed operation is a
200
+ `CommittedOperation`.
28
201
  - A persisted outbox and optimistic projection: a durable Message is visible
29
202
  immediately and rebased onto the authoritative order.
30
203
  - Reconciliation: committed operations are replayed in order, acknowledged or
31
204
  refused pending entries are dropped, and a checkpoint replaces the compacted
32
- prefix.
205
+ prefix. `replica.start` is the exchange loop: fork it to exchange once and then
206
+ after every `submit`, with a transport failure recorded in `status.lastError`
207
+ and retried on the next wake, so the application does not hand-roll it.
208
+ - A redacted status (`replica.status`): the pending count, the cursor, the last
209
+ exchange failure, and the operations the server refused — enough for a UI to
210
+ explain and recover without exposing Messages or the Model. `statusChanges` is
211
+ the same status, emitted on subscribe and re-emitted after every submit and
212
+ exchange. `changes` carries the status and the optimistic shared value together
213
+ from one read, so a UI holds one subscription instead of two.
33
214
  - Strict decoding: an operation is always validated with the application's
34
215
  Message schema, and a Message the contract does not call durable is refused.
216
+ - Branded positions: `Sequence` (a committed document position) and
217
+ `LocalSequence` (a replica's own 1-based counter) cannot be confused, so a
218
+ client counter is never passed where a committed position is expected.
35
219
  - Presence (`createPresence`): an ephemeral, TTL'd peer registry, deliberately
36
220
  outside the durable log. A peer that stops refreshing is dropped, not
37
- replayed. Every value is decoded through the required `decodeValue` before it
38
- is stored, so a hostile peer cannot inject a value your `Update` type does not
39
- describe. Presence can travel over a socket — `socketPresenceChannel` on the
40
- client and `servePresence` fanning through a `createPresenceHub` on the server
41
- — or in-process via `loopbackPresenceChannel`.
221
+ replayed. It is an Effect driven by the `Clock` (so a `TestClock` makes the
222
+ TTL deterministic), peers live in a `Ref`, and the channel is a `PubSub`.
223
+ Every value is decoded through the required `decodeValue` before it is stored,
224
+ so a hostile peer cannot inject a value your `Update` type does not describe.
225
+ Presence can travel over a socket — `socketPresenceChannel` on the client and
226
+ `servePresence` fanning through a `createPresenceHub` on the server — or
227
+ in-process via `loopbackPresenceChannel`. `presence.changes` is a `Stream` that
228
+ emits the current peers and re-emits them on every change, alongside the
229
+ `subscribe` callback.
42
230
  - The transport seam (`Transport`): an Effect service with a loopback layer, a
43
231
  bridge to and from the promise client the replica speaks, and a WebSocket
44
232
  client layer. The socket reconnects on an exponential, jittered backoff and
@@ -96,18 +284,21 @@ write cannot cause timestamp reuse after reload:
96
284
  import { Effect } from 'effect'
97
285
  import { indexedDb, openLwwClock } from 'foldkit-sync'
98
286
 
99
- const clock = await openLwwClock({
100
- documentId: 'todos',
101
- replicaId: 'tab-a',
102
- storage: await indexedDb('todos-tab-a-clock'),
103
- })
287
+ const clock = await Effect.runPromise(
288
+ Effect.gen(function* () {
289
+ const storage = yield* indexedDb('todos-tab-a-clock')
290
+ return yield* openLwwClock({ documentId: 'todos', replicaId: 'tab-a', storage })
291
+ }),
292
+ )
104
293
  try {
105
- const stamp = await clock.next(Effect.runSync(replica.shared).title.stamp.counter)
294
+ const stamp = await Effect.runPromise(
295
+ clock.next(Effect.runSync(replica.shared).title.stamp.counter),
296
+ )
106
297
  await Effect.runPromise(
107
298
  replica.submit({ _tag: 'Renamed', title: { stamp, value: 'Milk' } }),
108
299
  )
109
300
  } finally {
110
- await clock.close()
301
+ await Effect.runPromise(clock.close)
111
302
  }
112
303
  ```
113
304