foldkit-sync 0.8.0 → 0.10.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +238 -26
- package/dist/do.d.mts +60 -0
- package/dist/do.d.mts.map +1 -0
- package/dist/do.mjs +59 -0
- package/dist/do.mjs.map +1 -0
- package/dist/entity.d.mts +183 -0
- package/dist/entity.d.mts.map +1 -0
- package/dist/entity.mjs +143 -0
- package/dist/entity.mjs.map +1 -0
- package/dist/index.d.mts +45 -622
- package/dist/index.d.mts.map +1 -1
- package/dist/index.mjs +168 -340
- package/dist/index.mjs.map +1 -1
- package/dist/journal.d.mts +142 -0
- package/dist/journal.d.mts.map +1 -0
- package/dist/journal.mjs +212 -0
- package/dist/journal.mjs.map +1 -0
- package/dist/make-h5yD8STj.d.mts +714 -0
- package/dist/make-h5yD8STj.d.mts.map +1 -0
- package/dist/remote.d.mts +123 -0
- package/dist/remote.d.mts.map +1 -0
- package/dist/remote.mjs +163 -0
- package/dist/remote.mjs.map +1 -0
- package/dist/transport-DiKXy5zx.mjs +412 -0
- package/dist/transport-DiKXy5zx.mjs.map +1 -0
- package/package.json +38 -4
package/README.md
CHANGED
|
@@ -213,8 +213,13 @@ What each call does, and does not do:
|
|
|
213
213
|
Reload the page: the todos come back from the outbox and the committed
|
|
214
214
|
snapshot, the selection does not. That is the shared/local line, visible.
|
|
215
215
|
|
|
216
|
-
`mounted` also exposes `model()`, `subscribe`, `observe`, `committed`,
|
|
217
|
-
`dispose()`, which waits for in-flight persists and leaves the
|
|
216
|
+
`mounted` also exposes `model()`, `subscribe`, `observe`, `committed`,
|
|
217
|
+
`settled()`, and `dispose()`, which waits for in-flight persists and leaves the
|
|
218
|
+
replica open. `settled()` resolves once the Model has caught up with the
|
|
219
|
+
replica: every dispatched Message applied, no persist unanswered, and every
|
|
220
|
+
replica status applied. It waits for no frame and no I/O, so a test pairs it
|
|
221
|
+
with `Frames.track` from `foldkit-mixins/testing`: `await
|
|
222
|
+
frames.settle(mounted)`.
|
|
218
223
|
Close the replica with its own scope.
|
|
219
224
|
|
|
220
225
|
## The server
|
|
@@ -225,27 +230,115 @@ commits what it accepts, and returns what the replica is missing. With
|
|
|
225
230
|
snapshot, reducer, and any [authorization](#authorize-on-the-server):
|
|
226
231
|
|
|
227
232
|
```ts
|
|
228
|
-
import {
|
|
233
|
+
import { Effect } from 'effect'
|
|
234
|
+
import { ActorId, Journal, OpId } from 'foldkit-durable'
|
|
235
|
+
import type { SocketLike } from 'foldkit-sync'
|
|
236
|
+
import { serveJournal } from 'foldkit-sync/journal'
|
|
229
237
|
|
|
230
238
|
type Principal = { readonly actorId: string }
|
|
231
239
|
|
|
232
|
-
const
|
|
233
|
-
|
|
234
|
-
|
|
235
|
-
|
|
236
|
-
|
|
240
|
+
const server = Effect.gen(function* () {
|
|
241
|
+
const journal = yield* Journal.make({
|
|
242
|
+
...TodoSync.journalContract(),
|
|
243
|
+
file: 'todos.sqlite',
|
|
244
|
+
opId: operation => OpId.make(operation.opId),
|
|
245
|
+
actorId: (principal: Principal) => ActorId.make(principal.actorId),
|
|
246
|
+
})
|
|
247
|
+
// One accepted socket, as the principal its authentication established.
|
|
248
|
+
return (socket: SocketLike, principal: Principal) =>
|
|
249
|
+
serveJournal(socket, { sync: TodoSync, journal, principal })
|
|
237
250
|
})
|
|
238
251
|
```
|
|
239
252
|
|
|
240
|
-
|
|
241
|
-
|
|
242
|
-
|
|
243
|
-
|
|
244
|
-
|
|
245
|
-
|
|
253
|
+
`foldkit-sync/journal` is the exchange over the journal (`foldkit-durable`,
|
|
254
|
+
an optional peer of this package):
|
|
255
|
+
|
|
256
|
+
- **`serveJournal(socket, options)`** answers one socket's exchanges and sends
|
|
257
|
+
a notice after each commit, so its replica hears of others' edits.
|
|
258
|
+
`journalExchange(options)` is the same exchange as a `TransportClient`, for
|
|
259
|
+
`Sync.transport.fromPromise` in process; `journalChanges` the notice alone.
|
|
260
|
+
- **What cannot be taken is rejected by its id, not left to fail the
|
|
261
|
+
exchange.** An operation whose message does not decode, that names another
|
|
262
|
+
document, that `authorize` or `validate` refuses, or that reuses an id for
|
|
263
|
+
other content fails the same way on every retry, so the replica drops it
|
|
264
|
+
and the edits behind it still go. Only an entry with no id, a cursor ahead
|
|
265
|
+
of the journal's, a connection `opens` refuses, or `admit` or the journal
|
|
266
|
+
failing fails the exchange.
|
|
267
|
+
- **`admit(operation)`** decides before the journal reads it, by what the
|
|
268
|
+
transport established (a principal that may only read) or by what cannot
|
|
269
|
+
run under the journal's lock (a rule that reads the rows an edit names,
|
|
270
|
+
`foldkit-guard/sync`): `true` admits, a refusal is rejected by id with its
|
|
271
|
+
reason, and it may answer with an `Effect`. A rule about the document
|
|
272
|
+
belongs in `authorize`.
|
|
273
|
+
- **`opens`**, an `Effect` of whether the connection may open the document at
|
|
274
|
+
all: refused, every exchange fails and nothing is read or appended, and
|
|
275
|
+
`serveJournal` sends it no notice of a commit. It is asked each exchange, so
|
|
276
|
+
access taken away holds from the next one.
|
|
277
|
+
`defineDocumentHost` takes both as functions of the principal.
|
|
278
|
+
- **`settle`** runs after each exchange's appends: apply what committed to
|
|
279
|
+
another store, through `journal.recover`. It runs every exchange, so a
|
|
280
|
+
write that failed is tried again.
|
|
281
|
+
- The reply carries what committed since the cursor, a page at a time
|
|
282
|
+
(`limit`, `more`), a checkpoint when that history was compacted, and the
|
|
283
|
+
journal's epoch: a replica of a reset journal is answered from the start.
|
|
284
|
+
|
|
285
|
+
[The replicated state guide](../../docs/replication.md#3-the-server-an-exchange-over-the-journal)
|
|
286
|
+
says why each rule is there, and [`examples/sync`](../../examples/sync) writes
|
|
287
|
+
the exchange by hand, with an effect policy per committed operation.
|
|
246
288
|
|
|
247
289
|
A server that is not Durable implements the same [exchange](#the-exchange).
|
|
248
290
|
|
|
291
|
+
## Serving from a Durable Object
|
|
292
|
+
|
|
293
|
+
On Cloudflare, one object per document is the single writer the journal
|
|
294
|
+
wants. `defineDocumentHost` from `foldkit-sync/do` defines that class: each
|
|
295
|
+
accepted socket exchanges against the same journal through `serveJournal`,
|
|
296
|
+
and the replica side is the socket transport it already speaks:
|
|
297
|
+
|
|
298
|
+
```ts
|
|
299
|
+
import { DurableObject } from 'cloudflare:workers'
|
|
300
|
+
import { defineDocumentHost } from 'foldkit-sync/do'
|
|
301
|
+
|
|
302
|
+
export class SyncHost extends defineDocumentHost(DurableObject, {
|
|
303
|
+
sync: TodoSync,
|
|
304
|
+
// A D1 journal as the durable D1 guide opens one.
|
|
305
|
+
openJournal: (_ctx, env) => openTodosJournal(env.DB),
|
|
306
|
+
resolvePrincipal: request => request.headers.get('x-actor') ?? 'anon',
|
|
307
|
+
}) {}
|
|
308
|
+
```
|
|
309
|
+
|
|
310
|
+
```ts
|
|
311
|
+
// The worker: one object per document.
|
|
312
|
+
export default {
|
|
313
|
+
fetch: (request: Request, env: Env): Promise<Response> =>
|
|
314
|
+
env.SYNC_HOST.get(env.SYNC_HOST.idFromName('todos')).fetch(request),
|
|
315
|
+
}
|
|
316
|
+
```
|
|
317
|
+
|
|
318
|
+
```ts
|
|
319
|
+
// A replica: the reconnecting client it already speaks.
|
|
320
|
+
Sync.transport.socket({ url: 'wss://example.com/sync' })
|
|
321
|
+
```
|
|
322
|
+
|
|
323
|
+
- The base class is a parameter, so the module imports no worker runtime.
|
|
324
|
+
`openJournal` runs once, on the first upgrade: over D1 (`d1: true`, see
|
|
325
|
+
[the durable D1 guide](../durable/README.md#on-cloudflare-d1-d1-true)), or
|
|
326
|
+
the object's SQLite storage through `ctx`.
|
|
327
|
+
- Each upgrade resolves its own principal; non-upgrades are 404s that open
|
|
328
|
+
nothing. A journal that never opened is a 500 that says nothing of it.
|
|
329
|
+
- `settle: (env, journal) => Effect` runs after each exchange's appends, before
|
|
330
|
+
the reply is read, to apply what committed elsewhere — the Remote tables a
|
|
331
|
+
reader then queries. `journal.recover` makes it crash-safe and idempotent;
|
|
332
|
+
`env` and the journal are handed over because the store they reach lives
|
|
333
|
+
outside the exchange.
|
|
334
|
+
- A server socket is not a live stream: another request cannot wake the fiber
|
|
335
|
+
of an open SSE response in a worker, not even from the same isolate. What
|
|
336
|
+
crosses requests there is the store they share (D1 here), and the live side
|
|
337
|
+
re-reads it from inside its own request. See
|
|
338
|
+
[the Cloudflare example](../../examples/cloudflare).
|
|
339
|
+
- `Sync.transport.workerSocket` wraps the accepted end of a `WebSocketPair`
|
|
340
|
+
for `serveJournal`, anywhere else a platform hands one over.
|
|
341
|
+
|
|
249
342
|
## What makes a Message durable
|
|
250
343
|
|
|
251
344
|
Replay runs a durable Message through `update` on another machine, later,
|
|
@@ -296,20 +389,28 @@ error names why: `ReplayError` for a Message that cannot be durable,
|
|
|
296
389
|
|
|
297
390
|
```ts
|
|
298
391
|
const status = yield* replica.status
|
|
299
|
-
// { pending: number, cursor: Sequence, lastError: string | undefined, rejected: ReadonlyArray<
|
|
392
|
+
// { pending: number, cursor: Sequence, lastError: string | undefined, rejected: ReadonlyArray<Rejection> }
|
|
300
393
|
```
|
|
301
394
|
|
|
302
395
|
`pending` above zero means edits the server has not confirmed; `lastError` is
|
|
303
396
|
the last failed exchange, cleared by the next success; `rejected` lists recent
|
|
304
|
-
edits the server refused
|
|
305
|
-
|
|
306
|
-
|
|
397
|
+
edits the server refused, most recent first, each `{ opId, reason, operation }`,
|
|
398
|
+
where `reason` is an `Option` of the server's words for the person and
|
|
399
|
+
`operation` is the refused operation itself, which the replica has dropped
|
|
400
|
+
from the outbox, so a page reads what it changed from here. `replica.statusChanges` is a Stream of the same,
|
|
401
|
+
emitted now and after every submit and exchange (subscribed before its first
|
|
402
|
+
read, so a subscriber slow to take one misses no change); `replica.changes`
|
|
403
|
+
pairs it with the optimistic `shared` value so one subscription sees both.
|
|
307
404
|
|
|
308
405
|
**A refused edit** arrives as a rejection on the next exchange, not as an
|
|
309
406
|
error. The replica drops it and replays the rest; the mount re-installs the
|
|
310
407
|
shared slice, and the field the user edited reverts. `onPersistenceFailure`
|
|
311
408
|
does not run for this, since the local save succeeded. Watch `rejected` to
|
|
312
|
-
explain the revert
|
|
409
|
+
explain the revert: `journalExchange` gives each rejection a reason, the
|
|
410
|
+
`authorize` rule's own when it gave one, and otherwise a fixed sentence
|
|
411
|
+
("Not a valid operation"). It never sends an error's message, which may carry
|
|
412
|
+
internals, so a rule a person should read about belongs in `authorize` with a
|
|
413
|
+
`reason`, not in `validate`.
|
|
313
414
|
|
|
314
415
|
**Waiting for the server.** The Model shows an edit before the server has it.
|
|
315
416
|
Something that must not report success until the edit is committed, an agent
|
|
@@ -375,6 +476,9 @@ const Board = Authorized.make({
|
|
|
375
476
|
})
|
|
376
477
|
```
|
|
377
478
|
|
|
479
|
+
A rule returns `true`, `false`, or `{ allowed: false, reason }`; the reason
|
|
480
|
+
is sent to the replica whose edit it was, as its rejection's `reason`.
|
|
481
|
+
|
|
378
482
|
`message` is exactly that variant, `shared` is the authoritative snapshot, and
|
|
379
483
|
`principal` is whatever the server's transport established for the connection;
|
|
380
484
|
it is never read from the operation. A variant without a rule is allowed. A key
|
|
@@ -384,6 +488,42 @@ that is not a durable variant is a type error. The rules ride in
|
|
|
384
488
|
An edit the policy refuses still shows at once on the client, then comes back
|
|
385
489
|
as a rejection, as [above](#what-to-show-the-user).
|
|
386
490
|
|
|
491
|
+
### Stamp what only the commit decides
|
|
492
|
+
|
|
493
|
+
A fact carries its own nondeterminism, but a client cannot know where its
|
|
494
|
+
operation lands in the server's order. A stamp writes it in, on the server, as
|
|
495
|
+
the journal commits:
|
|
496
|
+
|
|
497
|
+
```ts
|
|
498
|
+
const Message = defineMessageUnion({
|
|
499
|
+
// `at` is absent on what a client sends: the server's sequence once committed.
|
|
500
|
+
PriceEdited: { id: Schema.String, cents: Schema.Number, at: Schema.optionalKey(Schema.Number) },
|
|
501
|
+
})
|
|
502
|
+
|
|
503
|
+
const Prices = Sync.forApplication(App).make({
|
|
504
|
+
documentId: DocumentId.make('prices'),
|
|
505
|
+
shared: Projection.pick(App.model.prices),
|
|
506
|
+
durable: MessageSet.make(App, [Message.PriceEdited]),
|
|
507
|
+
stamp: {
|
|
508
|
+
PriceEdited: ({ id, cents }, { sequence }) => Message.PriceEdited({ id, cents, at: sequence }),
|
|
509
|
+
},
|
|
510
|
+
})
|
|
511
|
+
```
|
|
512
|
+
|
|
513
|
+
The rule gets that variant and `{ sequence, actorId, replicaId }` (the
|
|
514
|
+
replica that sent it, which tells one person's tabs apart), and returns the same
|
|
515
|
+
variant; a key that is not durable is a type error, and a rule that returns
|
|
516
|
+
another variant has its operation rejected. It rides in `journalContract()` as
|
|
517
|
+
Durable's `stamp`, so the journal stores, reduces and sends the stamped
|
|
518
|
+
Message. A replica shows what it sent until the exchange, then replays the
|
|
519
|
+
stamped Message in its place, the same on every replica. A retry is still
|
|
520
|
+
recognized by what was sent.
|
|
521
|
+
|
|
522
|
+
Use it for what a reader compares with the server's order: a read model that
|
|
523
|
+
records the last sequence it applied can tell which edits it holds by comparing
|
|
524
|
+
the two. A client that sends the stamped field itself is claiming a commit it
|
|
525
|
+
did not get; refuse it in the journal's `validate`.
|
|
526
|
+
|
|
387
527
|
### Coalesce a burst of typing
|
|
388
528
|
|
|
389
529
|
Typing makes one durable Message per keystroke. `coalesce(last, next)` is
|
|
@@ -417,10 +557,68 @@ See [`examples/todo-app`](../../examples/todo-app) for both.
|
|
|
417
557
|
### Carry local state across a reinstall
|
|
418
558
|
|
|
419
559
|
When an exchange or a failed persist replaces the shared slice outside
|
|
420
|
-
`update`, `onReinstall(next, previous)` returns the transition
|
|
421
|
-
to keep a selection that points into the slice, or to return a
|
|
422
|
-
patches a DOM the change has to reach. Omitted, the Model is
|
|
423
|
-
is not called for a status that changed nothing the Model
|
|
560
|
+
`update`, `onReinstall(next, previous, { reset })` returns the transition
|
|
561
|
+
instead. Use it to keep a selection that points into the slice, or to return a
|
|
562
|
+
Command that patches a DOM the change has to reach. Omitted, the Model is
|
|
563
|
+
simply `next`. It is not called for a status that changed nothing the Model
|
|
564
|
+
shows. `reset` is true when the server's history is a new one (a new
|
|
565
|
+
`status.epoch`: the server was reset), so whatever the application derived
|
|
566
|
+
from the old one, such as reads keyed by commit sequence, is to be read again.
|
|
567
|
+
|
|
568
|
+
`afterUpdate(model)` runs after every update of the mounted application, a
|
|
569
|
+
durable Message's and a reinstall's included, and never in replay. A durable
|
|
570
|
+
Message is replayed through `update` on the server and in the replica, where
|
|
571
|
+
it may change only the shared slice; local state derived from the slice (the
|
|
572
|
+
Remote overlays below) is kept in step here instead.
|
|
573
|
+
|
|
574
|
+
### Edit rows a server owns
|
|
575
|
+
|
|
576
|
+
When rows are too many to replicate but edits to them must survive offline,
|
|
577
|
+
let the journal own the edits and the table be its read model, each row
|
|
578
|
+
carrying the highest sequence it has applied (`revision`). `foldkit-sync/entity`
|
|
579
|
+
keeps such edits one per cell, and `foldkit-sync/remote` shows them over the
|
|
580
|
+
rows Remote reads:
|
|
581
|
+
|
|
582
|
+
```text
|
|
583
|
+
cell shown = Remote's row, with each edit over it the row has not reached:
|
|
584
|
+
one pending, one committed after the row's revision, or one the
|
|
585
|
+
journal absorbed while the cached row is still below it
|
|
586
|
+
```
|
|
587
|
+
|
|
588
|
+
`EditableEntity.make(Product, { members: ['description', 'cents'] })` gives
|
|
589
|
+
the schemas (`Change` on the wire, `Edit` in the slice, the Message fields
|
|
590
|
+
`edited` and `absorbed`) and pure functions for `update`, the stamp
|
|
591
|
+
(`stamped`) and `onReinstall`: `merge`, `absorb`, `replaced` (a cell of this
|
|
592
|
+
tab's that another's later commit took), `cellsOf` (what a refusal undid) and
|
|
593
|
+
`changeAt` (what a row holds in a member, as the change that sets it: an
|
|
594
|
+
undo's value). It owns no state and does no I/O.
|
|
595
|
+
|
|
596
|
+
`RemoteEdits.make(Data, ProductEdits)` is the bridge to a Remote domain over
|
|
597
|
+
the same Entity. Its `reconcile(model, edits, replica)` brings Remote's
|
|
598
|
+
overlays (`Data.overlay`) in line with the slice, so every read of a row draws
|
|
599
|
+
its edits, a second view or a filter as well as the grid. It holds an edit the
|
|
600
|
+
journal absorbed while the cached row is below it, and lifts it once a read
|
|
601
|
+
reaches it; it returns those newly `held` (ask for their rows again) and this
|
|
602
|
+
replica's edits a read reached with another value (`replaced`: the table
|
|
603
|
+
applies in commit order, so a later edit took the cell). An edit to a field a
|
|
604
|
+
list orders or filters by moves its row there at once, as Remote places any
|
|
605
|
+
pending edit. To see that when the
|
|
606
|
+
row arrives live after its own edit landed, it keeps this replica's landed
|
|
607
|
+
edit, showing the row's own value, until the slice drops it. Call it from
|
|
608
|
+
`afterUpdate`. `shown(model)` lists what the overlays show, for a cell's
|
|
609
|
+
mark, and `clear` lifts them all, for a reset. Remote stores none of it: after
|
|
610
|
+
a reload, `reconcile` rebuilds the overlays from the slice the replica
|
|
611
|
+
restored. On the server, `editsJournal` from `foldkit-sync/journal` keeps
|
|
612
|
+
the table: its `settle` (for `journalExchange`) applies each committed change
|
|
613
|
+
through your `apply` as a recovery intent, from the journal's floor and keyed
|
|
614
|
+
by its epoch, and its `absorb` (on a clock) records what the table holds and
|
|
615
|
+
compacts behind it. It refuses to settle a table whose `tableRevision` is past
|
|
616
|
+
the journal's cursor (`TableAheadOfJournalError`: the table outlived a reset of
|
|
617
|
+
the journal), and an edit to a row the table lacks is the journal's `validate`
|
|
618
|
+
to refuse. [Editing server data through a
|
|
619
|
+
journal](../../docs/editing-server-data.md) explains the pattern, its failure
|
|
620
|
+
cases and its limits; [`examples/registry`](../../examples/registry) is the
|
|
621
|
+
whole of it.
|
|
424
622
|
|
|
425
623
|
## Testing without a browser or a server
|
|
426
624
|
|
|
@@ -540,6 +738,7 @@ request cursor
|
|
|
540
738
|
answer operations committed after the cursor, contiguous
|
|
541
739
|
acknowledged ids the server accepted without repeating them
|
|
542
740
|
rejected ids the server refused
|
|
741
|
+
reasons? [{ opId, reason }]: why, for some of rejected; at most 500 characters each
|
|
543
742
|
checkpoint? { cursor, model } when the history is gone
|
|
544
743
|
more? true when another page follows
|
|
545
744
|
epoch? the server's history
|
|
@@ -559,10 +758,18 @@ replica validates every field; see [when things go wrong](#when-things-go-wrong)
|
|
|
559
758
|
`maxRetries` (5) consecutive failed connections fails queued work and fails
|
|
560
759
|
new exchanges fast until a socket is healthy again. `maxQueue` (64) bounds
|
|
561
760
|
the queue.
|
|
761
|
+
Instead of a `url`, pass `makeSocket: () => SocketLike` to connect over
|
|
762
|
+
something else; it is called again on each reconnect.
|
|
562
763
|
- `Sync.transport.serve(socket, { exchange, changes? })`: the server side of one
|
|
563
764
|
accepted socket. With `changes`, a subscription to the document's commits, it
|
|
564
765
|
sends a notice after each; the client's loop exchanges on every notice, so a
|
|
565
766
|
tab that is only reading still sees others' edits without polling.
|
|
767
|
+
- `Sync.transport.portSocket(port, { signal? })`: a socket over a
|
|
768
|
+
`MessagePort`, for a server in a worker or in the page. A port tells
|
|
769
|
+
neither end when the other goes, so its `onClose` fires on `close()` or when
|
|
770
|
+
`signal` aborts: a host aborts it when the pane it serves is gone, and
|
|
771
|
+
`serve` stops. A client opens one per connection:
|
|
772
|
+
`Sync.transport.socket({ makeSocket: () => Sync.transport.portSocket(open()) })`.
|
|
566
773
|
- `Sync.transport.loopback(handler)`, `fromPromise(client)`, `toPromise(shape)`,
|
|
567
774
|
and `nativeSocket` for the platform `WebSocket`.
|
|
568
775
|
|
|
@@ -576,7 +783,11 @@ throttle? })` is a TTL'd peer registry outside the log: values are decoded befor
|
|
|
576
783
|
they enter it, a peer that stops refreshing is pruned, and `throttle` sends at
|
|
577
784
|
most one value per interval, the latest. `Sync.presence.socketChannel(transport.socket)`
|
|
578
785
|
shares the sync socket across reconnects; `Sync.presence.hub` and `.serve` are
|
|
579
|
-
the server side.
|
|
786
|
+
the server side. `.serve` stamps each frame with the connection's `peerId`, so a
|
|
787
|
+
peer cannot speak as another, and with `opens` (the exchange's own) joins only
|
|
788
|
+
a connection that may open the document, relaying nothing to or from one
|
|
789
|
+
refused. It asks again with each frame the connection sends, and one no
|
|
790
|
+
longer let open the document leaves. [`examples/pages`](../../examples/pages) shows remote carets.
|
|
580
791
|
|
|
581
792
|
### Last-writer-wins fields (experimental)
|
|
582
793
|
|
|
@@ -621,14 +832,15 @@ peer-to-peer merging, or migration of your own Messages and snapshots.
|
|
|
621
832
|
implement.
|
|
622
833
|
- One server order per document; no CRDT and no collaborative text algorithm.
|
|
623
834
|
- Binding the socket transport to a platform WebSocket server is glue the
|
|
624
|
-
examples show with `ws
|
|
835
|
+
examples show with `ws`; on workers, `foldkit-sync/do` defines the object
|
|
836
|
+
and `Sync.transport.workerSocket` wraps its accepted pair.
|
|
625
837
|
|
|
626
838
|
## Older spellings
|
|
627
839
|
|
|
628
840
|
`Sync` groups the exports and wraps nothing; every name is also exported on its
|
|
629
841
|
own with the same signature: `defineSync` is `Sync.define`, `syncMetrics` is
|
|
630
842
|
`Sync.metrics`, `layerSocket`/`layerLoopback`/`layerFromPromise`/`toPromise`/
|
|
631
|
-
`serveSocket`/`nativeSocket` are `Sync.transport.*`, `createPresence`/
|
|
843
|
+
`serveSocket`/`nativeSocket`/`workerSocket` are `Sync.transport.*`, `createPresence`/
|
|
632
844
|
`createPresenceHub`/`servePresence`/`socketPresenceChannel`/
|
|
633
845
|
`loopbackPresenceChannel` are `Sync.presence.*`, and `lwwRegister`/`openLwwClock`
|
|
634
846
|
are `Sync.lww.*`. `documentId('todos')` and `DocumentId.make('todos')` make the
|
package/dist/do.d.mts
ADDED
|
@@ -0,0 +1,60 @@
|
|
|
1
|
+
import { n as AuthorizeDecision, v as Operation, w as Sync } from "./make-h5yD8STj.mjs";
|
|
2
|
+
import { Effect } from "effect";
|
|
3
|
+
import { Journal } from "foldkit-durable/core";
|
|
4
|
+
//#region src/do.d.ts
|
|
5
|
+
/** The sliver of a Durable Object the host uses: state for the journal to open with. */
|
|
6
|
+
export interface DocumentHostContext {
|
|
7
|
+
readonly storage?: unknown;
|
|
8
|
+
}
|
|
9
|
+
/** What `defineDocumentHost` needs of the application, per document. */
|
|
10
|
+
export interface DocumentHostConfig<Message, Shared, Principal, Env> {
|
|
11
|
+
/** The document's contract: its codecs and journal shape. */
|
|
12
|
+
readonly sync: Sync<Message, Shared>;
|
|
13
|
+
/**
|
|
14
|
+
* Opens the document's journal, once: on the first upgrade, not on
|
|
15
|
+
* construction, so probes never provision storage. Over D1 (`d1: true`),
|
|
16
|
+
* or the object's SQLite storage through `ctx`. May build with Effect and
|
|
17
|
+
* unwrap; a rejection answers the upgrade with a 500, and the next upgrade
|
|
18
|
+
* tries again.
|
|
19
|
+
*/
|
|
20
|
+
readonly openJournal: (ctx: DocumentHostContext, env: Env) => Promise<Journal<Operation, Shared, Principal, Operation>>;
|
|
21
|
+
/** Who is opening this socket, from the upgrade request. May throw: a 401. */
|
|
22
|
+
readonly resolvePrincipal: (request: Request, env: Env) => Principal | Promise<Principal>;
|
|
23
|
+
/**
|
|
24
|
+
* Runs after each exchange's appends and before the reply is read: apply
|
|
25
|
+
* what committed to another store, through `journal.recover`. Takes the
|
|
26
|
+
* environment — a D1 binding, say — and the journal, because both live
|
|
27
|
+
* outside the exchange. See `serveJournal`.
|
|
28
|
+
*/
|
|
29
|
+
readonly settle?: ((env: Env, journal: Journal<Operation, Shared, Principal, Operation>) => Effect.Effect<void, unknown>) | undefined;
|
|
30
|
+
/** Whether `principal` may open the document. See `journalExchange`'s `opens`. */
|
|
31
|
+
readonly opens?: ((principal: Principal) => Effect.Effect<boolean>) | undefined;
|
|
32
|
+
/**
|
|
33
|
+
* Whether an operation from `principal` may be appended, before the journal
|
|
34
|
+
* sees it. See `journalExchange`'s `admit`.
|
|
35
|
+
*/
|
|
36
|
+
readonly admit?: ((principal: Principal, operation: Operation) => AuthorizeDecision | Effect.Effect<AuthorizeDecision, unknown>) | undefined;
|
|
37
|
+
/**
|
|
38
|
+
* Makes the socket pair for an upgrade: the server end is accepted before
|
|
39
|
+
* serving. Defaults to the `WebSocketPair` global where one exists (a
|
|
40
|
+
* worker, miniflare); pass one in tests.
|
|
41
|
+
*/
|
|
42
|
+
readonly pair?: (() => readonly [client: WebSocket, server: AcceptingWebSocket]) | undefined;
|
|
43
|
+
}
|
|
44
|
+
/**
|
|
45
|
+
* A server `WebSocket`: accepted once it serves. The `accept` member is a
|
|
46
|
+
* worker-runtime extension the DOM type does not declare.
|
|
47
|
+
*/
|
|
48
|
+
export interface AcceptingWebSocket extends WebSocket {
|
|
49
|
+
accept(): void;
|
|
50
|
+
}
|
|
51
|
+
/**
|
|
52
|
+
* Defines the Durable Object class serving one document's Sync exchange.
|
|
53
|
+
* Non-upgrades are 404s that open nothing; a throwing principal resolution
|
|
54
|
+
* is a 401; a journal that never opened is a 500 that says nothing of it.
|
|
55
|
+
*/
|
|
56
|
+
export declare const defineDocumentHost: <Message, Shared, Principal, Env>(Base: new (ctx: DocumentHostContext, env: Env) => object, config: DocumentHostConfig<Message, Shared, Principal, Env>) => new (ctx: DocumentHostContext, env: Env) => {
|
|
57
|
+
fetch(request: Request): Promise<Response>;
|
|
58
|
+
};
|
|
59
|
+
//#endregion
|
|
60
|
+
//# sourceMappingURL=do.d.mts.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"do.d.mts","names":[],"sources":["../src/do.ts"],"mappings":";;;;;iBAiCiB;WACN;;;iBAIM,mBAAmB,SAAS,QAAQ,WAAW;;WAErD,MAAM,KAAK,SAAS;;;;;;;;WAQpB,cACP,KAAK,qBACL,KAAK,QACF,QAAQ,QAAQ,WAAW,QAAQ,WAAW;;WAE1C,mBAAmB,SAAS,SAAS,KAAK,QAAQ,YAAY,QAAQ;;;;;;;WAOtE,WAEH,KAAK,KACL,SAAS,QAAQ,WAAW,QAAQ,WAAW,eAC5C,OAAO;;WAGP,UAAU,WAAW,cAAc,OAAO;;;;;WAK1C,UAEH,WAAW,WACX,WAAW,cACR,oBAAoB,OAAO,OAAO;;;;;;WAOlC,wBAAwB,QAAQ,WAAW,QAAQ;;;;;;iBAO7C,2BAA2B;EAC1C;;;;;;;qBAqBW,qBAAsB,SAAS,QAAQ,WAAW,KAC7D,WAAW,KAAK,qBAAqB,KAAK,gBAC1C,QAAQ,mBAAmB,SAAS,QAAQ,WAAW,cACjD,KAAK,qBAAqB,KAAK;EAAU,MAAM,SAAS,UAAU,QAAQ"}
|
package/dist/do.mjs
ADDED
|
@@ -0,0 +1,59 @@
|
|
|
1
|
+
import { u as workerSocket } from "./transport-DiKXy5zx.mjs";
|
|
2
|
+
import { serveJournal } from "./journal.mjs";
|
|
3
|
+
//#region src/do.ts
|
|
4
|
+
const defaultPair = () => {
|
|
5
|
+
const maker = globalThis.WebSocketPair;
|
|
6
|
+
if (maker === void 0) throw new Error("DocumentHost: no WebSocketPair global; pass `pair` explicitly");
|
|
7
|
+
const pair = new maker();
|
|
8
|
+
return [pair[0], pair[1]];
|
|
9
|
+
};
|
|
10
|
+
/**
|
|
11
|
+
* Defines the Durable Object class serving one document's Sync exchange.
|
|
12
|
+
* Non-upgrades are 404s that open nothing; a throwing principal resolution
|
|
13
|
+
* is a 401; a journal that never opened is a 500 that says nothing of it.
|
|
14
|
+
*/
|
|
15
|
+
const defineDocumentHost = (Base, config) => class extends Base {
|
|
16
|
+
hostContext;
|
|
17
|
+
hostEnv;
|
|
18
|
+
ready;
|
|
19
|
+
constructor(ctx, env) {
|
|
20
|
+
super(ctx, env);
|
|
21
|
+
this.hostContext = ctx;
|
|
22
|
+
this.hostEnv = env;
|
|
23
|
+
}
|
|
24
|
+
async fetch(request) {
|
|
25
|
+
if (request.headers.get("Upgrade")?.toLowerCase() !== "websocket") return Response.json({ error: "not found" }, { status: 404 });
|
|
26
|
+
let principal;
|
|
27
|
+
try {
|
|
28
|
+
principal = await config.resolvePrincipal(request, this.hostEnv);
|
|
29
|
+
} catch (error) {
|
|
30
|
+
return Response.json({ error: error instanceof Error ? error.message : String(error) }, { status: 401 });
|
|
31
|
+
}
|
|
32
|
+
let journal;
|
|
33
|
+
try {
|
|
34
|
+
journal = await (this.ready ??= config.openJournal(this.hostContext, this.hostEnv));
|
|
35
|
+
} catch {
|
|
36
|
+
this.ready = void 0;
|
|
37
|
+
return Response.json({ error: "Internal error" }, { status: 500 });
|
|
38
|
+
}
|
|
39
|
+
const [client, server] = (config.pair ?? defaultPair)();
|
|
40
|
+
server.accept();
|
|
41
|
+
const { admit } = config;
|
|
42
|
+
serveJournal(workerSocket(server), {
|
|
43
|
+
sync: config.sync,
|
|
44
|
+
journal,
|
|
45
|
+
principal,
|
|
46
|
+
...config.settle === void 0 ? {} : { settle: config.settle(this.hostEnv, journal) },
|
|
47
|
+
...config.opens === void 0 ? {} : { opens: config.opens(principal) },
|
|
48
|
+
...admit === void 0 ? {} : { admit: (operation) => admit(principal, operation) }
|
|
49
|
+
});
|
|
50
|
+
return new Response(null, {
|
|
51
|
+
status: 101,
|
|
52
|
+
webSocket: client
|
|
53
|
+
});
|
|
54
|
+
}
|
|
55
|
+
};
|
|
56
|
+
//#endregion
|
|
57
|
+
export { defineDocumentHost };
|
|
58
|
+
|
|
59
|
+
//# sourceMappingURL=do.mjs.map
|
package/dist/do.mjs.map
ADDED
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"do.mjs","names":[],"sources":["../src/do.ts"],"sourcesContent":["/**\n * `foldkit-sync/do`: serve one document's Sync exchange from a Cloudflare\n * Durable Object, the single writer its journal wants. Each socket the\n * object accepts exchanges against the same journal through `serveJournal`;\n * the replica side already exists as `Sync.transport.socket` over the\n * object's `wss://…` URL, which hears other writers' commits as notices.\n *\n * The class takes its `DurableObject` base as a parameter, so this module\n * imports no worker runtime and loads anywhere:\n *\n * ```ts\n * import { DurableObject } from 'cloudflare:workers'\n *\n * export class SyncHost extends defineDocumentHost(DurableObject, {\n * sync: Todos,\n * openJournal: (_ctx, env) => openTodosJournal(env.DB),\n * resolvePrincipal: request => ({ actorId: request.headers.get('x-actor') ?? 'anon' }),\n * }) {}\n * ```\n *\n * Open the journal once per object: on the first upgrade, so probes never\n * provision storage. Route one object per document (`idFromName(sync\n * .documentId)`); its in-memory cache and change stream are then exact\n * because nothing else writes.\n */\nimport type { Effect } from 'effect'\nimport type { Journal } from 'foldkit-durable/core'\nimport { serveJournal } from './journal.js'\nimport type { AuthorizeDecision } from './make.js'\nimport type { Operation, Sync } from './sync.js'\nimport { workerSocket } from './transport.js'\n\n/** The sliver of a Durable Object the host uses: state for the journal to open with. */\nexport interface DocumentHostContext {\n readonly storage?: unknown\n}\n\n/** What `defineDocumentHost` needs of the application, per document. */\nexport interface DocumentHostConfig<Message, Shared, Principal, Env> {\n /** The document's contract: its codecs and journal shape. */\n readonly sync: Sync<Message, Shared>\n /**\n * Opens the document's journal, once: on the first upgrade, not on\n * construction, so probes never provision storage. Over D1 (`d1: true`),\n * or the object's SQLite storage through `ctx`. May build with Effect and\n * unwrap; a rejection answers the upgrade with a 500, and the next upgrade\n * tries again.\n */\n readonly openJournal: (\n ctx: DocumentHostContext,\n env: Env,\n ) => Promise<Journal<Operation, Shared, Principal, Operation>>\n /** Who is opening this socket, from the upgrade request. May throw: a 401. */\n readonly resolvePrincipal: (request: Request, env: Env) => Principal | Promise<Principal>\n /**\n * Runs after each exchange's appends and before the reply is read: apply\n * what committed to another store, through `journal.recover`. Takes the\n * environment — a D1 binding, say — and the journal, because both live\n * outside the exchange. See `serveJournal`.\n */\n readonly settle?:\n | ((\n env: Env,\n journal: Journal<Operation, Shared, Principal, Operation>,\n ) => Effect.Effect<void, unknown>)\n | undefined\n /** Whether `principal` may open the document. See `journalExchange`'s `opens`. */\n readonly opens?: ((principal: Principal) => Effect.Effect<boolean>) | undefined\n /**\n * Whether an operation from `principal` may be appended, before the journal\n * sees it. See `journalExchange`'s `admit`.\n */\n readonly admit?:\n | ((\n principal: Principal,\n operation: Operation,\n ) => AuthorizeDecision | Effect.Effect<AuthorizeDecision, unknown>)\n | undefined\n /**\n * Makes the socket pair for an upgrade: the server end is accepted before\n * serving. Defaults to the `WebSocketPair` global where one exists (a\n * worker, miniflare); pass one in tests.\n */\n readonly pair?: (() => readonly [client: WebSocket, server: AcceptingWebSocket]) | undefined\n}\n\n/**\n * A server `WebSocket`: accepted once it serves. The `accept` member is a\n * worker-runtime extension the DOM type does not declare.\n */\nexport interface AcceptingWebSocket extends WebSocket {\n accept(): void\n}\n\nconst defaultPair = (): readonly [client: WebSocket, server: AcceptingWebSocket] => {\n const maker = (\n globalThis as {\n readonly WebSocketPair?:\n (new () => { readonly 0: WebSocket; readonly 1: AcceptingWebSocket }) | undefined\n }\n ).WebSocketPair\n if (maker === undefined)\n throw new Error('DocumentHost: no WebSocketPair global; pass `pair` explicitly')\n const pair = new maker()\n return [pair[0], pair[1]] as const\n}\n\n/**\n * Defines the Durable Object class serving one document's Sync exchange.\n * Non-upgrades are 404s that open nothing; a throwing principal resolution\n * is a 401; a journal that never opened is a 500 that says nothing of it.\n */\nexport const defineDocumentHost = <Message, Shared, Principal, Env>(\n Base: new (ctx: DocumentHostContext, env: Env) => object,\n config: DocumentHostConfig<Message, Shared, Principal, Env>,\n): new (ctx: DocumentHostContext, env: Env) => { fetch(request: Request): Promise<Response> } =>\n class extends Base {\n private readonly hostContext: DocumentHostContext\n private readonly hostEnv: Env\n private ready: Promise<Journal<Operation, Shared, Principal, Operation>> | undefined\n\n constructor(ctx: DocumentHostContext, env: Env) {\n super(ctx, env)\n this.hostContext = ctx\n this.hostEnv = env\n }\n\n async fetch(request: Request): Promise<Response> {\n if (request.headers.get('Upgrade')?.toLowerCase() !== 'websocket')\n return Response.json({ error: 'not found' }, { status: 404 })\n let principal: Principal\n try {\n principal = await config.resolvePrincipal(request, this.hostEnv)\n } catch (error) {\n return Response.json(\n { error: error instanceof Error ? error.message : String(error) },\n { status: 401 },\n )\n }\n let journal: Journal<Operation, Shared, Principal, Operation>\n try {\n journal = await (this.ready ??= config.openJournal(this.hostContext, this.hostEnv))\n } catch {\n // A failed open provisions nothing: the next upgrade tries again.\n this.ready = undefined\n return Response.json({ error: 'Internal error' }, { status: 500 })\n }\n const [client, server] = (config.pair ?? defaultPair)()\n server.accept()\n const { admit } = config\n serveJournal(workerSocket(server), {\n sync: config.sync,\n journal,\n principal,\n ...(config.settle === undefined ? {} : { settle: config.settle(this.hostEnv, journal) }),\n ...(config.opens === undefined ? {} : { opens: config.opens(principal) }),\n ...(admit === undefined\n ? {}\n : { admit: (operation: Operation) => admit(principal, operation) }),\n })\n return new Response(null, {\n status: 101,\n // The worker runtime reads the client out of the upgrade response;\n // the DOM's `ResponseInit` has no such member.\n webSocket: client,\n } as ResponseInit)\n }\n }\n"],"mappings":";;;AA8FA,MAAM,oBAA8E;CAClF,MAAM,QACJ,WAIA;CACF,IAAI,UAAU,KAAA,GACZ,MAAM,IAAI,MAAM,+DAA+D;CACjF,MAAM,OAAO,IAAI,MAAM;CACvB,OAAO,CAAC,KAAK,IAAI,KAAK,EAAE;AAC1B;;;;;;AAOA,MAAa,sBACX,MACA,WAEA,cAAc,KAAK;CACjB;CACA;CACA;CAEA,YAAY,KAA0B,KAAU;EAC9C,MAAM,KAAK,GAAG;EACd,KAAK,cAAc;EACnB,KAAK,UAAU;CACjB;CAEA,MAAM,MAAM,SAAqC;EAC/C,IAAI,QAAQ,QAAQ,IAAI,SAAS,CAAC,EAAE,YAAY,MAAM,aACpD,OAAO,SAAS,KAAK,EAAE,OAAO,YAAY,GAAG,EAAE,QAAQ,IAAI,CAAC;EAC9D,IAAI;EACJ,IAAI;GACF,YAAY,MAAM,OAAO,iBAAiB,SAAS,KAAK,OAAO;EACjE,SAAS,OAAO;GACd,OAAO,SAAS,KACd,EAAE,OAAO,iBAAiB,QAAQ,MAAM,UAAU,OAAO,KAAK,EAAE,GAChE,EAAE,QAAQ,IAAI,CAChB;EACF;EACA,IAAI;EACJ,IAAI;GACF,UAAU,OAAO,KAAK,UAAU,OAAO,YAAY,KAAK,aAAa,KAAK,OAAO;EACnF,QAAQ;GAEN,KAAK,QAAQ,KAAA;GACb,OAAO,SAAS,KAAK,EAAE,OAAO,iBAAiB,GAAG,EAAE,QAAQ,IAAI,CAAC;EACnE;EACA,MAAM,CAAC,QAAQ,WAAW,OAAO,QAAQ,YAAA,CAAa;EACtD,OAAO,OAAO;EACd,MAAM,EAAE,UAAU;EAClB,aAAa,aAAa,MAAM,GAAG;GACjC,MAAM,OAAO;GACb;GACA;GACA,GAAI,OAAO,WAAW,KAAA,IAAY,CAAC,IAAI,EAAE,QAAQ,OAAO,OAAO,KAAK,SAAS,OAAO,EAAE;GACtF,GAAI,OAAO,UAAU,KAAA,IAAY,CAAC,IAAI,EAAE,OAAO,OAAO,MAAM,SAAS,EAAE;GACvE,GAAI,UAAU,KAAA,IACV,CAAC,IACD,EAAE,QAAQ,cAAyB,MAAM,WAAW,SAAS,EAAE;EACrE,CAAC;EACD,OAAO,IAAI,SAAS,MAAM;GACxB,QAAQ;GAGR,WAAW;EACb,CAAiB;CACnB;AACF"}
|