foldkit-remote-server 0.18.0 → 0.19.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -66,6 +66,13 @@ That separation matters. `RemoteServer` can decide that a client may read
66
66
  that field comes from Postgres, another service, an in-memory index, or
67
67
  something else.
68
68
 
69
+ It does **not** own HTTP or WebSocket transport, authentication or session
70
+ storage, a database connection, the client's normalized cache, or Surface
71
+ activation. A durable write is the client's to keep (Remote's outbox) and the
72
+ writer's to recognise when it comes again (a ledger); this package only passes
73
+ its operation id through. Client-authored state that must merge across devices
74
+ is [`foldkit-sync`](../sync)'s.
75
+
69
76
  ## Install
70
77
 
71
78
  ```bash
@@ -297,10 +304,10 @@ const ProjectsByOwner = Query.make('ProjectsByOwner', {
297
304
  Result: Query.connection(Project),
298
305
  })
299
306
 
307
+ // `projectsPage` is your own reader: one page of edges for the window.
300
308
  const ProjectsByOwnerSource = RemoteServer.query(
301
309
  ProjectsByOwner,
302
- ({ input, window, principal, select }) =>
303
- projectsPage({ ownerId: input.ownerId, window, principal }),
310
+ ({ input, window, principal }) => projectsPage({ ownerId: input.ownerId, window, principal }),
304
311
  )
305
312
 
306
313
  const Server = RemoteServer.make({
@@ -311,7 +318,7 @@ const Server = RemoteServer.make({
311
318
 
312
319
  A Query returns:
313
320
 
314
- ```ts
321
+ ```ts sketch
315
322
  {
316
323
  edges,
317
324
  start,
@@ -330,78 +337,13 @@ of guessing from row count.
330
337
 
331
338
  Query input is decoded through the Query's own Schema before your callback runs.
332
339
  The callback receives the decoded input, the requested `QueryWindow`, the
333
- `select` slice when the client sent one, and the principal.
334
-
335
- ### Running a query body over rows you already hold
336
-
337
- A descriptor declared with
338
- [`Query.define`](../remote#queries-and-pagination) carries its meaning as a
339
- value, and `evaluate` runs that value directly:
340
-
341
- ```ts
342
- import { evaluate } from 'foldkit-entity'
343
-
344
- evaluate(PostsBySlug.body!, { slug: 'intro' }, rows)
345
- // the matching rows, in the order the body asks for
346
- ```
347
-
348
- It lives in [`foldkit-entity`](../entity), beside the IR it interprets, because
349
- it is that IR's reference semantics rather than anything this package does —
350
- and because a client judging rows it already holds needs it without depending on
351
- a server package. This package re-exports it, so importing it from either place
352
- means the same thing.
353
-
354
- It is pure — it reads the rows it is given and nothing else — and it is the
355
- reference the compiled interpreters are checked against. `foldkit-remote-drizzle`
356
- turns the same body into SQL; the test suite runs both over the same rows in a
357
- real SQLite and requires the same ids in the same order. That is what makes a
358
- body source-neutral in fact rather than in principle.
359
-
360
- **It follows SQL, not JavaScript.** The case that matters is null: `null = null`
361
- is *unknown* in SQL and matches no row, where JavaScript would call the two
362
- equal. A comparison keeps a row only when it is true, so a null on either side
363
- drops it.
364
-
365
- What it refuses rather than guesses:
366
-
367
- - **Ordering by a column that is null in some row.** SQLite sorts nulls first,
368
- Postgres sorts them last for `asc`: the databases disagree with each other, so
369
- there is no answer to be conformant to and picking one would make this wrong
370
- against the other.
371
- - **Comparing values it has no order for**, such as a number against a string.
372
- - **Searching text that holds a NUL character**, which Postgres text cannot
373
- hold and SQLite's `like` stops at.
374
-
375
- Each throws where the query runs, naming the query. An ordering is checked
376
- before anything is sorted: every term over every matched row, in row order, so
377
- the error names the first row that breaks it (`number and string` when a later
378
- row holds text), the same on every engine, whichever pair a sort would have
379
- compared first. One matched row, or none, has nothing to order and is never
380
- refused.
340
+ `select` slice when the client sent one, the principal, and the server's
341
+ `viewer` of it.
381
342
 
382
- ### Checking an interpreter against the semantics
383
-
384
- `evaluate` is the reference, and `cases` is what it is a reference *to*: the
385
- conformance suite, exported so an interpreter written elsewhere can be held to
386
- the same semantics.
387
-
388
- ```ts
389
- import { Subject, cases, rows } from 'foldkit-remote-server'
390
-
391
- for (const { what, body, input, expected } of cases) {
392
- expect(yourInterpreter(body, input, rows).map(row => row.id)).toEqual(expected)
393
- }
394
- ```
395
-
396
- Every case is chosen to make interpreters **disagree**, which is the only kind
397
- that tests anything: mixed case, nulls on both sides of a comparison, `%` and
398
- `_` as literal text, the empty search, a predicate compared to a boolean either
399
- way round. That is not caution for its own sake — `contains` reached a released
400
- package meaning three different things because every value in the original
401
- fixture happened to be lowercase.
402
-
403
- Both interpreters here run it: this package over rows in memory, and
404
- `foldkit-remote-drizzle` compiled to SQL against a real SQLite.
343
+ A query declared with `Query.define` carries its meaning as a value, so it
344
+ needs no hand-written Source: `RemoteServer.memory` runs its body with
345
+ `evaluate`, and `foldkit-remote-drizzle` compiles it to SQL (see [Reference
346
+ semantics](#reference-semantics-evaluate-and-the-conformance-suite)).
405
347
 
406
348
  ## Mutations
407
349
 
@@ -418,6 +360,7 @@ const RenameProject = Mutation.make('RenameProject', {
418
360
  Output: Schema.Struct({ id: Schema.String }),
419
361
  })
420
362
 
363
+ // `renameProject` is your own write, answering the Output.
421
364
  const RenameProjectSource = RemoteServer.mutation(
422
365
  RenameProject,
423
366
  ({ input, principal }) =>
@@ -443,14 +386,9 @@ A refusal of who is asking, rather than of what they asked, is
443
386
  `Effect.fail(RemoteServer.forbid('Only an admin may archive.'))`: sent as
444
387
  `Forbidden` with the reason (a 403 over JSON), or with none when `forbid()` is
445
388
  given none, so the refusal says nothing of why. It is the same for every
446
- mutation, so it needs no `Refusal` declared. The Source is also given `requestId`, the client's id for the
447
- request, and `operation`, the operation's id across clients, both the same on
448
- every retry of it. Record a write under `operation`, never `requestId`: a
449
- request id is a counter in one client's Model, so two clients' first requests
450
- share it, and a record keyed by it would answer one client with another's write.
451
- `operation` is none from a client that sends none, and bounded on the wire (128
452
- characters of `A-Z a-z 0-9 : . _ -`). A Source
453
- may additionally return connection changes:
389
+ mutation, so it needs no `Refusal` declared.
390
+
391
+ A Source may additionally return connection changes:
454
392
 
455
393
  ```ts
456
394
  RemoteServer.prepend(connection, Project.ref(id))
@@ -461,49 +399,124 @@ RemoteServer.remove(connection, Project.ref(id))
461
399
  Those confirmed changes replace the corresponding optimistic connection layers
462
400
  on the client.
463
401
 
402
+ ### A write sent again: `operation`
403
+
404
+ The Source is also given `requestId`, the client's id for the request, and
405
+ `operation`, the operation's id across clients, both the same on every retry of
406
+ it. Record a write under `operation`, never `requestId`: a request id is a
407
+ counter in one client's Model, so two clients' first requests share it, and a
408
+ record keyed by it would answer one client with another's write. `operation`
409
+ is none from a client that sends none, and bounded on the wire (128
410
+ characters of `A-Z a-z 0-9 : . _ -`).
411
+
412
+ Expect the same operation more than once. A write that no answer came back
413
+ for fails on the client with `Unreachable` (`RemoteUnreachable`): the request
414
+ did not go, or what came back was no Remote answer (not JSON, or under a 408,
415
+ 429, 502, 503 or 504, which this package never answers with but a gateway or
416
+ a rate limiter may). The server may have landed it. `Data.retry` sends it
417
+ again under the same operation, and a durable write (`Data.mutate(…, {
418
+ durable: true })`) is stored in the page's outbox and sent again on its own,
419
+ after an `Unreachable` and after a reload, with the same operation id and the
420
+ same `now`. What a retry does depends on the write: setting a value sets it
421
+ again, an `expect`ed revision answers `Conflict`, and only a ledger (below)
422
+ recognises the operation and answers it from its record. A durable Mutation
423
+ therefore needs a writer with a ledger, or a handler that records by
424
+ `operation`.
425
+
426
+ The outbox never sends one principal's waiting writes as another's: it holds
427
+ its queue for one store key and scope, and sends none of it once either
428
+ changes. The server still decides each send as whoever that request
429
+ authenticates as.
430
+
431
+ ### Declared writes: no handler
432
+
464
433
  A mutation declared as a `Write` (`Mutation.write`) needs no handler.
465
434
  `RemoteServer.write(mutation, writer)` binds the input to the write, only the
466
435
  keys the client named when it named some, and hands the row's id and new values
467
436
  to an `EntityWriter`, which writes them and answers the row as written:
468
437
 
469
438
  ```ts
470
- RemoteServer.write(EditProject, writer(Db.Project)) // `writer` from foldkit-remote-drizzle
439
+ // `EditProject` is a Mutation.write; `writer` and the binding `Db` are foldkit-remote-drizzle's.
440
+ RemoteServer.write(EditProject, writer(Db.Project))
471
441
  ```
472
442
 
473
- The answer is the mutation's patch, or for a delete its deletion. With
474
- `expect`, the writer is told the revision the row was read at; no row at it is
475
- refused as `Refusal.conflict`. With `when`, the writer is handed the
476
- precondition and the input encoded, checks it in the statement that writes, and
477
- fails a row at the revision that does not hold it with `RemoteUnmet`
478
- (`RemoteServer.unmet()`), which is refused as `Refusal.unmet`: absent first, then
479
- a moved revision, then the precondition. A row the write would leave that
480
- breaks an invariant of its Entity (`Entity.invariant`) is the writer's to
481
- refuse, with `RemoteServer.invalid(message, rule, path)`, which is refused as
482
- `Refusal.invalid`: the invariant's id and path, or none and empty when the
483
- caller may not be told them. A writer says it judges invariants
484
- (`invariants: true`), and a write over an Entity that has some, through a writer
485
- that does not, is refused where it is served. A writer says it checks one
486
- (`preconditions: true` on `EntityWriter`), and a write with a `when` over a
487
- writer that does not is refused where it is served, rather than written whatever
488
- the row holds. A write's increments reach the writer apart from its values
489
- (`increments`, each field's amount, encoded), to be added in the statement that
490
- writes; a writer says it adds them (`increments: true`), and a write with one
491
- over a writer that does not is refused where it is served; so are computed
492
- assignments (`assignments`, each a `Computation`), over a writer that computes
493
- them (`assignments: true`). An increment or a computed assignment that expects
494
- no revision is served only over a writer with a ledger (`ledger: true`), which
495
- records each update under its `operation` and answers a retry from the record
496
- (`recorded`), and only for a request carrying an operation id. Even so, a
497
- concurrent pair must land as one result: an increment only of an integer, and a
498
- computed assignment only where the writer `commutes` it, unless the writer pins
499
- the row it read (`pins: true`, a guard with a revision). A writer
500
- may refuse a write for what only its store knows (`refuses`, asked once where
501
- the write is served): the Drizzle writer refuses an amount that may be a
502
- fraction for a column that holds integers. An update of no row fails; an insert of an id
503
- already holding what it writes answers that row, and a delete of a row already
504
- gone succeeds, so a retry of either is safe. An insert of an id holding
505
- anything else fails: that row is not its answer. The writer must be for the write's own Entity,
506
- which is checked where the mutation is served.
443
+ The answer is the mutation's patch, or for a delete its deletion. The writer
444
+ must be for the write's own Entity, which is checked where the mutation is
445
+ served. The [declared writes guide](../../docs/declared-writes.md) walks
446
+ through the whole path; what this package does with each part of a write:
447
+
448
+ - **What the server chooses.** A write of an Entity with `Entity.sets` is
449
+ handed the members its context sets as well, never from the input (an
450
+ insert all of them, an update those set `on: 'write'`): `Expr.viewer` from
451
+ `RemoteServer.make({ viewer: principal => Option<{ entity, id }> })`, and
452
+ `Expr.now` from the write's time. A value that comes to null where the
453
+ member holds none (a visitor's) is refused as forbidden; a server that
454
+ serves a write setting the viewer with no `viewer` is refused where it is
455
+ made. A path from the viewer is read through the server's own sources, as
456
+ the asker may see each row (the run context's `read`); a server that mounts
457
+ no source for a row the path reaches is refused where it is made, and a
458
+ path reaching no row the asker may see is null.
459
+ - **The write's time** is the client's when it sent one (`Data.mutate`'s
460
+ `now`, the moment the reader acted, which the client showed the row at) and
461
+ it lies within `RemoteServer.make({ clockSkew })` of the server's Effect
462
+ `Clock`: 60 seconds unless given, `0` to never take a client's time.
463
+ Otherwise it is the server's own, and a time past a bound above zero is
464
+ reported as a `ClockSkewed` diagnostic: a durable write sent long after it
465
+ was made takes the server's time. The bound is how far a client can backdate
466
+ or postdate a row; nothing else of a write's context comes from the client.
467
+ A hand-written mutation reads the same time as `now` in its context. A
468
+ retried insert, which only a writer's ledger recognises, keeps its first
469
+ time. A retried update is a new write to a server without a ledger, and
470
+ stores the time of the attempt that ran: the same sent `now` while it lies
471
+ within `clockSkew`, else the server's clock at that attempt (#291). A ledger
472
+ answers the retry from its record, so the first time stays.
473
+ - **`expect`**: the writer is told the revision the row was read at; no row
474
+ at it is refused as `Refusal.conflict`.
475
+ - **`when`**: the writer is handed the precondition and the input encoded,
476
+ checks it in the statement that writes, and fails a row at the revision
477
+ that does not hold it with `RemoteUnmet` (`RemoteServer.unmet()`), refused
478
+ as `Refusal.unmet`: absent first, then a moved revision, then the
479
+ precondition.
480
+ - **Invariants** (`Entity.invariant`) are the writer's to judge over the row
481
+ the write would leave, refusing with `RemoteServer.invalid(message, rule,
482
+ path)`, sent as `Refusal.invalid`: the invariant's id and path, or none and
483
+ empty when the caller may not be told them.
484
+ - **Increments and computed assignments** reach the writer apart from its
485
+ values (`increments`, each field's amount, encoded; `assignments`, each a
486
+ `Computation`), to be applied in the statement that writes. One that
487
+ expects no revision is served only over a writer with a ledger, and only
488
+ for a request carrying an operation id; even so, a concurrent pair must land
489
+ as one result: an increment only of an integer, and a computed assignment
490
+ only where the writer `commutes` it, unless the writer pins the row it read
491
+ (`pins: true`, a guard with a revision).
492
+ - **A rule over another row.** A writer that holds an update's or a delete's
493
+ row to a `requires` (`RowRequirement`: bodies over the row, all or any of
494
+ them) in its own statement refuses a row that fails it with `Forbidden`:
495
+ `foldkit-guard` hands it a rule that reads another row that way.
496
+ - **Creates and deletes.** An update of no row fails, and a delete of a row
497
+ already gone succeeds, so its retry is safe. An insert of an id already
498
+ taken is refused as `Refusal.taken` (the writer fails `insert` with
499
+ `RemoteTaken`, `RemoteServer.taken()`), with one answer whatever the row
500
+ there holds and whether its asker may see it: comparing that row with the
501
+ insert would tell the asker what it holds. Only a writer with a ledger tells a retry from another create
502
+ of the id, by the operation the client sent, and answers it with the row;
503
+ without one, a create whose answer was lost is refused when it is sent
504
+ again. Give a writer a ledger where a client may resend a create, and make
505
+ ids that a guess cannot find where whether a row exists is itself private.
506
+
507
+ A writer says what it can do, and `RemoteServer.write` refuses, where the
508
+ write is served, one that needs what its writer lacks rather than writing it
509
+ wrongly:
510
+
511
+ | `EntityWriter` says | It | Needed by |
512
+ | --- | --- | --- |
513
+ | `preconditions: true` | checks `when` in the statement | a write with a `when` |
514
+ | `invariants: true` | judges the Entity's invariants | a write of an Entity that has any |
515
+ | `increments: true` | adds `increments` in the statement | a write with an increment |
516
+ | `assignments: true` | computes `assignments` in the statement | a write with a computed assignment |
517
+ | `ledger: true` (with `recorded`) | records each update and insert under its `operation`, and answers a retry from the record | an increment or computed assignment with no `expect` |
518
+ | `requires: true` | holds the row to `requires` | `GuardServer.writer` over a rule reading another row, which throws where it is made without it |
519
+ | `refuses(write)` | names what only its store knows it cannot land (the Drizzle writer: a fractional amount for an integer column) | every write, asked once |
507
520
 
508
521
  A writer with a `read` (the row as a principal may see it, under the read
509
522
  rule its Source applies) answers through it: the patch is the written row as
@@ -515,34 +528,128 @@ it where the server would not list it. A writer may refuse with
515
528
  rows they point at, so such a guard can tell a write that unlinks a hidden row
516
529
  from one that changes nothing; `foldkit-remote-drizzle`'s `writer` has both.
517
530
 
518
- Remote mutations are immediate requests against **server-owned** state. They are
519
- not a durable offline intent log; client-authored edits that must survive offline
520
- belong to `foldkit-sync`.
531
+ ### What an answer carries
532
+
533
+ A Source answers the rows it changed; the client says what it shows of them.
534
+ `Data.mutate` sends, with each request, what its active Surfaces read of each
535
+ entity and every field its optimistic layer shows (`shown`). After the Source runs, the handler reads, of each row the
536
+ outcome names (a patch, or a list insert), the shown fields the patch left out
537
+ and every shown relation, and answers that read beside the patch (`read`):
538
+
539
+ ```text
540
+ outcome rows × client's shown fields − fields the patch carries -> read -> answer
541
+ ```
542
+
543
+ So a declared write, which answers the columns it stored, also answers the
544
+ derived count, the `may` and the author's name the page reads of the row, and
545
+ the client releases its optimistic layer with nothing left to fetch. The read
546
+ is an ordinary one: the Sources' `read`, `authorize` and `filter` decide it, a
547
+ field the asker may not read comes back settled, and a row they cannot see is
548
+ left out of it, keeping what the patch gave.
549
+
550
+ A Source whose write the server cannot see (a row of a join table, a count it
551
+ keeps itself) names the row with no values, and the read answers all of it:
552
+
553
+ ```ts
554
+ // Sketch, inside a mutation Source over a `Post` Entity: a like in a join table
555
+ // changes its post, so name the post and read nothing yourself.
556
+ return { output: { liked: true }, entities: [Remote.patch(Post, input.postId, {})] }
557
+ ```
558
+
559
+ The read runs after the write commits, outside its transaction. If it fails,
560
+ it is reported as an `AnswerUnread` [diagnostic](#diagnostics) and the
561
+ mutation is answered without it: the write has
562
+ landed, and the client fetches what it lacks as it would have. A deleted row is
563
+ not read.
564
+
565
+ A declared write (`RemoteServer.write`) also reads each row its row points at
566
+ through a `one` relation, before and after the write, through the writer's
567
+ `linked`, where that link can change one of the row's derived members: a
568
+ `Derived.count` or `Derived.exists` over that relation, or any supplied member
569
+ (`Derived.make`), which may read anything. Inserting a `Like` reads what the
570
+ page shows of its post, the new `likeCount` included
571
+ ([#233](https://github.com/doeixd/foldkit-plus/issues/233)), and not its user,
572
+ whose members no like changes. A member declared after `Entity.relate` counts:
573
+ the Entity has one set of members, whichever descriptor declared them
574
+ ([#332](https://github.com/doeixd/foldkit-plus/issues/332)). A row's derived
575
+ members change only with the rows pointing at it, so no app names them. These rows are read, never sent as patches of their own: one the
576
+ asker may not see is not answered at all. Live subscribers hear the outcome's
577
+ own patches, not the read, which is one asker's view.
521
578
 
522
579
  ## Live data
523
580
 
524
- There are two ways to produce live changes.
581
+ A live subscription is one stream per Entity, from that Entity's live Source.
582
+ There are three ways to produce one:
583
+
584
+ - write a live Source directly, below;
585
+ - over Drizzle, `foldkit-remote-drizzle`'s `pollLive`, a live Source that
586
+ re-reads the subscribed rows and lists from inside the subscriber's own
587
+ request, which is what Workers need;
588
+ - in a long-lived process, a `liveHub` that re-reads what each write changed.
525
589
 
526
590
  ### Write a live Source directly
527
591
 
528
592
  ```ts
593
+ // `projectEvents` is your own stream of `LiveChange`s.
529
594
  const ProjectLive = RemoteServer.live(Project, {
530
- subscribe: ({ requirements, after, principal }) =>
595
+ subscribe: ({ requirements, queries, after, principal }) =>
531
596
  projectEvents({
532
597
  requirements,
598
+ queries,
533
599
  after,
534
600
  principal,
535
601
  }),
536
602
  })
537
603
  ```
538
604
 
539
- `after` is the client's resume cursor. Events at or before it are duplicates.
605
+ `after` is the client's resume cursor. Events at or before it are duplicates,
606
+ and the stream's cursors count on from it: its first event is `after + 1`, or
607
+ the client takes the skipped cursors for a gap and restarts the stream.
540
608
  The stream carries Remote `LiveChange`s: entity patches/deletes and connection
541
609
  insert/remove/invalidate events.
542
610
 
611
+ `queries` are the lists of this Entity the subscriber reads live
612
+ (`Data.query(…, { live: true })`), each a `LiveQuery` the handler has already
613
+ admitted and resolved: `identity` (the connection as the client keys it, for
614
+ the connection events), `select` (what the read draws of each row), and
615
+ `edges`, the registered query run as the subscriber with the page's window.
616
+ The handler refuses, with `RemoteLiveRefused` and before any live Source is
617
+ asked, a list of a query it does not register, an input that does not
618
+ decode, one with no `select` or no `first`, one whose `select` is of another
619
+ Entity than its query lists (a `QuerySource` names that Entity: `RemoteServer.query`
620
+ reads it from the descriptor's connection), and one whose `first`, added to
621
+ the ids of its Entity the subscription names, passes `maxIdsPerEntity`. A
622
+ subscription watches at most 32 lists (`MAX_LIVE_QUERIES`), refused at decode.
623
+
543
624
  If an Entity has no live Source, it contributes no stream. That is not an error:
544
625
  the client planner can fall back to refetching it.
545
626
 
627
+ A live Source that reads rows to send them must read them as a read would, or
628
+ it sends what the read handlers withhold. `RemoteServer.readAs` is that read:
629
+ only the asked fields the Source declares and its `authorize` allows, each
630
+ record through its `filter`, and no other field.
631
+
632
+ ```ts
633
+ const asReader = RemoteServer.readAs(ProjectSource, {
634
+ ids: ['p1'],
635
+ fields: ['name', 'privateNotes'],
636
+ principal: { isAdmin: false },
637
+ })
638
+ // An Effect that succeeds with Option.some([{ id: 'p1', values: { name: 'Apollo' } }]),
639
+ // or Option.none() when the principal may read none of the fields asked.
640
+ ```
641
+
642
+ `RemoteServer.allowedFields(ProjectSource, principal, fields)` says which of
643
+ `fields` that read would read, without reading: a Source reading several rows
644
+ together, each asked for fields of its own, reads them in one read and still
645
+ tells nothing of a row whose fields the principal may read none of.
646
+
647
+ A live Source that sends only what it reads through one Source names it as
648
+ `reads`, so a startup check can trust it as far as it trusts that Source:
649
+ `GuardServer.validate` accepts a live Source over a guarded Entity only when
650
+ it `reads` the guarded Source it checked. `foldkit-remote-drizzle`'s `pollLive`
651
+ reads through `readAs` and sets `reads`.
652
+
546
653
  ### Let a `liveHub` re-read changed fields
547
654
 
548
655
  A hub fans **what changed** out to live subscribers. Give it to the handlers,
@@ -551,6 +658,7 @@ fields it carries, each deletion as a deletion. A mutation Source names nothing
551
658
  twice.
552
659
 
553
660
  ```ts
661
+ // Inside an Effect.gen: building a hub is an Effect.
554
662
  const entitySources = [ProjectSource]
555
663
  const hub = yield* RemoteServer.liveHub(entitySources)
556
664
 
@@ -559,13 +667,14 @@ const handlers = RemoteServer.handlers(Server, principal, {
559
667
  })
560
668
 
561
669
  // Only for a write that is not a mutation: a database trigger consumer, a job.
562
- yield* hub.changed(Project.ref(projectId), ['status', 'updatedAt'])
563
- yield* hub.deleted(Project.ref(projectId))
670
+ yield* hub.changed(Project.ref('p1'), ['name'])
671
+ yield* hub.deleted(Project.ref('p1'))
564
672
  ```
565
673
 
566
674
  A mutation's publish runs after its Source returned, so after a write that
567
- commits as it goes. A failed publish is logged and the mutation still answers:
568
- its write has happened, and a failed answer would be retried.
675
+ commits as it goes. A failed publish is reported as a `PublishFailed`
676
+ [diagnostic](#diagnostics) and the mutation still answers: its write has
677
+ happened, and a failed answer would be retried.
569
678
 
570
679
  The hub remembers each subscriber's requirements and principal. `changed`:
571
680
 
@@ -586,7 +695,11 @@ Subscribers that selected none of the changed fields do no work, and
586
695
  subscribers with one principal share one re-read. One principal means the same
587
696
  value, unless the hub is given a `key`: then two requests whose principals
588
697
  have one key, each with a principal object of its own, share it, read as the
589
- first of them. A key must therefore name everything the Sources decide by:
698
+ first of them. A key must therefore name everything the Sources decide by,
699
+ the reader itself included when a Source answers a viewer-relative member (a
700
+ `Derived.exists`, whose value is each reader's own; `foldkit-guard`'s
701
+ `Policy.keyOf` names every field of the principal). A key by role, as here,
702
+ fits Sources that answer only by role:
590
703
 
591
704
  ```ts
592
705
  const hub = yield* RemoteServer.liveHub(entitySources, {
@@ -622,14 +735,16 @@ const SharedProjectSource = RemoteServer.entity<Principal>(Project, {
622
735
  re-read.
623
736
  - `foldkit-remote-drizzle`'s `source` and `RemoteServer.memory` take it too,
624
737
  and `foldkit-guard`'s `GuardServer.validate` refuses a shared field a Read
625
- rule could tell readers apart by.
738
+ rule could tell readers apart by. `source` refuses a `Derived.exists` as
739
+ shared: it is each reader's own.
626
740
 
627
741
  A hub is an in-memory fan-out, so whatever calls `changed` must be able to wake
628
742
  the subscriber's fiber. In a worker it cannot: one request cannot wake another's
629
743
  stream, so a hub fed from a second request delivers nothing and tears the first
630
- stream down (see `AGENTS.md`). On Workers, poll the shared store from inside the
631
- subscriber's own request instead — `examples/cloudflare` has the shape. A hub is
632
- right for a long-lived server process that owns both sides.
744
+ stream down: the open response belongs to the request that opened it, and the
745
+ Workers runtime cancels it when another request wakes it. On Workers, poll the
746
+ shared store from inside the subscriber's own request instead, as `pollLive`
747
+ does. A hub is right for a long-lived server process that owns both sides.
633
748
 
634
749
  A hand-written live Source and a hub number their events independently. Use one
635
750
  or the other for a given subscription path rather than mixing cursor domains.
@@ -688,7 +803,7 @@ A complete server definition is just the collection of Sources:
688
803
 
689
804
  ```ts
690
805
  const Server = RemoteServer.make({
691
- entities: [ProjectSource, UserSource],
806
+ entities: [ProjectSource],
692
807
  queries: [ProjectsByOwnerSource],
693
808
  mutations: [RenameProjectSource],
694
809
  live: [ProjectLive],
@@ -700,7 +815,7 @@ Source names at startup:
700
815
 
701
816
  ```ts
702
817
  const Data = Remote.define({
703
- entities: [Project, User],
818
+ entities: [Project],
704
819
  queries: [ProjectsByOwner],
705
820
  mutations: [RenameProject],
706
821
  })
@@ -711,6 +826,15 @@ RemoteServer.validate(Data, Server)
711
826
  That catches an Entity, Query, or Mutation Source whose name is not declared by
712
827
  the shared domain instead of letting it silently answer nothing at runtime.
713
828
 
829
+ `viewer` says who asks, as a row, and is the server's one answer to it:
830
+ `RemoteServer.make({ ..., viewer: principal => Option<{ entity, id }> })`. The
831
+ server hands it to every Source it mounts, with each read, query and live
832
+ subscription (`EntitySourceContext.viewer`), and to every write. A Source
833
+ answering a member per reader declares it (`perViewer`, as a `Derived.exists`
834
+ does), and a server with no `viewer` refuses that Source where it is made,
835
+ naming it. A query body's `Expr.viewer` and `Expr.now` read this viewer and the
836
+ server's Effect `Clock` (`exprContext`).
837
+
714
838
  Then compile it into handlers, once per principal, or once for every request
715
839
  with a Context key as the principal (see Authentication vs authorization):
716
840
 
@@ -768,13 +892,26 @@ The handler applies bounded, protocol-level safeguards before or around Sources:
768
892
  - the wire decoder also rejects a selection deeper than
769
893
  `MAX_RELATION_DEPTH` (`8`) or more than `MAX_FIELDS_PER_REQUEST` (`256`)
770
894
  fields for one Entity;
895
+ - a collection relation read with no window is the Source's to bound:
896
+ `foldkit-remote-drizzle`'s `source` reads at most `maxCollection` children
897
+ of one row (default 100) and refuses a read that finds more, rather than
898
+ answering a list cut short;
771
899
  - returned field names are copied with `Object.hasOwn` into null-prototype
772
900
  objects, so names such as `__proto__` or `toString` cannot reach prototypes;
773
901
  - Source errors are remapped to protocol error types without echoing application
774
902
  internals.
775
903
 
776
- A `liveHub` applies the same id bound to a subscription. A hand-written live
777
- Source is responsible for any additional bounds appropriate to its backend.
904
+ A live subscription is held to the same bounds, checked once by the handler
905
+ before any live Source or the hub sees it: more than `maxIdsPerEntity` ids of
906
+ one Entity (the `first` of each live list of it counted with them), or more
907
+ pages of one relation than a read may ask, fails the stream with
908
+ `RemoteLiveRefused` and nothing is read; more than `MAX_LIVE_QUERIES` (32) live
909
+ lists fails the wire decoder before that. That matters most for a polling
910
+ Source such as `pollLive`, which re-reads every subscribed row each tick. A
911
+ refusal is not truncation: the client does not reconnect it, and
912
+ `Data.liveStatus` reads `Refused`. A hand-written live Source can rely on what
913
+ it is given having passed, and is responsible for any further bounds its
914
+ backend needs.
778
915
 
779
916
  ## A backend held in memory
780
917
 
@@ -788,10 +925,9 @@ is refused where the server is built.
788
925
 
789
926
  ```ts
790
927
  const backend = RemoteServer.memory({
791
- domain: Data, // or the definition from Remote.define
928
+ domain: Data, // the definition from Remote.define, or a domain from Remote.make
792
929
  rows: {
793
- User: [{ id: 'u1', name: 'Ada' }],
794
- Project: [{ id: 'p1', name: 'Apollo', status: 'active', owner: 'User:u1' }],
930
+ Project: [{ id: 'p1', name: 'Apollo', privateNotes: 'Internal' }],
795
931
  },
796
932
  // Optional: each mutation writes through the store, and the next read sees it.
797
933
  mutations: store => [
@@ -942,11 +1078,21 @@ const route = async (request: Request): Promise<Response> => {
942
1078
  call, of whatever kind, with `_tag: 'Unauthenticated'`.
943
1079
  - **403** with `{ error, forbidden: { reason? } }`: who is asking may not do
944
1080
  this (`RemoteServer.forbid`).
1081
+ - **409** with `{ error, refusal }`: a declared insert's id is already a
1082
+ row's, refused as `Refusal.taken`.
945
1083
  - **422** with `{ error, refusal }`: the mutation refused what was asked, in
946
1084
  its own `Refusal`.
947
1085
  - **500** with `{ error }`: the handler failed, and the message is its own;
948
1086
  or something on this side broke, and the message is `Internal error`,
949
- nothing more.
1087
+ nothing more. The cause is reported as a `ServerDefect`
1088
+ [diagnostic](#diagnostics), by default to the log, so a Worker's log
1089
+ (`wrangler tail`) shows what broke.
1090
+
1091
+ It never answers 408, 429, 502, 503 or 504. Remote's client reads a body
1092
+ under one of those, or one that is not JSON, as no answer at all
1093
+ (`Unreachable`): a proxy, gateway or rate limiter in front that answers one
1094
+ leaves the client unsure whether the write ran, and a durable write is sent
1095
+ again (see [A write sent again](#a-write-sent-again-operation)).
950
1096
 
951
1097
  It binds no principal: pass the `handlers` for the request's own,
952
1098
  established by your authentication, as everywhere else. It carries no live
@@ -964,6 +1110,7 @@ what `layer` provides (a session store, an auth service):
964
1110
  ```ts
965
1111
  import { serveFetch } from 'foldkit-remote-server/fetch'
966
1112
 
1113
+ // `server` is your RemoteServer.make; `databaseLayer` and `drizzle` provide what its Sources need.
967
1114
  export default {
968
1115
  fetch: serveFetch({
969
1116
  server,
@@ -974,8 +1121,12 @@ export default {
974
1121
  }
975
1122
  ```
976
1123
 
977
- Only `POST`s to `path` (default `/remote`) are answered; anything else is a 404. A body that is not JSON, or not a request, is a 400 that reaches no
978
- handler; a handler's failure a 500 with its message.
1124
+ Only `POST`s to `path` (default `/remote`) are answered; anything else is a
1125
+ 404. A body that is not JSON, or not a request, is a 400 that reaches no
1126
+ handler; a handler's failure a 500 with its message. Every response, refusals
1127
+ and errors included, says `cache-control: no-store` unless `headers` gives a
1128
+ `cache-control` of its own: each answer is one principal's rows (and `may`), so
1129
+ no shared cache may keep one.
979
1130
 
980
1131
  A resolver refuses the caller by failing with
981
1132
  `RemoteServer.unauthenticated(reason?)`: a 401 whose body is
@@ -1015,27 +1166,164 @@ the events it serves, so a stream with nothing to serve still ends.
1015
1166
  calls it must be able to wake the subscriber's fiber: in a worker, one request
1016
1167
  cannot wake another's stream, so a hub fed from a second request delivers
1017
1168
  nothing and ends the first stream. Poll the shared store from inside the
1018
- subscriber's own request instead — `examples/cloudflare`'s `pollLive` is the
1019
- shape. A hub is right for a long-lived process that owns both sides.
1169
+ subscriber's own request instead: `foldkit-remote-drizzle`'s `pollLive` does,
1170
+ with a trigger-kept row log (`rowChangeLog`, `rowLog`) so a quiet tick reads
1171
+ no rows. A hub is right for a long-lived process that owns both sides.
1172
+
1173
+ ### A page's reads, in the same process: `pageReader`
1174
+
1175
+ A Worker that renders pages reads what each page shows before rendering it
1176
+ (`Data.satisfy`), as the reader who asked. `pageReader` is that reader, made
1177
+ from the same config `serveFetch` takes: the request's `RemoteClient` as a
1178
+ Layer, answered by the server's own handlers with no HTTP in between. The
1179
+ server, its `layer` and its limits are the ones `/remote` answers with.
1180
+
1181
+ ```ts
1182
+ import { pageReader, serveFetch } from 'foldkit-remote-server/fetch'
1183
+
1184
+ const remote = { server, resolvePrincipal, layer: (env: Env) => databaseLayer(drizzle(env.DB)) }
1185
+ const served = serveFetch(remote)
1186
+ const reader = pageReader({ ...remote, resolvePrincipal: resolvePage })
1187
+
1188
+ export default {
1189
+ fetch: (request: Request, env: Env) =>
1190
+ new URL(request.url).pathname === '/remote'
1191
+ ? served(request, env)
1192
+ : pages.fetch(PageContext.attach(request, { reader: reader(request, env) })),
1193
+ }
1194
+ ```
1195
+
1196
+ `pages` is the fetch handler Foldkit's build emits, and `PageContext` an
1197
+ `SSR.requestContext` that carries the reader to `renderPage`
1198
+ ([`foldkit-ssr`](../ssr/README.md#what-the-host-hands-the-page-ssrrequestcontext)).
1199
+
1200
+ The resolver is the page's own: a page `GET` is a navigation, which carries
1201
+ no header a script adds, so a resolver that asks for one (Yielded's CSRF
1202
+ header) refuses it. With `foldkit-auth`, give
1203
+ `AuthServer.resolver({ …, requests: 'navigation' })`. When the reader is
1204
+ provided, the principal is resolved and `layer` built, and `layer` closes
1205
+ when that Effect ends, so provide it to the pre-read itself
1206
+ (`Data.satisfy(…).pipe(Effect.provide(reader))`). A refusal fails it with
1207
+ `Unauthenticated`, as a 401 would.
1208
+
1209
+ The reader only reads. A mutation fails with `RemoteMutationError` and a live
1210
+ open with `RemoteLiveError`: another site can start a navigation, so the
1211
+ principal it resolves must never write, and a render keeps no stream.
1212
+
1213
+ ## Diagnostics
1214
+
1215
+ A server answers some failures without telling the caller why. Each is a
1216
+ typed diagnostic, reported through `Diagnostics.Reporter` (`foldkit-surface`'s
1217
+ channel, re-exported by `foldkit-remote`), which by default is Effect's log:
1218
+
1219
+ | `_tag` | `level` | When |
1220
+ | --- | --- | --- |
1221
+ | `ServerDefect` | `error` | Something on this side broke (a layer, a resolver, a handler's defect) and the caller was answered `Internal error`; it carries the `cause`. |
1222
+ | `AnswerUnread` | `warning` | A mutation committed, but reading its answer failed: the client reads those rows again. |
1223
+ | `PublishFailed` | `error` | A mutation committed, but its live publish failed: live subscribers do not hear it. |
1224
+ | `ClockSkewed` | `warning` | A write's time was past `clockSkew` from the server's clock (a client clock that is off, or a durable write that waited), so the server wrote its own; it carries `sent`, `own` and `bound`. |
1225
+
1226
+ Provide another reporter as any service: `Layer.succeed(Diagnostics.Reporter,
1227
+ report)` in the layer the handlers run in. `serveFetch` takes one with the
1228
+ request beside it, used for everything that request runs, a broken layer
1229
+ included:
1230
+
1231
+ ```ts
1232
+ serveFetch({
1233
+ server,
1234
+ resolvePrincipal,
1235
+ layer: (env: Env) => databaseLayer(env),
1236
+ diagnostics: (diagnostic, request) =>
1237
+ Effect.logError(`${request.method} ${request.url}: ${diagnostic.message}`),
1238
+ })
1239
+ ```
1240
+
1241
+ In a test, `Diagnostics.collect()` gives a reporter and the array it fills.
1242
+
1243
+ ## Reference semantics: `evaluate` and the conformance suite
1244
+
1245
+ ### Running a query body over rows you already hold
1246
+
1247
+ A descriptor declared with
1248
+ [`Query.define`](../remote#queries-and-pagination) carries its meaning as a
1249
+ value (`body`), and `evaluate` runs a body directly:
1250
+
1251
+ ```ts
1252
+ import { Entity, Expr, Order, Query, evaluate } from 'foldkit-entity'
1253
+
1254
+ const Post = Entity.define('Post', Schema.Struct({ id: Schema.String, slug: Schema.String }))
1255
+ const intro = Query.from(Post).pipe(
1256
+ Query.where(Expr.eq(Post.fields.slug, 'intro')),
1257
+ Query.orderBy(Order.asc(Post.fields.id)),
1258
+ )
1259
+
1260
+ evaluate(intro, {}, [{ id: 'p1', slug: 'intro' }, { id: 'p2', slug: 'outro' }])
1261
+ // [{ id: 'p1', slug: 'intro' }]: the matching rows, in the order the body asks for
1262
+ ```
1263
+
1264
+ It lives in [`foldkit-entity`](../entity), beside the IR it interprets, because
1265
+ it is that IR's reference semantics rather than anything this package does —
1266
+ and because a client judging rows it already holds needs it without depending on
1267
+ a server package. This package re-exports it, so importing it from either place
1268
+ means the same thing.
1020
1269
 
1021
- ## What it does not own
1270
+ It is pure — it reads the rows it is given and nothing else — and it is the
1271
+ reference the compiled interpreters are checked against. `foldkit-remote-drizzle`
1272
+ turns the same body into SQL; the test suite runs both over the same rows in a
1273
+ real SQLite and Postgres and requires the same ids in the same order. That is
1274
+ what makes a body source-neutral in fact rather than in principle.
1022
1275
 
1023
- `foldkit-remote-server` intentionally does **not** own:
1276
+ **It follows SQL, not JavaScript.** The case that matters is null: `null = null`
1277
+ is *unknown* in SQL and matches no row, where JavaScript would call the two
1278
+ equal. A comparison keeps a row only when it is true, so a null on either side
1279
+ drops it. Where an ordering puts null is the term's to say
1280
+ (`Order.asc(field, { nulls: 'first' })`); unsaid, it is Postgres's (last
1281
+ ascending, first descending), which the SQL compiler writes out for SQLite too.
1282
+
1283
+ What it refuses rather than guesses, each where the query runs:
1284
+
1285
+ - **Ordering by values it has no order for**, such as a number in one row and
1286
+ text in another, or by a backend's locale collation, which only that backend
1287
+ has. Every term is checked over every matched row before anything is sorted,
1288
+ so the refusal is the same on every engine, whichever pair a sort would have
1289
+ compared first. One matched row, or none, has nothing to order.
1290
+ - **NaN**, which SQLite stores as null and Postgres orders after every number.
1291
+ - **Text holding a NUL character** in a comparison or a search, which Postgres
1292
+ text cannot hold and SQLite's `like` stops at.
1293
+ - **A term with no context.** A body reading `Expr.viewer` or `Expr.now` is
1294
+ refused when run without them (`ContextMissingError`), never read as null.
1024
1295
 
1025
- - HTTP/WebSocket transport or serialization;
1026
- - authentication protocol or session storage;
1027
- - a database connection;
1028
- - the client's normalized cache;
1029
- - client rendering or Surface activation;
1030
- - durable/offline client-authored state.
1296
+ ### Checking an interpreter against the semantics
1031
1297
 
1032
- Those boundaries keep this package an interpreter of the Remote contract rather
1033
- than another application runtime.
1298
+ `evaluate` is the reference, and `cases` (in `foldkit-entity/conformance`) is
1299
+ what it is a reference *to*: the conformance suite, exported so an interpreter
1300
+ written elsewhere can be held to the same semantics. Here `run` is your
1301
+ interpreter:
1302
+
1303
+ ```ts
1304
+ import { cases, rows } from 'foldkit-entity/conformance'
1305
+
1306
+ for (const { what, body, input, expected } of cases) {
1307
+ expect(run(body, input, rows).map(row => row.id), what).toEqual(expected)
1308
+ }
1309
+ ```
1310
+
1311
+ Every case is chosen to make interpreters **disagree**, which is the only kind
1312
+ that tests anything: mixed case, nulls on both sides of a comparison, `%` and
1313
+ `_` as literal text, the empty search, a predicate compared to a boolean either
1314
+ way round. That is not caution for its own sake — `contains` reached a released
1315
+ package meaning three different things because every value in the original
1316
+ fixture happened to be lowercase.
1317
+
1318
+ Both interpreters here run it: this package over rows in memory, and
1319
+ `foldkit-remote-drizzle` compiled to SQL against a real SQLite and Postgres.
1034
1320
 
1035
1321
  ## See also
1036
1322
 
1037
1323
  - [Server-derived state](../../docs/remote.md) — the client/server mental model
1038
1324
  and ownership rules.
1325
+ - [Errors and failures](../../docs/errors.md) — what a client is and is not
1326
+ told, beside the failures every other package owns.
1039
1327
  - [`foldkit-remote`](../remote) — requirements, normalized client Model,
1040
1328
  subscriptions, mutations, live reads, and retention.
1041
1329
  - [`foldkit-remote-drizzle`](../remote-drizzle) — compile Entity selections and