foldkit-remote 0.5.0 → 0.6.0

package/README.md

@@ -55,67 +55,17 @@ ownership.
 
 ## The mental model
 
-The client side is a Foldkit Submodel:
-
-```text
-                         pure
-                    Projection read
-                         |
-                         v
-+-------------+     +---------+      +------+
-| Foldkit     |---->| Surface |----->| View |
-| Model       |     +---------+      +------+
-|             |
-| Remote.Model|
-| normalized  |
-| cache       |
-+------+------+ 
-       ^
-       | Remote Message
-       |
-+------+-------+
-| Data.reduce |
-+------+-------+
-       ^
-       |
-       | read/query/mutate/live
-       |
-+------+--------+
-| RemoteClient |
-+------+--------+
-       |
-       v
-     server
-```
-
-The other half of the model is the **requirement planner**:
-
 ```text
-Surface says:
-  "I need Project p1: id, name, owner.name"
-
-                    |
-                    v
-Remote compares that requirement
-with the normalized cache in Model
-
-                    |
-                    v
-missing:
-  Project:p1.name
-  User:u7.name
-
-                    |
-                    v
-active subscription performs one read
-
-                    |
-                    v
-result -> Remote Message -> Data.reduce -> Model
+Surface Projection → requirements → active Subscription → RemoteClient I/O
+        ↑                                                    |
+        └──────── read Model ← Data.reduce ← Remote Message ──┘
 ```
 
-That separation is why reading stays pure. Rendering a Surface never starts a
-request.
+A Projection reads the current cache and carries requirements such as
+`Project:p1.name` and `User:u7.name`. An active Subscription compares those
+requirements with the cache, then requests missing or stale fields. The
+result returns as a Message; only reducing it changes what the next read sees.
+Rendering the same Projection twice starts no work by itself.
 
 ## Four pieces to remember
 
@@ -133,7 +83,7 @@ Everything else builds on those four pieces.
 ## Install
 
 ```bash
-pnpm add foldkit-remote
+pnpm add foldkit-remote foldkit-entity
 ```
 
 `foldkit` and `effect` are peer dependencies; `foldkit-surface` comes with the
@@ -291,7 +241,14 @@ Remote.Model
 ProjectPage now reads Ready(...)
 ```
 
-Finally, provide the client implementation to the runtime:
+The first example starts on the `home` route, so no project read runs yet.
+Your route update must activate `{ _tag: 'project', projectId: 'p1' }`, and the
+runtime must install `subscriptions`. Expect `Initial` until work starts,
+`Loading` during the first request, and `Ready` once its result is reduced.
+
+Finally, provide the client implementation to the runtime. Here `rpcClient`
+is your configured Effect RPC client, not a value created by the declarations
+above; the [server guide](../remote-server/README.md) supplies the other side:
 
 ```ts
 const clientLayer = Remote.clientLayer(rpcClient)
@@ -354,15 +311,19 @@ missing ------------------------------------------------> Loading
 
 server says entity is absent ---------------------------> NotFound
 stored value fails Selection decoding -----------------> Failed
+a read or a list's query fails ------------------------> Failed (with the old value, if any)
 ```
 
 The states are:
 
 - **`Initial`** — required data is absent and **nothing is currently fetching it**.
-- **`Loading`** — required data is absent and a read is in flight.
+- **`Loading`** — required data is absent and a read, or a list's query, is in
+  flight.
 - **`Ready`** — every selected field is present and decodes.
 - **`Refreshing`** — the current value remains visible while it is being refetched.
-- **`Failed`** — stored server data does not decode against the Selection.
+- **`Failed`** — stored server data does not decode against the Selection, or
+  the read or query behind it failed. A value that was already shown is kept as
+  `previous`.
 - **`NotFound`** — the entity is represented by a tombstone: a live event
   deleted it, or the server was asked for it by id and answered without it. That
   id is then known absent, whether it never existed, is gone, or is not this
@@ -706,6 +667,159 @@ their required fields. `Data.next` / `Data.previous` preserve the page size and
 use the loaded boundaries; `hasNext` / `hasPrevious` come from those boundaries,
 not from guessing based on row counts.
 
+### Filtering a loaded list without asking the server
+
+A search box over a page you already have should not be a round trip.
+`Data.filtered` narrows a loaded list by a query body:
+
+```ts
+const found = Data.filtered(model, projects, ActiveProjects, {})
+
+found.items     // decoded exactly as the list decodes them
+found.complete  // whether the answer was about the whole list
+```
+
+**It filters a list; it does not run a query**, and the distinction is the
+design. "Which rows match" would need to know this list holds every row the body
+could match — predicate containment, which Remote deliberately does not do.
+"Which rows *of this list* match" is decidable from what is already here.
+
+`complete` is false unless three things hold: every edge could be judged (no row
+missing a field the *body* reads), every match could be shown (no row missing a
+field the *Selection* reads), and the list is terminal at both ends. So
+empty-and-complete and empty-and-partial stay different answers — one means
+"none", the other means "none that I can see yet".
+
+Nothing is created: no connection, nothing new to retain, nothing new to fetch.
+The server stays authoritative for which rows exist.
+
+### Judging the rows you already hold
+
+A connection's rows are edges the server delivered, and a query body describes
+*which rows* — so until something evaluates the body on the client, nothing here
+can say whether a row belongs to a query. `Remote.matching` says:
+
+```ts
+const judged = Remote.matching(Data.storeOf(model), ProjectsByOwner, { ownerId: 'u1' })
+
+judged.matched  // keys satisfying the body, in the order it asks for
+judged.skipped  // keys held, but missing a field the body reads
+```
+
+It is pure, and it runs the **same interpreter the server checks itself
+against** — the reference implementation in
+[`foldkit-entity`](../entity) — so a body means one thing in both places. The
+conformance suite is run over a store as well as over rows and a table.
+
+Three things about it are deliberate.
+
+**It answers "which of the rows I hold match", never "which rows match."** Those
+are different questions, and the difference is not recoverable from a list of
+keys — so it is in the shape. A caller that knows it holds the whole population
+(a connection terminal at both ends) is the one that can turn this into a
+complete answer.
+
+**`skipped` is the honest half.** A row missing a field the body reads cannot be
+judged: it is neither a match nor a non-match. Dropping it silently would turn
+"I could not tell" into "no", so it is named instead, and the caller decides
+whether to fetch the field, ask the server, or say the answer is partial.
+
+**It takes the input decoded and encodes it itself.** A store holds wire values,
+so a comparison has to happen in that space — and handing an interpreter a
+decoded value is a mistake *no runtime error catches*: the answer is simply
+empty, which reads exactly like a correct one. Doing the encoding here means a
+caller cannot get it wrong.
+
+It is **not authorization**. On the server a compiled `where` is conjoined with
+the binding's `visible` rule; there is no `visible` here. This is safe only
+because the client holds only rows the server already released to it, and a
+local filter is not an access decision.
+
+### An input that changes as fast as someone types
+
+A connection's identity is its query plus its **input**, which is exactly right
+for caching and exactly wrong for a search box. Bound naively, every keystroke
+is a different connection, a different request, and a different thing to retain:
+
+```ts
+// Don't: `postSearch` changes per keystroke, so the query's input does too.
+h.OnInput(text => Message.Searched({ text }))
+```
+
+Remote has no debounce, and should not: its job is to be a faithful function of
+the Model, so if the Model says the search is `Engi` then `Engi` is what the
+query asks. **The debounce belongs between the input Message and the Model field
+the query reads** — which is ordinary application state, and
+[`foldkit-primitives`](../primitives) already has the piece:
+
+```ts
+import { debounce } from 'foldkit-primitives/time'
+
+const SearchInput = debounce({ name: 'PostSearch', value: Schema.String })
+const SearchBox = Bundle.declare(SearchInput, 'search')
+
+// `latest` is what the box shows, so typing stays immediate.
+// The settled OutMessage is what moves the field the query reads.
+Page.at(SearchBox, {
+  args: { delayMs: 250 },
+  onOut: out => model => ({ model: evo(model, { postSearch: () => out.value }) }),
+})
+```
+
+Two fields, deliberately: the one the box draws changes on every keystroke, and
+the one the query reads changes a quarter second after the last one.
+[`examples/entity`](../../examples/entity) does exactly this.
+
+The same applies to any high-frequency query input — a slider, a map viewport, a
+date scrubber. What makes a search box the usual case is that the query is often
+written *for* it: `Expr.contains` over an empty string matches everything, so an
+empty box and a filled one are one query rather than two.
+
+### A page that overruns its window is refused
+
+The client asks for `first: 25`, and a page carrying more than that is a
+protocol disagreement rather than a windfall: the client cannot tell which of
+the edges the window meant, and the connection's boundaries stop describing what
+it holds. Such a page becomes a `QueryFailed` with a named protocol error, and
+none of its edges reach the store — a connection already loaded is left exactly
+as it was, so a refresh that overruns cannot replace good rows with a rejected
+page.
+
+A window with neither `first` nor `last` bounds nothing: `after`/`before` says
+where to start, not how much to take.
+
+### When a read fails
+
+A failure is kept on the Model until something settles it, and the read says
+so. Queries fail per connection, and entity reads per field. A read with
+nothing to show reads `Failed` with the error. One that had a value (a failed
+refresh, or a failed "load more") reads `Failed` with that value as
+`previous`, which `RemoteData.render` draws as data with a `Stale` freshness,
+so it stays on screen. A failure reaches every read that needs it: a list
+fails when one of its rows' fields failed, and a project read fails when its
+owner's did.
+
+**It is not retried on its own.** A persistent error would otherwise be asked
+again every time some unrelated read restarted the entry. What failed is left
+out of the plan, and everything else is still fetched: the rows a failed list
+holds, the other fields of an entity. What asks again:
+
+- `Data.refresh(model, projection)` — the retry button. It works on something
+  that never loaded, too.
+- The value arriving anyway: a page for a list, or a read, live patch or
+  mutation result that writes the field.
+- The server invalidating a list over a live stream, or deleting the entity.
+- Retention dropping it: something nothing reads any more forgets its failure,
+  so coming back to it later asks the server again.
+
+`Remote.inspect(model.remote).failures` lists what failed, by connection
+identity and by `entity\0id\0field` mark.
+
+A broken live stream is not a failed read. Its `ReadFailed` carries the
+`stream`, and records a gap in `RemoteModel.gaps` rather than failing any
+field: nothing was being read, and the values on screen are still what the
+server last said.
+
 ## Mutations and optimistic state
 
 Register mutations on the domain:
@@ -882,27 +996,34 @@ and refetch; Sync recovery must preserve unsent client intent.**
 
 ## Persistence and hydration
 
-Persist only the disposable entity cache, not session machinery such as live
-cursors, optimistic layers, mutation bookkeeping, gaps, or retention roots.
+A snapshot is `{ entities, connections }`. The entity cache is disposable and
+always goes; a connection goes only if named, and keeps its edges alone — never
+a cursor, which may name server state that is gone. Session machinery (live
+cursors, optimistic layers, mutation bookkeeping, gaps, retention roots) is
+never in one.
 
 ```ts
-import { emptyStore, RemotePersistence } from 'foldkit-remote'
+import { RemotePersistence } from 'foldkit-remote'
 
 // SSR: prefetch on the server, embed, then hydrate on the client.
-const snapshot = RemotePersistence.dehydrate(
-  Data.storeOf(loaded),
+// `snapshotOf` names the connections that survive; none, here.
+const text = RemotePersistence.dehydrate(
+  RemotePersistence.snapshotOf(loaded.remote),
   { scope: userId },
 )
 
+const restored =
+  RemotePersistence.hydrate(text, { scope: userId }) ?? RemotePersistence.emptySnapshot
+
 Data.reduce(model, {
   _tag: 'Hydrated',
-  entities:
-    RemotePersistence.hydrate(snapshot, { scope: userId }) ?? emptyStore,
+  entities: restored.entities,
+  connections: restored.connections,
   merge: 'preserve-existing',
 })
 
 // Or persist through Effect's KeyValueStore.
-RemotePersistence.save(store, {
+RemotePersistence.save(RemotePersistence.snapshotOf(model.remote), {
   key: 'remote-cache',
   scope: userId,
   maxBytes: 512_000,
@@ -966,6 +1087,22 @@ See [`foldkit-remote-server`](../remote-server) for Source and authorization
 rules, and [`examples/kitchen-sink`](../../examples/kitchen-sink) for the real
 server packages together.
 
+## Finding the next API
+
+| Task | Start here |
+| --- | --- |
+| Explain `Initial`, `Loading`, or a stale value | [`RemoteData`](#remotedata-what-does-the-model-know-right-now) |
+| Fetch outside an active screen | [Policies and prefetch](#reading-policies-and-prefetch) |
+| Load an ordered list | [Queries and pagination](#queries-and-pagination) |
+| Save and show an optimistic result | [Mutations](#mutations-and-optimistic-state) |
+| Release data no active feature needs | [Retention](#retention-and-garbage-collection) |
+| Seed a cache after reload or SSR | [Hydration](#persistence-and-hydration) |
+
+A successful mutation and an optimistic overlay have different guarantees:
+the overlay is only a temporary view of server-owned data. Use Sync when the
+operation itself must survive offline, rather than persisting an optimistic
+Remote cache and treating it as an outbox.
+
 ## Introspection
 
 Remote exposes its decisions as data rather than requiring DevTools to reach
@@ -974,6 +1111,7 @@ into private state:
 ```ts
 Data.inspect(model)            // serializable cache/domain summary
 Data.plan(model, projection)   // the entity plan a read would execute
+Data.explain(model, query)     // what one query read is, and currently is
 Remote.planQueries(...)        // missing/stale query work
 Remote.inspectEntity(...)      // one normalized entity
 ```
@@ -992,6 +1130,53 @@ asked. That is deliberate. Runtime activity is not a second source of truth
 here, and a view that rendered from it would stop being reproducible from the
 Model.
 
+### Explaining one query read
+
+A query read is assembled from more pieces than it looks like: a definition, an
+input, a connection identity that excludes the window, a window that does not,
+and a Selection that is a slice asked of the server. `Data.explain` puts them in
+one serializable value, beside what the read answers from this Model right now:
+
+```ts
+Data.explain(model, projects)
+// {
+//   domain: 'remote',
+//   query: 'ProjectsByOwner',
+//   input: { ownerId: 'u1' },
+//   identity: 'ProjectsByOwner\u0000{"ownerId":"u1"}',
+//   window: { first: 25 },
+//   select: { entity: 'Project', fields: ['id', 'name'] },
+//   body: 'FROM Project\nWHERE Project.ownerId = $ownerId\nORDER BY Project.name DESC',
+//   dependencies: { fields: [...], inputs: ['ownerId'], operations: ['eq'] },
+//   state: 'Ready',
+// }
+```
+
+`body` is the query's meaning as readable text, and it is neither SQL nor
+whatever the backend compiled — an input shows as `$ownerId` rather than a bound
+parameter, and `contains` is named rather than rendered as somebody's `like`. A
+definition made with `Query.make` has no `body`, because its meaning lives on
+the server that answers it.
+
+`state` comes from the projection's own read, so an explanation and the view
+cannot disagree about whether the data is there.
+
+A Projection cannot say which Surfaces read it — several may — so pass the
+active record `subscriptions` takes and `explain` reports all of them, plus why
+each is active where that is a readable fact (`Surface.when`) rather than a
+callback (`Surface.at`):
+
+```ts
+Data.explain(model, projects, { surfaces: actives })
+// surfaces:   ['ProjectPage']
+// activation: [{ surface: 'ProjectPage', path: ['route'], tag: 'Project' }]
+```
+
+There is still no **executor**, because what answers a query is a
+`RemoteClient` Layer in the runtime rather than a value in the Model — the same
+purity that lets this be replayed from a recorded Model is what keeps it out of
+reach.
+
 ## Advanced: the kernel
 
 Everything above compiles to the lower-level exports below; there is not a