foldkit-remote 0.2.0 → 0.3.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -1,38 +1,134 @@
1
1
  # `foldkit-remote`
2
2
 
3
- Server data in a Foldkit application, kept in the Model instead of in a cache
4
- beside it. You declare your entities and the fields each screen reads;
5
- `foldkit-remote` works out what is missing, fetches it in one batched request,
6
- stores it normalized so two screens share one copy, and gives each screen its
7
- own slice as a plain value. Reading is pure — a view never starts a fetch.
8
-
9
- There is no hidden mutable cache. Every new fact — a read result, a page, a
10
- mutation result, a live event — arrives as a Message and goes through the same
11
- `update` that moves the rest of the application, so replay, time travel, and
12
- tests see remote data like any other state.
13
-
14
- Reach for it when the server owns the data, several screens read overlapping
15
- slices of it, and you would rather have fetching, caching, pagination,
16
- optimistic updates, and live patches be one mechanism than five. It is not where
17
- edits that must survive the process or the network belong — that is
18
- [`foldkit-durable`](https://github.com/doeixd/foldkit-plus/tree/main/packages/durable)
19
- and [`foldkit-sync`](https://github.com/doeixd/foldkit-plus/tree/main/packages/sync)
20
- — and it is not a transport: the wire is Effect RPC (`RemoteRpc`) and the
21
- application supplies the client and server layers (HTTP, WebSocket, worker,
22
- in-process). The server half is
23
- [`foldkit-remote-server`](https://github.com/doeixd/foldkit-plus/tree/main/packages/remote-server);
24
- [`examples/remote`](https://github.com/doeixd/foldkit-plus/tree/main/examples/remote)
25
- is the worked end-to-end trace and
26
- [Revision Plan §8](https://github.com/doeixd/foldkit-plus/blob/main/docs/design/REVISION_PLAN.md#8-remote)
27
- the design rationale.
28
-
29
- The package has two faces. The **application API** is the bound domain `Data`:
30
- `Data.get`, `Data.live`, `Data.query`, `Data.mutate`, `Data.subscriptions`, and
31
- a handful more, each inferred from the descriptors it is given. The **kernel**
32
- underneath — requirements, the planner, the reducer, the Subscription entries,
33
- the transport seam — stays exported for tooling, SSR, tests, and package
34
- authors, and is documented under [Advanced](#advanced-the-kernel). Ordinary
35
- application code needs only the first.
3
+ Server-owned data, cached **inside the Foldkit Model**.
4
+
5
+ A feature declares the server facts it needs. `foldkit-remote` compares those
6
+ requirements with what the Model already knows, fetches only what is missing or
7
+ stale, normalizes the result so every consumer shares one copy, and returns new
8
+ facts to the application as ordinary Messages.
9
+
10
+ The central rule is:
11
+
12
+ > **A Remote Projection does not fetch. It declares what server-owned facts a
13
+ > consumer requires. I/O happens outside render, and its results reduce back into
14
+ > the Model.**
15
+
16
+ There is no hidden mutable cache beside the application. A read result, query
17
+ page, mutation result, live event, hydration, and retention change all become
18
+ Remote Messages and move the same Model through one pure reducer.
19
+
20
+ Use Remote when the **server owns the truth** and the client needs a normalized,
21
+ disposable view of it. If an edit is client-authored and must survive offline,
22
+ restart, or network failure until it converges, that belongs to
23
+ [`foldkit-sync`](../sync) and [`foldkit-durable`](../durable), not Remote.
24
+
25
+ ## Which state belongs here?
26
+
27
+ A useful rule across Foldkit Plus is **one owner per datum**:
28
+
29
+ | State | Owner |
30
+ | --- | --- |
31
+ | Route, selected item, local form state, transient UI errors | the application Model and `update` |
32
+ | Server-derived facts that can be refetched | `foldkit-remote` |
33
+ | Client-authored state that must survive offline and converge | `foldkit-sync` |
34
+ | Local state represented in the URL or another store | the Model, observed by `foldkit-mirror` |
35
+
36
+ The distinction between Remote and Sync is especially important:
37
+
38
+ ```text
39
+ Remote
40
+ server owns the fact
41
+ cache is disposable
42
+ refetch is recovery
43
+
44
+ Sync
45
+ client owns the edit
46
+ intent must survive offline/restart
47
+ replay + reconciliation is recovery
48
+ ```
49
+
50
+ A Remote mutation is therefore an immediate request against server-owned data.
51
+ Its result updates the cache. It is **not** a durable local intent queue.
52
+
53
+ A Surface may project all of these owners at once. Observation does not transfer
54
+ ownership.
55
+
56
+ ## The mental model
57
+
58
+ The client side is a Foldkit Submodel:
59
+
60
+ ```text
61
+ pure
62
+ Projection read
63
+ |
64
+ v
65
+ +-------------+ +---------+ +------+
66
+ | Foldkit |---->| Surface |----->| View |
67
+ | Model | +---------+ +------+
68
+ | |
69
+ | Remote.Model|
70
+ | normalized |
71
+ | cache |
72
+ +------+------+
73
+ ^
74
+ | Remote Message
75
+ |
76
+ +------+-------+
77
+ | Data.reduce |
78
+ +------+-------+
79
+ ^
80
+ |
81
+ | read/query/mutate/live
82
+ |
83
+ +------+--------+
84
+ | RemoteClient |
85
+ +------+--------+
86
+ |
87
+ v
88
+ server
89
+ ```
90
+
91
+ The other half of the model is the **requirement planner**:
92
+
93
+ ```text
94
+ Surface says:
95
+ "I need Project p1: id, name, owner.name"
96
+
97
+ |
98
+ v
99
+ Remote compares that requirement
100
+ with the normalized cache in Model
101
+
102
+ |
103
+ v
104
+ missing:
105
+ Project:p1.name
106
+ User:u7.name
107
+
108
+ |
109
+ v
110
+ active subscription performs one read
111
+
112
+ |
113
+ v
114
+ result -> Remote Message -> Data.reduce -> Model
115
+ ```
116
+
117
+ That separation is why reading stays pure. Rendering a Surface never starts a
118
+ request.
119
+
120
+ ## Four pieces to remember
121
+
122
+ Most application code only needs four roles:
123
+
124
+ | Piece | Responsibility |
125
+ | --- | --- |
126
+ | `Data.get` / `Data.live` / `Data.query` | create pure Projections that declare what a consumer needs |
127
+ | `Data.subscriptions` | turn the requirements of **active** Surfaces into reads, live subscriptions, and retention roots |
128
+ | `Data.reduce` | reduce returned Remote Messages into the application's `Remote.Model` |
129
+ | `RemoteClient` | perform the actual read/query/mutate/live I/O |
130
+
131
+ Everything else builds on those four pieces.
36
132
 
37
133
  ## Install
38
134
 
@@ -40,225 +136,286 @@ application code needs only the first.
40
136
  pnpm add foldkit-remote
41
137
  ```
42
138
 
43
- `foldkit` and `effect` are peer dependencies; `foldkit-surface` comes with it.
44
- The server half is `foldkit-remote-server`.
45
-
46
- ## Quick start
139
+ `foldkit` and `effect` are peer dependencies; `foldkit-surface` comes with the
140
+ package. The server-side interpreter is [`foldkit-remote-server`](../remote-server).
47
141
 
48
- One feature, end to end: a project page that reads a project live, lists the
49
- owner's projects, and renames a project.
142
+ ## Sixty seconds: one entity, one Surface
50
143
 
51
- ### 1. Declare the domain
144
+ Start with one server entity and one selection:
52
145
 
53
146
  ```ts
54
147
  import { Schema } from 'effect'
55
- import { Entity, Mutation, Query } from 'foldkit-remote'
148
+ import { Entity } from 'foldkit-remote'
56
149
 
57
- const User = Entity.make('User', Schema.Struct({ id: Schema.String, name: Schema.String }))
58
150
  const Project = Entity.make(
59
151
  'Project',
60
152
  Schema.Struct({
61
153
  id: Schema.String,
62
154
  name: Schema.String,
63
155
  status: Schema.String,
64
- owner: Entity.ref(User),
65
156
  }),
66
157
  )
67
158
 
68
- const UserSummary = User.select({ id: true, name: true })
69
- const ProjectSummary = Project.select({ id: true, name: true, status: true, owner: UserSummary })
70
-
71
- const ProjectsByOwner = Query.make('ProjectsByOwner', {
72
- Input: { ownerId: Schema.String },
73
- Result: Project,
74
- })
75
- const RenameProject = Mutation.make('RenameProject', {
76
- Input: { id: Schema.String, name: Schema.String },
77
- Output: { id: Schema.String },
159
+ // A Selection is the exact server-owned shape this consumer needs.
160
+ const ProjectSummary = Project.select({
161
+ id: true,
162
+ name: true,
78
163
  })
79
164
  ```
80
165
 
81
- An entity is its fields; a relation is a reference codec (`Entity.ref(User)`),
82
- never an inline target schema, so a recursive relation (`Node.parent:
83
- Entity.refTo('Node')`) stays finite. `Project.select` picks fields and reads
84
- **through** a relation into its target (`owner: UserSummary`); an unknown field
85
- or a nested selection of the wrong entity is a compile error. `Input`,
86
- `Output`, and `Result` take a codec or the fields of the `Schema.Struct` it
87
- would be; `Result: Project` is a connection over `Project`.
88
-
89
- ### 2. Embed the submodel and bind the domain
166
+ Embed Remote's Submodel in the application and bind the domain to that field:
90
167
 
91
168
  ```ts
92
169
  import { defineMessageUnion } from 'foldkit/message'
93
170
  import type * as Update from 'foldkit/update'
94
- import { Remote } from 'foldkit-remote'
171
+ import { Remote, type RemoteClient } from 'foldkit-remote'
95
172
  import { Surface } from 'foldkit-surface'
96
173
 
97
- // Your own routes; this one has a project route carrying the id the page reads.
98
174
  const Route = Schema.Union([
99
175
  Schema.Struct({ _tag: Schema.Literal('home') }),
100
- Schema.Struct({ _tag: Schema.Literal('project'), projectId: Schema.String }),
176
+ Schema.Struct({
177
+ _tag: Schema.Literal('project'),
178
+ projectId: Schema.String,
179
+ }),
101
180
  ])
102
181
 
103
- const Model = Schema.Struct({ route: Route, remote: Remote.Model })
182
+ const Model = Schema.Struct({
183
+ route: Route,
184
+ remote: Remote.Model,
185
+ })
104
186
  type Model = typeof Model.Type
187
+
105
188
  const Message = defineMessageUnion({
106
189
  ...Remote.messages,
107
- ClickedRename: { id: Schema.String, name: Schema.String },
108
- ClickedMore: {},
109
190
  })
110
191
  type Message = typeof Message.Type
111
192
 
193
+ function update(
194
+ model: Model,
195
+ message: Message,
196
+ ): Update.Return<Model, Message, RemoteClient> {
197
+ // Remote owns the `remote` Submodel, so all Remote Messages go through its reducer.
198
+ if (Remote.reduces(message)) return { model: Data.reduce(model, message) }
199
+ return { model }
200
+ }
201
+
112
202
  const App = Surface.application({
113
203
  Model,
114
204
  Message,
115
- initial: { route: { _tag: 'home' }, remote: Remote.initial },
116
- update, // step 5: a function declaration, so it may follow Data
205
+ initial: {
206
+ route: { _tag: 'home' },
207
+ remote: Remote.initial,
208
+ },
209
+ update,
117
210
  })
118
211
 
119
212
  const Data = Remote.make({
120
- model: App.model.remote,
121
- entities: [User, Project],
122
- queries: [ProjectsByOwner],
123
- mutations: [RenameProject],
213
+ model: App.fields.remote,
214
+ entities: [Project],
124
215
  })
125
216
  ```
126
217
 
127
- `Remote.Model` is the submodel's schema, the same for every domain, so the
128
- Model embeds it before the domain is bound. Remote's Messages are cases of the
129
- application's own union (`Remote.messages`). `Data` is the bound domain: it
130
- knows the Model, the place of the store in it, and the entities, queries, and
131
- mutations it may be asked about — anything else is a compile error naming the
132
- descriptor (`Entity "Team" is not registered with this Remote domain`), and a
133
- runtime error naming the domain.
218
+ `Remote.Model` is not a second application store. It is a Submodel **inside** the
219
+ application Model. `Data` is the bound API for this domain: it knows where that
220
+ Submodel lives and which descriptors this application registered.
134
221
 
135
- ### 3. Read in a Surface
222
+ Now declare what a feature needs:
136
223
 
137
224
  ```ts
138
- const projects = Data.query(ProjectsByOwner, { ownerId: 'u1' }, { select: ProjectSummary, first: 25 })
139
-
140
225
  const ProjectPage = App.surface('ProjectPage', {
141
226
  params: { projectId: Schema.String },
142
- model: ({ params }) => ({ project: Data.live(ProjectSummary, params.projectId), projects }),
143
- messages: [Message.ClickedRename, Message.ClickedMore],
227
+
228
+ model: ({ params }) => ({
229
+ // Important: this performs no I/O.
230
+ // It is a pure Projection over Model plus a requirement for these fields.
231
+ project: Data.get(ProjectSummary, params.projectId),
232
+ }),
144
233
  })
145
234
  ```
146
235
 
147
- The Surface's Model is `{ project: RemoteData<ProjectSummary>; projects:
148
- RemoteData<Page<ProjectSummary>> }`, read purely from the Model, no I/O. Every
149
- remote value is a `RemoteData`:
150
-
151
- - `Initial` — a selected field is not present, and **nothing is fetching it**,
152
- - `Loading` — not present, and a read is in flight,
153
- - `Ready` — present,
154
- - `Refreshing` — present, and being refetched (the value stays visible),
155
- - `Failed` — the stored value did not decode against the selection,
156
- - `NotFound` — the entity is a tombstone.
236
+ `ProjectPage` reads a `RemoteData<{ id: string; name: string }>` from the Model.
237
+ If those fields are absent, the Projection still does not fetch them.
157
238
 
158
- `Loading` is the absent-value twin of `Refreshing`: both mean a read is in
159
- flight. The distinction from `Initial` is worth rendering differently, because
160
- nothing fetches a projection no active Surface observes. Such a projection reads
161
- `Initial` forever, and showing a spinner for it hides the wiring mistake, while
162
- `Loading` is the state where waiting is the right thing to do.
163
-
164
- `RemoteData.match` is exhaustive — omitting a case is a compile error. `Data.get` reads once and refreshes by policy; `Data.live` also
165
- follows the entity's changes; `Data.query` reads a connection as a `Page` of
166
- selected items, `Initial` until the page and every item's fields are present.
167
-
168
- ### 4. Fetch, subscribe, and retain from the active Surfaces
239
+ The active Surface drives I/O through a Foldkit Subscription:
169
240
 
170
241
  ```ts
171
242
  import * as Subscription from 'foldkit/subscription'
172
- import { RemoteClient, RemotePolicy } from 'foldkit-remote'
173
243
 
174
244
  const subscriptions = Subscription.make<Model, Message, RemoteClient>()(() =>
175
- Data.subscriptions(
176
- {
177
- page: Surface.at(ProjectPage, model =>
178
- model.route._tag === 'project' ? { projectId: model.route.projectId } : undefined,
179
- ),
180
- },
181
- { policy: RemotePolicy.staleWhileRevalidate({ maxAge: 30_000 }), grace: '5 seconds' },
182
- ),
245
+ Data.subscriptions({
246
+ project: Surface.at(ProjectPage, model =>
247
+ model.route._tag === 'project'
248
+ ? { projectId: model.route.projectId }
249
+ : undefined,
250
+ ),
251
+ }),
183
252
  )
184
253
  ```
185
254
 
186
- Activation is a fact of the Model: `Surface.at` gives the Surface its params as
187
- a function of the Model, `undefined` while it is inactive. From the active
188
- Surfaces `Data.subscriptions` derives what to fetch (only the fields and pages
189
- the Model lacks), what to subscribe to (what is read through `Data.live`), and
190
- what to keep (everything the active Surfaces reach); a fully-known Surface
191
- fetches nothing, and there is never network work during render.
255
+ `Surface.at` makes activation a fact of the Model. When the route activates the
256
+ page, Remote sees its requirements, diffs them against the cache, and fetches
257
+ only what is missing. When the page is inactive, it creates no read work.
258
+
259
+ The complete loop is:
260
+
261
+ ```text
262
+ ProjectPage Projection
263
+ |
264
+ | requirements
265
+ v
266
+ Data.subscriptions
267
+ |
268
+ | plan missing fields
269
+ v
270
+ RemoteClient
271
+ |
272
+ | server result
273
+ v
274
+ Remote Message
275
+ |
276
+ v
277
+ Data.reduce
278
+ |
279
+ v
280
+ Remote.Model
281
+ |
282
+ v
283
+ ProjectPage now reads Ready(...)
284
+ ```
192
285
 
193
- ### 5. Mutate and page from `update`
286
+ Finally, provide the client implementation to the runtime:
194
287
 
195
288
  ```ts
196
- function update(model: Model, message: Message): Update.Return<Model, Message, RemoteClient> {
197
- if (Remote.reduces(message)) return { model: Data.reduce(model, message) }
198
- switch (message._tag) {
199
- case 'ClickedRename': {
200
- const { id, name } = message
201
- const { model: started, command } = Data.mutate(model, RenameProject, { id, name }, {
202
- optimistic: [Project.patch(id, { name })],
203
- })
204
- return { model: started, commands: [command] }
205
- }
206
- case 'ClickedMore': {
207
- const next = Data.next(model, projects)
208
- return { model, commands: next === undefined ? [] : [Data.fetch(next)] }
209
- }
210
- }
211
- }
289
+ const clientLayer = Remote.clientLayer(rpcClient)
212
290
  ```
213
291
 
214
- Every new fact — a read batch, a page, a mutation result, a live event — arrives
215
- as one of Remote's Messages, and `Data.reduce` is the one reducer for all of
216
- them. The Commands run through `RemoteClient`, so `update` names it as the
217
- resource its Commands need (`Update.Return<Model, Message, RemoteClient>`) and
218
- the runtime is given the client layer (step 6). `Data.mutate` starts the request in the Model (its id comes from the
219
- Model's own sequence, so `update` stays pure) and returns the Command whose
220
- Message settles it; the optimistic patch shows until then. `Data.next` is the
221
- `QueryRef` of the page after the loaded end, or `undefined`, and `Data.fetch`
222
- the Command that merges it, after which the same `projects` projection reads
223
- every loaded page.
292
+ `Remote.clientLayer` adapts an Effect RPC client for `RemoteRpc` into the
293
+ `RemoteClient` service used by subscriptions and Commands. The transport itself
294
+ is not owned by Remote.
224
295
 
225
- ### 6. Provide the client
296
+ ## `RemoteData`: what does the Model know right now?
297
+
298
+ A Remote Projection never lies by pretending an absent value is present. It
299
+ returns a closed `RemoteData` union:
300
+
301
+ ```text
302
+ no active fetch
303
+ missing ------------------------------------------------> Initial
304
+
305
+ fetch starts
306
+ missing ------------------------------------------------> Loading
307
+ |
308
+ result
309
+ v
310
+ Ready
311
+ |
312
+ refetch starts
313
+ v
314
+ Refreshing
315
+ old value stays
316
+ |
317
+ result
318
+ v
319
+ Ready
320
+
321
+ server says entity is absent ---------------------------> NotFound
322
+ stored value fails Selection decoding -----------------> Failed
323
+ ```
324
+
325
+ The states are:
326
+
327
+ - **`Initial`** — required data is absent and **nothing is currently fetching it**.
328
+ - **`Loading`** — required data is absent and a read is in flight.
329
+ - **`Ready`** — every selected field is present and decodes.
330
+ - **`Refreshing`** — the current value remains visible while it is being refetched.
331
+ - **`Failed`** — stored server data does not decode against the Selection.
332
+ - **`NotFound`** — the entity is represented by a tombstone.
333
+
334
+ `Initial` is intentionally different from `Loading`. A Projection belonging to
335
+ no active Surface may remain `Initial` forever. Rendering a spinner for
336
+ `Initial` can therefore hide an activation/wiring mistake; `Loading` is the
337
+ state that actually means "wait for this request."
338
+
339
+ `RemoteData.match` is exhaustive, so adding or omitting a state is visible at
340
+ compile time.
341
+
342
+ ## Normalized entities and selections
343
+
344
+ Remote stores an entity once by identity, regardless of how many Surfaces read
345
+ it. Consumers select different views of the same normalized fact.
346
+
347
+ Relations are references, not nested copies:
226
348
 
227
349
  ```ts
228
- const clientLayer = Remote.clientLayer(rpcClient) // an Effect RPC client for RemoteRpc
229
- // … provide it to the Foldkit runtime; in tests or in-process, RemoteServer.handlers(…) is one.
350
+ const User = Entity.make(
351
+ 'User',
352
+ Schema.Struct({
353
+ id: Schema.String,
354
+ name: Schema.String,
355
+ }),
356
+ )
357
+
358
+ const Project = Entity.make(
359
+ 'Project',
360
+ Schema.Struct({
361
+ id: Schema.String,
362
+ name: Schema.String,
363
+ owner: Entity.ref(User),
364
+ }),
365
+ )
366
+
367
+ const UserSummary = User.select({ id: true, name: true })
368
+ const ProjectSummary = Project.select({
369
+ id: true,
370
+ name: true,
371
+ owner: UserSummary,
372
+ })
373
+ ```
374
+
375
+ The store remains normalized:
376
+
377
+ ```text
378
+ Project:p1 name PRESENT owner PRESENT -> User:u7
379
+ User:u7 name PRESENT
230
380
  ```
231
381
 
232
- `Remote.clientLayer` adapts an Effect RPC client for `RemoteRpc` to the
233
- `RemoteClient` service the subscriptions and Commands run through, and coalesces
234
- its reads and queries. The server side is
235
- [`foldkit-remote-server`](https://github.com/doeixd/foldkit-plus/tree/main/packages/remote-server) with sources compiled by
236
- [`foldkit-remote-drizzle`](https://github.com/doeixd/foldkit-plus/tree/main/packages/remote-drizzle) or written by hand.
237
-
238
- ## Reading
239
-
240
- `Data.get(selection, id)` and `Data.live(selection, id)` return
241
- `Projection<AppModel, RemoteData<Value>>`, the value typed from the selection.
242
- `Data.live` marks the projection's requirements `live`; the mark survives
243
- `Projection.struct` and never reaches the wire, and `Data.subscriptions`
244
- subscribes what is marked. What a field the Model already holds means is a
245
- `RemotePolicy`, an option of `Data.subscriptions` and `Data.prefetch`:
246
-
247
- - `RemotePolicy.cacheFirst` (default) — fetch only missing, stale, or
248
- re-windowed fields.
249
- - `RemotePolicy.staleWhileRevalidate({ maxAge })` — keep present values
250
- visible and refetch an entry older than `maxAge` milliseconds.
251
- - `RemotePolicy.networkOnly` — fetch every selected field; cached values stay
252
- visible meanwhile.
253
-
254
- A refreshing policy marks the refetched fields stale first, so the projection
255
- reads `Refreshing` with the old value until the read lands. The clock the policy
256
- reads is the `now` option (default `Date.now`), so tests inject time.
257
-
258
- `Data.prefetch(model, projection, options?)` runs the same plan through
259
- `RemoteClient` (the projection's pending queries first, then one read) and
260
- returns the Model with the results reduced in, for SSR, route or hover
261
- prefetch, and tests. It never runs during render.
382
+ The Projection assembles the nested consumer value by following the reference.
383
+ An unknown field or a nested Selection for the wrong entity is a type error.
384
+ Recursive relations remain finite because the entity schema stores references
385
+ (`Entity.ref(User)` / `Entity.refTo('Node')`), not recursively inlined schemas.
386
+
387
+ Presence is tracked separately from the JavaScript value. These are distinct:
388
+
389
+ ```text
390
+ field missing
391
+ field present with undefined
392
+ field present with null
393
+ field stale
394
+ entity not found
395
+ ```
396
+
397
+ Remote never infers presence from `value === undefined`.
398
+
399
+ ## Reading, policies, and prefetch
400
+
401
+ `Data.get(selection, id)` returns a pure Projection. `Data.live(selection, id)`
402
+ returns the same kind of Projection but marks its requirements as live so
403
+ `Data.subscriptions` also follows server changes.
404
+
405
+ What Remote should do when the Model already contains the selected fields is a
406
+ `RemotePolicy`:
407
+
408
+ - `RemotePolicy.cacheFirst` (default) — fetch missing, stale, or re-windowed data.
409
+ - `RemotePolicy.staleWhileRevalidate({ maxAge })` — keep a present value visible
410
+ and refetch when it is older than `maxAge`.
411
+ - `RemotePolicy.networkOnly` — request every selected field; cached values stay
412
+ visible while the request is in flight.
413
+
414
+ A refreshing policy emits `RefreshStarted` before the read, so the Projection
415
+ becomes `Refreshing` without discarding the old value. Planning accepts `now`
416
+ (default `Date.now`) as an input, so tests can control time.
417
+
418
+ For SSR, route prefetch, hover prefetch, and tests, run the same plan explicitly:
262
419
 
263
420
  ```ts
264
421
  import { Effect } from 'effect'
@@ -270,344 +427,522 @@ const loaded = await Effect.runPromise(
270
427
  )
271
428
  ```
272
429
 
273
- ## Queries and pages
430
+ `Data.prefetch` performs I/O explicitly and returns the Model with the resulting
431
+ Remote Messages reduced into it. It never changes the semantics of the
432
+ Projection itself.
433
+
434
+ ### Refreshing from `update`
435
+
436
+ To revalidate what a screen already declares — a refresh button, a focus
437
+ regained — hand its Projection (or a Surface without params) to `Data.refresh`:
274
438
 
275
439
  ```ts
276
- const projects = Data.query(ProjectsByOwner, { ownerId }, { select: ProjectSummary, first: 25 })
277
- projects.read(model) // RemoteData<Page<{ id; name; status; owner: { name } }>>
278
- Data.next(model, projects) // QueryRef | undefined, from the loaded end
279
- Data.previous(model, projects) // …from the loaded start
280
- Data.fetch(ref) // Command yielding the ConnectionMerged (or QueryFailed) that reduces it
440
+ case 'ClickedRefresh': {
441
+ if (model.route._tag !== 'project') return { model }
442
+ const page = ProjectPage.projection({ projectId: model.route.projectId })
443
+ return { model: Data.refresh(model, page) }
444
+ }
281
445
  ```
282
446
 
283
- `select` is a selection of the query's entity (another entity is an error
284
- naming both); the window is `first`/`after` or `last`/`before`, never a mix,
285
- a page size is a non-negative integer (anything else is an error at the call,
286
- and refused by the wire), and no window asks for the server's default page. The projection is `Initial`
287
- until the page and every item's selected fields are present — never a partial
288
- page — `Ready` once they are, `Refreshing` while the connection or any item is
289
- being refetched, and `Failed` if an item does not decode. An optimistic insert
290
- into a page should therefore carry every field the page selects, or the page
291
- reads `Initial` until the read entry fetches the rest.
292
-
293
- The projection carries its connection, so a Surface that reads a page needs
294
- nothing more: the read entry runs the query when the Model does not hold the
295
- connection (or holds it stale), and once the page is in the Model its items
296
- are planned and read like any other field. A failed query yields
297
- `QueryFailed`, which ends the refresh and keeps the pages.
298
- `Data.next` and `Data.previous` keep the page size and are `undefined` at a
299
- terminal or unknown boundary; the merged pages read through the same
300
- projection, with `hasNext`/`hasPrevious` derived from the boundaries, never
301
- from row counts. Two projections of one connection plan one query, selecting
302
- the union of their fields.
303
-
304
- ## Mutations
447
+ `Data.refresh` performs no I/O and restates no request. It returns the Model
448
+ with every selected field the store holds reading `Refreshing` and every loaded
449
+ connection invalidated; the `Data.subscriptions` read entries then refetch it,
450
+ since stale data is planned again under every policy, so the page is requested
451
+ once. The Projection must be observed, as it is while it is on screen; for data
452
+ nothing observes, use `Data.prefetch` with `RemotePolicy.networkOnly`.
453
+
454
+ - A refreshed connection's first page replaces its loaded pages, so items the
455
+ server removed or reordered follow it; later pages are fetched again with
456
+ `Data.next`.
457
+ - A read already in flight restarts instead of landing after the refresh.
458
+ - Refreshing what is already refreshing, or nothing, returns the same Model.
459
+
460
+ ## Queries and pagination
461
+
462
+ Entities answer "which fields of this known thing?" A Query answers "which
463
+ entities belong in this server-owned list?"
464
+
465
+ Declare the query and register it with the domain:
466
+
467
+ ```ts
468
+ import { Query } from 'foldkit-remote'
469
+
470
+ const ProjectsByOwner = Query.make('ProjectsByOwner', {
471
+ Input: { ownerId: Schema.String },
472
+ Result: Project,
473
+ })
474
+
475
+ const Data = Remote.make({
476
+ model: App.fields.remote,
477
+ entities: [User, Project],
478
+ queries: [ProjectsByOwner],
479
+ })
480
+ ```
481
+
482
+ Then a query is still just a Projection:
483
+
484
+ ```ts
485
+ const projects = Data.query(
486
+ ProjectsByOwner,
487
+ { ownerId },
488
+ { select: ProjectSummary, first: 25 },
489
+ )
490
+
491
+ projects.read(model)
492
+ // RemoteData<Page<ProjectSummary>>
493
+
494
+ Data.next(model, projects) // QueryRef | undefined
495
+ Data.previous(model, projects) // QueryRef | undefined
496
+ Data.fetch(ref) // Command whose Message merges the page
497
+ ```
498
+
499
+ The connection stores entity references and explicit boundaries rather than one
500
+ flat array. That matters when page 1 and page 3 are loaded but page 2 is not:
501
+ Remote represents an honest gap instead of pretending the visible rows are
502
+ adjacent.
503
+
504
+ A query Projection is `Initial` until the page **and every selected field of its
505
+ visible items** are present. Once a page lands, those items become ordinary
506
+ entity requirements and can be fulfilled in the same planning loop.
507
+
508
+ Two Projections of the same connection plan one query and select the union of
509
+ their required fields. `Data.next` / `Data.previous` preserve the page size and
510
+ use the loaded boundaries; `hasNext` / `hasPrevious` come from those boundaries,
511
+ not from guessing based on row counts.
512
+
513
+ ## Mutations and optimistic state
514
+
515
+ Register mutations on the domain:
516
+
517
+ ```ts
518
+ import { Mutation } from 'foldkit-remote'
519
+
520
+ const RenameProject = Mutation.make('RenameProject', {
521
+ Input: { id: Schema.String, name: Schema.String },
522
+ Output: { id: Schema.String },
523
+ })
524
+ ```
525
+
526
+ A mutation starts from ordinary application `update`:
527
+
528
+ ```ts
529
+ case 'ClickedRename': {
530
+ const { id, name } = message
531
+
532
+ const { model: started, command } = Data.mutate(
533
+ model,
534
+ RenameProject,
535
+ { id, name },
536
+ {
537
+ optimistic: [Project.patch(id, { name })],
538
+ },
539
+ )
540
+
541
+ return { model: started, commands: [command] }
542
+ }
543
+ ```
544
+
545
+ `Data.mutate` does two things:
546
+
547
+ 1. it applies `MutationStarted` to the Model, including any optimistic overlays;
548
+ 2. it returns the Command that calls `RemoteClient` and eventually emits
549
+ `MutationSucceeded` or `MutationFailed`.
550
+
551
+ Those settlement Messages go through `Data.reduce` like every other Remote fact.
552
+ The request id comes from the Remote Model's sequence, so `update` stays pure.
553
+ Settling is idempotent per `requestId`.
554
+
555
+ Optimistic entity patches are **layers over the base store**, not inverse
556
+ patches. If multiple mutations overlap, the visible cache is recomputed as base
557
+ plus the still-pending layers, so settling one does not require trying to undo
558
+ its old value manually.
559
+
560
+ Connections can be optimistic too:
305
561
 
306
562
  ```ts
307
563
  import { ConnectionChange } from 'foldkit-remote'
308
564
 
309
- const { model: started, requestId, tempId, command } = Data.mutate(model, AddComment, input, {
565
+ const { model: started, command } = Data.mutate(model, AddComment, input, {
310
566
  optimistic: ({ tempId }) => [
311
567
  Comment.patch(tempId, { id: tempId, body: input.body }),
312
- ConnectionChange.prepend(CommentsForPost.ref({ postId }), Comment.ref(tempId)),
568
+ ConnectionChange.prepend(
569
+ CommentsForPost.ref({ postId }),
570
+ Comment.ref(tempId),
571
+ ),
313
572
  ],
314
573
  })
315
- return { model: started, commands: [command] }
316
574
  ```
317
575
 
318
- `Data.mutate` accepts only the domain's mutations and the mutation's own
319
- `Input`. It applies `MutationStarted` (with the optimistic operations) to the
320
- Model and returns the Command whose Message (`MutationSucceeded` or
321
- `MutationFailed`) settles it through `Data.reduce`; the Command never fails.
322
- The request id is `<domain>-<n>` from the Model's mutation sequence; `{
323
- requestId }` overrides it for a retry or a durable bridge, and `optimistic` may
324
- be a function of `{ requestId, tempId }` so a created entity carries `tempId`
325
- until the result names the real one.
326
-
327
- A mutation owns its optimistic operations: entity patches (`Project.patch(id,
328
- values)`) and connection changes (`ConnectionChange.prepend`/`append`/`remove`,
329
- by `QueryRef` or identity), applied together when it starts and released
330
- together when it settles. Patches are ordered layers over the base store, not
331
- inverse patches, so overlapping layers rebase for free; connection changes are
332
- overlays outside the server-known region. `MutationSucceeded` writes the
333
- server's patches and puts the result's confirmed `connections` where the
334
- request's overlays were, so a temporary edge becomes the real one without a
335
- flicker; `MutationFailed` releases both, revealing the base. Settling is
336
- idempotent per `requestId`, so a transport retry cannot apply the same change
337
- twice.
338
-
339
- A Remote mutation is an immediate, server-derived command: it runs now, against
340
- the server that owns the data, and its result is cache. It is not durable
341
- intent. An edit that must survive the process or the network belongs to
342
- `foldkit-sync` and `foldkit-durable`, which own an ordered log and its
343
- idempotency; Remote adds no second queue.
576
+ A successful server result can replace the request's temporary connection
577
+ overlays with confirmed ones in place.
578
+
579
+ ### Remote mutation vs Sync operation
580
+
581
+ Do not use Remote mutation as a durable offline-write mechanism:
582
+
583
+ ```text
584
+ Remote mutation
585
+ request server now
586
+ server owns truth
587
+ result updates disposable cache
588
+ no durable outbox
589
+
590
+ Sync operation
591
+ client authored the durable fact/intent
592
+ survives offline/restart
593
+ enters an ordered log
594
+ converges with other replicas
595
+ ```
596
+
597
+ If losing an unsent edit would be data loss, it belongs to Sync rather than
598
+ Remote.
344
599
 
345
600
  ## Live data
346
601
 
347
- `Data.live` subscribes the Surfaces that read it to the entity's changes, from
348
- the cursor the Model keeps (`RemoteModel.live`), so the application tracks no
349
- cursor of its own. Events carry a monotonic cursor per stream: duplicates are
350
- ignored, and an event ahead of the cursor is a gap — it is not applied, and the
351
- stream is recorded in `RemoteModel.gaps` so the host can resync rather than
352
- silently miss facts. The gap clears when an in-order event applies, or on a
353
- `GapCleared` message. `EntityPatched` updates the store, `EntityDeleted` writes
354
- a tombstone, and `ConnectionInsert`/`ConnectionRemove`/`ConnectionInvalidate`
355
- change connection membership and ordering. A stream break arrives as a
356
- `ReadFailed` Message like any other failure.
602
+ `Data.live` marks a Projection so active `Data.subscriptions` follow server
603
+ changes for it.
604
+
605
+ The Remote Model owns each stream's cursor. Live events carry monotonically
606
+ ordered cursors:
607
+
608
+ - an old/repeated cursor is a duplicate and is ignored;
609
+ - the next cursor is applied;
610
+ - a cursor ahead of the expected value is a **gap** and is not applied.
611
+
612
+ A gap is recorded in `RemoteModel.gaps` so the host can resynchronize rather than
613
+ silently accepting missing history. An in-order event or `GapCleared` clears it.
614
+
615
+ Entity patches update normalized fields, deletes create tombstones, and
616
+ connection insert/remove/invalidate events reconcile the same connection model
617
+ used by query pages and optimistic overlays. A stream failure becomes a Remote
618
+ failure Message rather than mutating anything out of band.
619
+
620
+ ## Retention and garbage collection
621
+
622
+ Remote is a cache, so it should be allowed to forget facts no active feature
623
+ needs.
624
+
625
+ `Data.subscriptions` derives retention roots from the active Surfaces. A root
626
+ keeps:
627
+
628
+ - the entities and fields its Projection requires;
629
+ - referenced targets reached by nested selections;
630
+ - retained connections and the selected fields of their visible items;
631
+ - data touched by pending optimistic work.
632
+
633
+ After the configured grace period, `RetentionChanged` enters the Model and the
634
+ pure Remote reducer collects unreachable cache data. An inactive Surface that is
635
+ not otherwise retained may therefore lose its cache, which is safe because the
636
+ server remains authoritative and the planner can refetch it.
637
+
638
+ This is another important difference from Sync: **Remote recovery may discard
639
+ and refetch; Sync recovery must preserve unsent client intent.**
357
640
 
358
641
  ## Persistence and hydration
359
642
 
360
- A snapshot is the entity store and nothing else: connections' live cursors,
361
- optimistic layers, the mutation ledger, gaps, and retention roots belong to the
362
- session and never appear in one.
643
+ Persist only the disposable entity cache, not session machinery such as live
644
+ cursors, optimistic layers, mutation bookkeeping, gaps, or retention roots.
363
645
 
364
646
  ```ts
365
647
  import { emptyStore, RemotePersistence } from 'foldkit-remote'
366
648
 
367
- // SSR: the server prefetches, dehydrates into the page, the client hydrates.
368
- const snapshot = RemotePersistence.dehydrate(Data.storeOf(loaded), { scope: userId })
649
+ // SSR: prefetch on the server, embed, then hydrate on the client.
650
+ const snapshot = RemotePersistence.dehydrate(
651
+ Data.storeOf(loaded),
652
+ { scope: userId },
653
+ )
654
+
369
655
  Data.reduce(model, {
370
656
  _tag: 'Hydrated',
371
- entities: RemotePersistence.hydrate(snapshot, { scope: userId }) ?? emptyStore,
657
+ entities:
658
+ RemotePersistence.hydrate(snapshot, { scope: userId }) ?? emptyStore,
372
659
  merge: 'preserve-existing',
373
660
  })
374
661
 
375
- // A browser cache, through Effect's KeyValueStore:
376
- RemotePersistence.save(store, { key: 'remote-cache', scope: userId, maxBytes: 512_000 })
377
- RemotePersistence.restore({ key: 'remote-cache', scope: userId, maxBytes: 512_000 })
662
+ // Or persist through Effect's KeyValueStore.
663
+ RemotePersistence.save(store, {
664
+ key: 'remote-cache',
665
+ scope: userId,
666
+ maxBytes: 512_000,
667
+ })
668
+
669
+ RemotePersistence.restore({
670
+ key: 'remote-cache',
671
+ scope: userId,
672
+ maxBytes: 512_000,
673
+ })
674
+ ```
675
+
676
+ Snapshots are deterministic, versioned, scoped, and optionally size-bounded. A
677
+ snapshot from another version/scope, an oversized snapshot, or malformed data is
678
+ discarded and the planner refetches. `Hydrated` is itself a Remote Message, so
679
+ hydration still changes the application through the reducer.
680
+
681
+ ## How the server packages fit
682
+
683
+ `foldkit-remote` owns **client cache semantics**, not your source database or
684
+ network transport.
685
+
686
+ ```text
687
+ client
688
+
689
+ Surface Projection
690
+ |
691
+ v
692
+ foldkit-remote
693
+ requirements + normalized cache
694
+ |
695
+ v
696
+ RemoteClient / Effect RPC
697
+
698
+ ---------------- server boundary ----------------
699
+
700
+ foldkit-remote-server
701
+ Sources + field authorization
702
+ |
703
+ +---------------------+
704
+ | |
705
+ v v
706
+ foldkit-remote-drizzle hand-written Source
707
+ (optional SQL compiler) API / service / database
378
708
  ```
379
709
 
380
- `dehydrate` and `hydrate` are the string forms; `save` and `restore` put them
381
- behind `KeyValueStore`, so the backend (memory, filesystem, Web Storage, SQL) is
382
- the application's choice. The text is deterministic (equal stores give
383
- byte-equal snapshots), a `scope` (a user, a tenant, a build) keeps one reader's
384
- cache from another, and `maxBytes` bounds what is written and read. The cache is
385
- server-derived and disposable: a snapshot that is oversized, another version,
386
- another scope, or malformed is discarded (and its key removed) and the planner
387
- refetches. `Hydrated` is a Message like every other cache change; `replace`
388
- takes the snapshot's entries and `preserve-existing` keeps entries the store
389
- already holds, which are at least as fresh. Hydrating the same snapshot twice is
390
- the same as once.
710
+ The package roles are:
711
+
712
+ | Package | Responsibility |
713
+ | --- | --- |
714
+ | `foldkit-remote` | normalized cache, requirements, planning, queries, optimistic overlays, live cursors, retention |
715
+ | `foldkit-remote-server` | interpret client selections/queries against server Sources and enforce semantic-field authorization |
716
+ | `foldkit-remote-drizzle` | optionally compile those selections and queries into typed Drizzle access |
717
+ | Effect RPC / your layers | transport and deployment |
718
+
719
+ `Remote.clientLayer(rpcClient)` adapts `RemoteRpc` to `RemoteClient` and
720
+ coalesces compatible reads/queries. HTTP, WebSocket, worker, and in-process
721
+ execution are layer choices rather than Remote semantics.
722
+
723
+ See [`foldkit-remote-server`](../remote-server) for Source and authorization
724
+ rules, and [`examples/kitchen-sink`](../../examples/kitchen-sink) for the real
725
+ server packages together.
391
726
 
392
727
  ## Introspection
393
728
 
394
- `Data.inspect(model)` returns a serializable summary of the cache — entities
395
- with their present/stale fields, connection keys, live streams, gaps, and the
396
- mutation ledger — and `Remote.inspectEntity(remote, key)` returns one entity of
397
- the store slice. Both are pure, so DevTools never reach into the private layout.
398
- `Data.plan(model, projection)` is the plan the read entry would run, and
399
- `Remote.planQueries` the queries; both are plain data.
729
+ Remote exposes its decisions as data rather than requiring DevTools to reach
730
+ into private state:
731
+
732
+ ```ts
733
+ Data.inspect(model) // serializable cache/domain summary
734
+ Data.plan(model, projection) // the entity plan a read would execute
735
+ Remote.planQueries(...) // missing/stale query work
736
+ Remote.inspectEntity(...) // one normalized entity
737
+ ```
738
+
739
+ These are pure and useful in tests, tooling, and debugging. A particularly
740
+ useful question is: **"Why is this Projection still Initial?"** `Data.plan`
741
+ shows whether Remote believes anything is actually missing; active Surface
742
+ wiring determines whether that plan is being executed.
400
743
 
401
744
  ## Advanced: the kernel
402
745
 
403
- Everything above compiles to the exports below; nothing is a parallel
404
- implementation. Reach for them in tooling, SSR, tests, a package that builds on
405
- Remote, or an application that needs one piece by hand.
746
+ Everything above compiles to the lower-level exports below; there is not a
747
+ second implementation. Reach for the kernel in tooling, SSR, tests, packages
748
+ built on Remote, or applications that deliberately need one piece by hand.
406
749
 
407
750
  ### Descriptors and binding
408
751
 
409
- `Remote.define({ entities, queries, mutations })` is the domain without a
410
- place in a Model: `Model`, `initial`, `Message`, `update`, `rpc`, and a
411
- name-keyed `registry` of the declared descriptors (consumed by
412
- `RemoteServer.validate`). `Remote.at(definition, modelRef)` binds it to a
413
- `ModelRef`; `Remote.make` is the two in one step. `Selection.make(Entity,
414
- {…})`, `Selection.connection(Entity, window, nested?)`, and
415
- `Entity.patch(ref, values)` are the kernel constructors behind the entity's
416
- `select` and `patch` methods. `Remote.select(bound, selection)(id)` is
752
+ `Remote.define({ entities, queries, mutations })` creates a domain without a
753
+ place in an application Model. It exposes the Remote `Model`, `initial`,
754
+ `Message`, reducer, RPC group, and a name-keyed registry consumed by
755
+ `RemoteServer.validate`.
756
+
757
+ `Remote.at(definition, modelRef)` binds that domain to a `ModelRef`;
758
+ `Remote.make` combines definition and binding in one step.
759
+
760
+ `Selection.make(Entity, { ... })`, `Selection.connection(...)`, and
761
+ `Entity.patch(ref, values)` are the kernel constructors behind the entity
762
+ helpers. `Remote.select(bound, selection)(id)` is the lower-level form of
417
763
  `Data.get`.
418
764
 
419
765
  ### Requirements and the planner
420
766
 
421
- A projection's `requirements` are plain data — entity, id, fields, a pagination
422
- window per relation, and through `relations`, the slice required of each
423
- relation's target — and its `connections` the query connections it reads
424
- (`{ identity, window, select }`). `Remote.plan(bound, model, projection,
425
- options?)` diffs them against the visible store and returns only the missing
426
- or stale fields, deterministically: a relation whose field is being fetched
427
- rides on the request so the server resolves the graph in one read, a relation
428
- the store already holds is followed into concrete requirements for its
429
- targets, and a known connection contributes its visible items' fields under
430
- `select`. `Remote.planQueries` is the connections it does not hold, or holds
431
- stale. `options` is a `PlanOptions`: `freshness` (`{ now, freshness }`)
432
- refreshes an entry older than the window, `force` plans every field;
433
- `RemotePolicy.toPlan(policy, now)` compiles a policy to it. `Remote.storeOf` is
434
- the visible store — the base under the pending optimistic layers, computed once
435
- per Model state so every read and plan of one render shares it; a read's
436
- result is memoized per that store (and, for a page, per connection), so equal
437
- reads of one Model state assemble and decode once and return one value — and
438
- `Remote.prefetch(bound, model, projection, options?)` runs the entity plan and
439
- returns the new store.
767
+ A Projection carries its requirements as Remote's own metadata, attached with
768
+ `RemoteRequirements.of` / `RemoteConnections.of` and read with
769
+ `requirementsOf(projection)` / `connectionsOf(projection)`. Requirements are
770
+ plain data: entity, id, fields, any relation pagination windows, and the nested
771
+ target selections reached through relations. Connections describe query
772
+ connections as `{ identity, window, select }`.
773
+
774
+ `Remote.plan(bound, model, projection, options?)` deterministically diffs those
775
+ requirements against the visible store and returns only missing/stale work. A
776
+ relation field being fetched rides on the same request so the server can resolve
777
+ the graph in one read; a relation already present in the store is followed into
778
+ requirements for the concrete target.
779
+
780
+ `Remote.planQueries` performs the same job for absent/stale connections.
781
+ `RemotePolicy.toPlan(policy, now)` compiles a high-level policy into planner
782
+ options.
783
+
784
+ `Remote.storeOf` is the visible store after pending optimistic layers. Reads are
785
+ memoized per visible store (and connection where applicable), so equal reads of
786
+ one Model state assemble/decode once and return one value.
787
+
788
+ `Remote.prefetch(bound, model, projection, options?)` runs the same plans
789
+ imperatively and reduces their results into a new Model.
440
790
 
441
791
  ### The reducer and its Messages
442
792
 
443
- `updateRemote(remote, message)` — `Data.update` on a bound domain, `Data.reduce`
444
- on the application Model — is the one pure reducer over
445
- `ReadReceived`/`ReadFailed`/`RefreshStarted`,
446
- `MutationStarted`/`Succeeded`/`Failed`, `LiveReceived`/`GapCleared`,
447
- `ConnectionMerged`/`Invalidated`/`Refreshed`/`QueryFailed`,
448
- `RetentionChanged`, and `Hydrated`. `Remote.messages` is the case record for
449
- `defineMessageUnion`, `Remote.reduces(message)` narrows an application's union
450
- to those cases, and `Remote.writeRead(store, requests, result, now)` is the
451
- store write a `ReadReceived` performs, for a caller that manages its own store.
793
+ `updateRemote(remote, message)` is the pure reducer over Remote facts;
794
+ `Data.update` applies it to a bound Remote state and `Data.reduce` applies it to
795
+ the application Model.
796
+
797
+ Its Messages cover reads and refreshes, mutations, live events/gaps, query
798
+ connections, retention, and hydration. `Remote.messages` is the case record for
799
+ `defineMessageUnion`, and `Remote.reduces(message)` narrows an application union
800
+ to those cases.
452
801
 
453
- ### The Subscription entries
802
+ `Remote.writeRead(store, requests, result, now)` exposes the store write behind a
803
+ read result for callers deliberately managing their own Remote store.
454
804
 
455
- `Data.subscriptions` is built from three kernel entries, each for one Surface
456
- with fixed params and a `toMessage` that wraps the `RemoteMessage` in an
457
- application union that does not spread `Remote.messages`:
805
+ ### Subscription entries
806
+
807
+ `Data.subscriptions` is built from three kernel entries:
458
808
 
459
809
  ```text
460
- Remote.observe(bound, surface, params, toMessage?, options?) plan + read + queries
461
- Remote.live(bound, surface, params, toMessage?, options?) the live stream, from the Model's cursor
462
- Remote.retain(projections, toMessage?, { connections?, grace? }) roots → RetentionChanged
810
+ Remote.observe(bound, surface, params, toMessage?, options?)
811
+ plan + entity reads + query reads
812
+
813
+ Remote.live(bound, surface, params, toMessage?, options?)
814
+ live streams from the Model-owned cursor
815
+
816
+ Remote.retain(projections, toMessage?, { connections?, grace? })
817
+ active roots -> RetentionChanged
463
818
  ```
464
819
 
465
- The read entry's dependencies are the plan (`{ requirements, queries }`); it
466
- emits nothing when both are empty, `RefreshStarted` before a refreshing read,
467
- and runs the entity read and the queries concurrently. Every Message it emits
468
- changes the Model, so Foldkit recomputes the dependencies and restarts the
469
- stream: a merged page's items are planned by that next computation, and a
470
- read the restart interrupts is joined by the coalescer rather than repeated.
471
- A page and its refresh are one `ConnectionMerged` (`refreshes: true`), since a
472
- second Message could be lost to the restart. The retain entry's dependencies
473
- are the roots (the projections' requirements, their connections with what
474
- each page selects of its items, plus any `connections` listed by identity); it
475
- emits `RetentionChanged` once the roots have been stable for `grace`, and
476
- the reducer applies the pure `gc(state, roots)`: a root entity, the targets
477
- its retained fields refer to, the targets a nested relation selects, a retained
478
- connection's edges and what its `select` reaches through them, and anything a
479
- pending request touches survive; everything else is dropped, settled overlays
480
- on a dropped connection included.
820
+ The observe entry's dependencies are the current plan and the Model's refresh
821
+ generation (`{ requirements, queries, refresh }`), so `Data.refresh` restarts
822
+ it. If entity requirements and queries are both empty, it performs no I/O. A
823
+ refreshing read emits `RefreshStarted` first, then a result Message.
824
+
825
+ Every emitted Message changes the Model, so Foldkit recomputes Subscription
826
+ dependencies. For example, after a query page lands, the next computation sees
827
+ the page's entity ids and plans any selected fields those entities still lack.
828
+ An interrupted identical read is joined by the client coalescer instead of
829
+ needlessly restarted.
830
+
831
+ Retention uses the Projection requirements/connections as roots and eventually
832
+ emits `RetentionChanged`; the reducer's pure `gc` keeps everything reachable
833
+ from those roots plus pending work and collects the rest.
481
834
 
482
835
  ### Connections by hand
483
836
 
484
- A connection stores entity references in segments with explicit boundaries,
485
- so an unloaded middle page is a gap rather than an implied adjacency; pages,
486
- live inserts, and optimistic inserts are all evidence about the same structure
487
- and merge through the reducer. A `QueryRef` is a server list operation with a
488
- canonical identity (the descriptor plus the encoded input, excluding the
489
- window), so `first(25)` and `after(cursor).first(25)` merge into one
490
- connection.
837
+ A connection stores entity references in segments with explicit boundaries, so
838
+ an unloaded middle page remains a gap rather than an implied adjacency. Query
839
+ pages, live inserts/removals, and optimistic overlays are reconciled as evidence
840
+ about that same structure.
841
+
842
+ A `QueryRef` has a canonical identity based on its descriptor and encoded input,
843
+ excluding the page window. Thus `first(25)` and `after(cursor).first(25)` merge
844
+ into the same logical connection.
491
845
 
492
846
  ```ts
493
- // Inside an `Effect.gen`, with `RemoteClient` provided.
847
+ // Inside Effect.gen, with RemoteClient provided.
494
848
  const ref = Query.first(25)(ProjectsByOwner.ref({ ownerId }))
495
- const page = yield* Remote.query(ref) // Effect<QueryResult, RemoteQueryError, RemoteClient>
496
- updateRemote(remote, Remote.queryMessage(ref, page)) // ConnectionMerged
497
- Remote.visibleItems(remote, ref) // edges a view shows: overlays placed, removals and tombstones hidden
849
+ const page = yield* Remote.query(ref)
850
+
851
+ updateRemote(remote, Remote.queryMessage(ref, page))
852
+ Remote.visibleItems(remote, ref)
498
853
  ```
499
854
 
500
- `items`, `hasNext`, `hasPrevious`, and `isGapped` read the server-known region.
501
- A merged page is newer than the settled overlays it covers: it drops a live
502
- insert it carries and a live removal it contradicts, and leaves a pending
503
- request's overlays alone; a replayed live event is a duplicate by cursor and
504
- changes nothing. `ConnectionInvalidated` marks a connection stale while it
505
- keeps showing its items; a `ConnectionMerged` with `refreshes: true`,
506
- `ConnectionRefreshed`, or `QueryFailed` clears it.
855
+ `items`, `hasNext`, `hasPrevious`, and `isGapped` describe the known region.
856
+ `ConnectionInvalidated` keeps visible rows but marks the connection stale until
857
+ a refresh/query result settles it. A `ConnectionMerged` page with
858
+ `refreshes: true` replaces a stale connection's pages rather than merging into
859
+ them.
507
860
 
508
861
  ### Mutations by hand
509
862
 
510
863
  `Remote.mutateInto(bound, model, mutation, input, requestId, { optimistic })` is
511
- the one-step imperative form (start, run, settle) for SSR and tests.
512
- `Remote.mutate(mutation, input, requestId)` is the call itself: it decodes the
513
- typed `Output` and returns the result's normalized `entities` and confirmed
514
- `connections`, for a caller that reduces them through `updateRemote`. A
864
+ the one-step imperative start/run/settle form for SSR and tests.
865
+
866
+ `Remote.mutate(mutation, input, requestId)` is the transport call itself. It
867
+ decodes the typed `Output` and returns normalized entity patches and confirmed
868
+ connection changes for a caller to reduce through `updateRemote`.
869
+
515
870
  `MutationSucceeded` reconciles at most once per `requestId`; an unknown or
516
- already-applied result is a no-op.
871
+ already-applied settlement is a no-op.
517
872
 
518
873
  ### The transport seam
519
874
 
520
- `RemoteClient` is an Effect service with `read`, `query`, `mutate`, and
521
- `live`; the wire schemas (`ReadBatch`, `QueryRequest`, `MutationRequest`,
522
- `LiveRequirement`, their results and errors) and the `RemoteRpc` group are
523
- exported. `Remote.clientLayer(rpcClient, { window? })` adapts an Effect RPC
524
- client, including the `LiveChange`-to-`LiveEvent` mapping; what the client
525
- requires (a database under in-process `RemoteServer.handlers`) is supplied to
526
- the layer with `Layer.provide`. Reads and queries through it coalesce:
527
- requirements issued together become one `ReadBatch` (ids batched, fields
528
- unioned), a requirement or an identical query already in flight is joined, a
529
- failed read releases it, and a requirement that pages a relation reads alone
530
- because a page answers exactly one window. `Remote.coalesced(layer, options)`
531
- wraps a hand-written client the same way; `window` widens the batching delay
532
- beyond "issued concurrently".
533
-
534
- ## What it owns
535
-
536
- - **Entity identity and references.** `Entity.make`, typed `EntityRef`s, and
537
- reference codecs; an entity's `select` and `patch` methods (and `Entity.patch`)
538
- type a selection or a patch against its fields.
539
- - **Field selections.** `Selection.make` derives a Struct from the picked fields
540
- and rejects unknown ones; a nested selection reads through a relation, and
541
- the requirement carries the graph so one read resolves it.
542
- - **The normalized store.** Values, per-field presence, staleness, and tombstones
543
- are tracked separately, so `undefined`, `null`, absent, stale, and not-found
544
- are distinct. Presence is never inferred from `value === undefined`.
545
- - **The requirement planner.** `Remote.plan` diffs requirements against the
546
- visible store and returns only missing or stale fields, deterministically.
547
- - **The Remote submodel and the bound domain.** `Remote.Model`/`initial` are
548
- the submodel; `Remote.define` returns `Message`, `update`, `rpc`, and a
549
- name-keyed `registry`; `Remote.make` binds it and adds `get`, `live`,
550
- `query`, `next`, `previous`, `fetch`, `subscriptions`, `plan`, `storeOf`,
551
- `prefetch`, `mutate`, `reduce`, and `inspect`, each rejecting a descriptor
552
- the domain never declared at compile time (a branded error naming it) and at
553
- runtime. `updateRemote` is the single reducer over reads, mutation results,
554
- live events, connections, and optimistic layers.
555
- - **Mutation reconciliation.** Idempotent per `requestId`, with a bounded
556
- settled-request ledger.
557
- - **Connections.** Segmented ordered data with explicit boundaries and overlay
558
- placement.
559
- - **Queries.** `Data.query` reads a connection as a page of selected items and
560
- carries the connection for the planner; `Data.next`/`previous`/`fetch` page
561
- it. `Remote.query(ref)` encodes and runs a `QueryRef` by hand and
562
- `Remote.queryMessage(ref, page)` merges the result into the connection keyed
563
- by `ref.identity`.
564
- - **Live classification.** Per-stream cursor ordering, duplicate suppression, and
565
- gap detection; a gap clears when an in-order event applies or a `GapCleared`
566
- message arrives.
567
- - **Introspection.** `Remote.inspect` and `Remote.inspectEntity` return a pure,
568
- serializable cache view for DevTools.
569
- - **The transport seam.** `RemoteClient`, an Effect service with `read`, `query`,
570
- `mutate`, and `live`; the wire schemas and `RemoteRpc` group.
571
- `Remote.clientLayer(rpcClient)` adapts an Effect RPC client for `RemoteRpc` to
572
- `RemoteClient`, including the `LiveChange`-to-`LiveEvent` mapping; what the
573
- client requires (a database under in-process `RemoteServer.handlers`) is
574
- supplied to the layer with `Layer.provide`.
575
- - **Disposable cache persistence.** Deterministic, scoped, size-bounded
576
- snapshots of the entity store: `dehydrate`/`hydrate` as text, `save`/`restore`
577
- over `KeyValueStore`, and `Hydrated` to merge one into the Model.
875
+ `RemoteClient` is an Effect service with `read`, `query`, `mutate`, and `live`.
876
+ The wire schemas (`ReadBatch`, `QueryRequest`, `MutationRequest`,
877
+ `LiveRequirement`, results/errors) and the `RemoteRpc` group are exported.
878
+
879
+ `Remote.clientLayer(rpcClient, { window? })` adapts an Effect RPC client,
880
+ including live-change mapping. Compatible reads and queries coalesce:
881
+ requirements issued together become one `ReadBatch`, an identical in-flight
882
+ requirement/query is joined, and a failed request releases the coalesced entry.
883
+ A relation request with its own pagination window stays separate because one
884
+ response answers one window exactly.
885
+
886
+ `Remote.coalesced(layer, options)` adds the same behavior to a hand-written
887
+ `RemoteClient` layer; `window` widens batching beyond work issued concurrently.
888
+
889
+ ## What Remote owns
890
+
891
+ Remote owns:
892
+
893
+ - typed entity identity, references, selections, and patches;
894
+ - the normalized entity store, field presence/staleness, and tombstones;
895
+ - pure requirements and deterministic planning;
896
+ - the `Remote.Model` Submodel and its reducer;
897
+ - query connections and pagination boundaries;
898
+ - optimistic mutation overlays and idempotent settlement bookkeeping;
899
+ - live cursors, duplicate suppression, and gap detection;
900
+ - retention/garbage collection of disposable cache data;
901
+ - deterministic, scoped cache snapshots;
902
+ - the `RemoteClient` semantic transport seam and `RemoteRpc` schemas;
903
+ - pure introspection of the cache and plans.
904
+
905
+ Remote does **not** own:
906
+
907
+ - your source database or authoritative server records;
908
+ - local UI/application state outside `Remote.Model`;
909
+ - durable offline client intent or replica convergence;
910
+ - HTTP/WebSocket deployment details;
911
+ - authentication; server-side authorization belongs at the Source boundary;
912
+ - a background scheduler or general-purpose database/query engine.
578
913
 
579
914
  ## Limits
580
915
 
581
- - The transport is not part of the package. Effect RPC is the wire; the
582
- application supplies the client and server protocol layers. `ReadBatch` and
583
- `LiveRequirement` name `REMOTE_PROTOCOL_VERSION`; a server refuses another
584
- version with `RemoteProtocolError`, so a wire change is a version bump, never
585
- silent drift. A request names at most `MAX_FIELDS_PER_REQUEST` (256) fields
586
- of one entity and nests relations at most `MAX_RELATION_DEPTH` (8) deep; the
587
- wire refuses more at decode rather than truncating.
588
- - The wire `LiveChange` union carries entity patches, deletes, and connection
589
- insert/remove/invalidate changes; the client adapter reconstructs a `LiveEvent`
590
- from it. `RemoteServer.live` serves them as a stream.
591
- - Coalescing is per `RemoteClient` layer: two Remote domains with separate
592
- layers do not share a batch. Retention keeps what the active Surfaces reach
593
- (and what `Remote.retain` lists); a Surface that is neither active nor listed
594
- loses its data on the next `RetentionChanged`.
595
- - Two applications with the same Model type cannot be told apart by the types;
596
- `Data.subscriptions` rejects a Surface of another application at runtime by
597
- its owner token.
598
- - A refreshing policy's `maxAge` applies to entities, which the store stamps
599
- with a clock; a connection has no age, so a page is re-queried only when the
600
- connection is invalidated (`ConnectionInvalidated`, a live
601
- `ConnectionInvalidate`, or `networkOnly`'s `force`).
916
+ - Effect RPC is the wire contract; the application supplies deployment layers.
917
+ `ReadBatch` and `LiveRequirement` carry `REMOTE_PROTOCOL_VERSION`, and a
918
+ mismatched version fails with `RemoteProtocolError` rather than silently
919
+ drifting.
920
+ - One request may name at most `MAX_FIELDS_PER_REQUEST` (256) fields of one
921
+ entity and nest relations at most `MAX_RELATION_DEPTH` (8) levels; the wire
922
+ rejects larger inputs.
923
+ - The wire `LiveChange` union carries entity patches/deletes and connection
924
+ insert/remove/invalidate changes; `RemoteServer.live` serves them as a stream.
925
+ - Coalescing is scoped to one `RemoteClient` layer. Separate layers do not share
926
+ batches.
927
+ - Retention keeps what active Surfaces (and explicit retain entries) reach. An
928
+ inactive/unretained Surface may lose its cache on `RetentionChanged`.
929
+ - Two applications with the same Model type are not distinguishable by TypeScript
930
+ alone; `Data.subscriptions` also checks the Surface owner token at runtime.
931
+ - `staleWhileRevalidate({ maxAge })` ages entity data. Connections have no age;
932
+ they re-query when invalidated or when `networkOnly` forces them.
602
933
  - `RemoteData` is a closed union.
603
- - A selection picks at least one field: `Selection.make(User, {})` throws,
604
- since it would require nothing and read `Ready` for any id.
934
+ - A Selection must choose at least one field; an empty Selection would require
935
+ nothing and misleadingly read `Ready` for every id, so it is rejected.
605
936
 
606
937
  ## See also
607
938
 
608
- - [Server-derived state](https://github.com/doeixd/foldkit-plus/blob/main/docs/remote.md) — the mental model, and
609
- when to reach for something else.
610
- - [`foldkit-remote-server`](https://github.com/doeixd/foldkit-plus/tree/main/packages/remote-server) — the server half; [`foldkit-remote-drizzle`](https://github.com/doeixd/foldkit-plus/tree/main/packages/remote-drizzle) compiles its
611
- selections and queries to SQL.
612
- - [`examples/remote`](https://github.com/doeixd/foldkit-plus/tree/main/examples/remote) — a worked plan, prefetch, render, mutate, retain trace;
613
- [`examples/kitchen-sink`](https://github.com/doeixd/foldkit-plus/tree/main/examples/kitchen-sink) runs the same path over the real server packages.
939
+ - [Server-derived state](../../docs/remote.md) — the deeper architectural guide
940
+ and ownership model.
941
+ - [`foldkit-remote-server`](../remote-server) — server Sources, resolution, and
942
+ semantic-field authorization.
943
+ - [`foldkit-remote-drizzle`](../remote-drizzle) — optional compilation of Remote
944
+ selections/queries into Drizzle access.
945
+ - [`examples/remote`](../../examples/remote) — an asserted trace through plan,
946
+ prefetch, refresh, query, render, mutation, retention, and decode failure.
947
+ - [`examples/kitchen-sink`](../../examples/kitchen-sink) — the same architecture
948
+ using the real server packages.