@drupflare/spool 0.4.0 โ†’ 1.0.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (162) hide show
  1. package/README.md +206 -1756
  2. package/dist/angular/index.d.ts +13 -0
  3. package/dist/angular/index.js +25 -0
  4. package/dist/angular/pages.d.ts +28 -0
  5. package/dist/angular/pages.js +85 -0
  6. package/dist/cdn/client.js +1 -1
  7. package/dist/cdn/location.d.ts +6 -0
  8. package/dist/cdn/location.js +8 -0
  9. package/dist/cdn/manifest.json +7 -7
  10. package/dist/cdn/spool-sw.js +1 -1
  11. package/dist/client/bootstrap.d.ts +57 -0
  12. package/dist/client/bootstrap.js +307 -0
  13. package/dist/client/contact.d.ts +24 -0
  14. package/dist/client/contact.js +66 -0
  15. package/dist/client/declared.d.ts +23 -0
  16. package/dist/client/declared.js +68 -0
  17. package/dist/client/entry.js +91 -2
  18. package/dist/client/gate.d.ts +1 -0
  19. package/dist/client/gate.js +148 -0
  20. package/dist/client/index.d.ts +25 -0
  21. package/dist/client/index.js +1 -0
  22. package/dist/client/live.d.ts +66 -0
  23. package/dist/client/live.js +225 -0
  24. package/dist/client/navigate-entry.d.ts +1 -0
  25. package/dist/client/navigate-entry.js +4 -0
  26. package/dist/client/navigate.d.ts +54 -0
  27. package/dist/client/navigate.js +272 -0
  28. package/dist/client/persist.d.ts +2 -0
  29. package/dist/client/persist.js +9 -0
  30. package/dist/client/register.d.ts +11 -1
  31. package/dist/client/register.js +19 -8
  32. package/dist/express/index.d.ts +11 -0
  33. package/dist/express/index.js +209 -0
  34. package/dist/hono/index.d.ts +19 -0
  35. package/dist/hono/index.js +49 -0
  36. package/dist/next/index.d.ts +33 -0
  37. package/dist/next/index.js +41 -0
  38. package/dist/next/pages.d.ts +22 -0
  39. package/dist/next/pages.js +39 -0
  40. package/dist/node/listener.d.ts +3 -0
  41. package/dist/node/listener.js +57 -2
  42. package/dist/nuxt/index.d.ts +22 -0
  43. package/dist/nuxt/index.js +104 -0
  44. package/dist/nuxt/pages.d.ts +26 -0
  45. package/dist/nuxt/pages.js +48 -0
  46. package/dist/nuxt/runtime/index.d.ts +24 -0
  47. package/dist/nuxt/runtime/index.js +58 -0
  48. package/dist/plugins/coverage.d.ts +14 -0
  49. package/dist/plugins/coverage.js +27 -0
  50. package/dist/plugins/mode.d.ts +19 -0
  51. package/dist/plugins/mode.js +20 -0
  52. package/dist/plugins/path.d.ts +16 -0
  53. package/dist/plugins/path.js +139 -0
  54. package/dist/plugins/report.d.ts +22 -0
  55. package/dist/plugins/report.js +18 -0
  56. package/dist/plugins/routes.d.ts +38 -0
  57. package/dist/plugins/routes.js +52 -0
  58. package/dist/plugins/static.d.ts +42 -0
  59. package/dist/plugins/static.js +97 -0
  60. package/dist/protocol/bundle.d.ts +111 -0
  61. package/dist/protocol/bundle.js +428 -0
  62. package/dist/protocol/config.d.ts +65 -1
  63. package/dist/protocol/config.js +28 -0
  64. package/dist/protocol/dataset.d.ts +6 -0
  65. package/dist/protocol/dataset.js +6 -0
  66. package/dist/protocol/envelope.d.ts +33 -1
  67. package/dist/protocol/envelope.js +15 -1
  68. package/dist/protocol/errors.d.ts +1 -1
  69. package/dist/protocol/index-rows.d.ts +22 -0
  70. package/dist/protocol/index-rows.js +53 -0
  71. package/dist/protocol/index.d.ts +2 -0
  72. package/dist/protocol/index.js +2 -0
  73. package/dist/protocol/normalise.d.ts +4 -4
  74. package/dist/protocol/normalise.js +9 -5
  75. package/dist/protocol/ring.d.ts +58 -0
  76. package/dist/protocol/ring.js +6 -0
  77. package/dist/protocol/route.d.ts +57 -11
  78. package/dist/protocol/route.js +95 -3
  79. package/dist/protocol/tag.d.ts +22 -2
  80. package/dist/protocol/tag.js +25 -4
  81. package/dist/protocol/view.d.ts +2 -0
  82. package/dist/protocol/view.js +8 -2
  83. package/dist/protocol/views.d.ts +6 -0
  84. package/dist/protocol/views.js +15 -0
  85. package/dist/protocol/write.d.ts +92 -4
  86. package/dist/protocol/write.js +251 -9
  87. package/dist/proxy/eligible.d.ts +22 -1
  88. package/dist/proxy/eligible.js +62 -15
  89. package/dist/proxy/proxy.js +28 -18
  90. package/dist/react-router/index.d.ts +20 -0
  91. package/dist/react-router/index.js +60 -0
  92. package/dist/react-router/pages.d.ts +34 -0
  93. package/dist/react-router/pages.js +83 -0
  94. package/dist/report/demand.d.ts +49 -0
  95. package/dist/report/demand.js +60 -0
  96. package/dist/report/economics.d.ts +55 -0
  97. package/dist/report/economics.js +189 -0
  98. package/dist/report/headline.d.ts +158 -0
  99. package/dist/report/headline.js +473 -0
  100. package/dist/report/index.d.ts +2 -0
  101. package/dist/report/index.js +2 -0
  102. package/dist/report/inputs.d.ts +79 -0
  103. package/dist/report/inputs.js +115 -0
  104. package/dist/report/measurement.d.ts +1 -1
  105. package/dist/report/measurement.js +5 -1
  106. package/dist/report/traffic.d.ts +8 -0
  107. package/dist/report/traffic.js +7 -0
  108. package/dist/server/bundle.d.ts +55 -0
  109. package/dist/server/bundle.js +172 -0
  110. package/dist/server/dataset.d.ts +8 -2
  111. package/dist/server/dataset.js +51 -4
  112. package/dist/server/embedded.d.ts +7 -3
  113. package/dist/server/embedded.js +7 -3
  114. package/dist/server/index.d.ts +5 -1
  115. package/dist/server/index.js +5 -1
  116. package/dist/server/live-sockets.d.ts +54 -0
  117. package/dist/server/live-sockets.js +131 -0
  118. package/dist/server/live.d.ts +146 -0
  119. package/dist/server/live.js +228 -0
  120. package/dist/server/ring.d.ts +77 -0
  121. package/dist/server/ring.js +311 -0
  122. package/dist/server/spool.d.ts +139 -4
  123. package/dist/server/spool.js +457 -28
  124. package/dist/server/static.d.ts +42 -0
  125. package/dist/server/static.js +29 -0
  126. package/dist/server/stub.d.ts +43 -5
  127. package/dist/server/stub.js +126 -8
  128. package/dist/server/write.d.ts +23 -1
  129. package/dist/server/write.js +96 -0
  130. package/dist/stores/durable-object.d.ts +46 -0
  131. package/dist/stores/durable-object.js +114 -0
  132. package/dist/sw/analytics.d.ts +1 -1
  133. package/dist/sw/analytics.js +2 -2
  134. package/dist/sw/coverage.d.ts +6 -0
  135. package/dist/sw/coverage.js +34 -0
  136. package/dist/sw/family.d.ts +15 -0
  137. package/dist/sw/family.js +48 -0
  138. package/dist/sw/idb.d.ts +2 -0
  139. package/dist/sw/idb.js +2 -2
  140. package/dist/sw/index.d.ts +2 -0
  141. package/dist/sw/index.js +1 -0
  142. package/dist/sw/pack/store.d.ts +128 -0
  143. package/dist/sw/pack/store.js +737 -0
  144. package/dist/sw/policy.d.ts +2 -0
  145. package/dist/sw/policy.js +11 -0
  146. package/dist/sw/probe.d.ts +16 -0
  147. package/dist/sw/probe.js +25 -0
  148. package/dist/sw/router.d.ts +3 -2
  149. package/dist/sw/router.js +15 -2
  150. package/dist/sw/runtime.d.ts +48 -1
  151. package/dist/sw/runtime.js +548 -68
  152. package/dist/sw/scheduler.d.ts +6 -1
  153. package/dist/sw/scheduler.js +22 -5
  154. package/dist/sw/write.d.ts +41 -1
  155. package/dist/sw/write.js +188 -5
  156. package/dist/version.d.ts +1 -1
  157. package/dist/version.js +1 -2
  158. package/dist/vite/index.d.ts +26 -0
  159. package/dist/vite/index.js +66 -0
  160. package/dist/vue/index.d.ts +36 -0
  161. package/dist/vue/index.js +53 -0
  162. package/package.json +64 -4
package/README.md CHANGED
@@ -6,7 +6,7 @@
6
6
  [![Browser](https://github.com/drupflare/spool/actions/workflows/browser.yml/badge.svg)](https://github.com/drupflare/spool/actions/workflows/browser.yml)
7
7
  [![Prettier](https://github.com/drupflare/spool/actions/workflows/prettier.yml/badge.svg)](https://github.com/drupflare/spool/actions/workflows/prettier.yml)
8
8
  [![codecov](https://codecov.io/gh/drupflare/spool/branch/master/graph/badge.svg)](https://codecov.io/gh/drupflare/spool)
9
- [![License](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE)
9
+ [![License](https://img.shields.io/badge/license-MIT-blue.svg)](https://github.com/drupflare/spool/blob/master/LICENSE)
10
10
 
11
11
  spool removes repeated execution demand from your server. It keeps a replica of what a page needs
12
12
  on the visitor's device. The server sends templates and authorised, versioned data, and a service
@@ -16,7 +16,7 @@ the server hits of a local view, and in Chromium and Firefox the count is zero.
16
16
 
17
17
  spool has three uses. In a drupflare deployment it takes eligible public views off the Worker.
18
18
  For a team on a free or constrained plan it lowers the requests the plan has to carry, by an
19
- amount that depends on the site's traffic ([Request Arithmetic](#request-arithmetic)). It is also
19
+ amount that depends on the site's traffic ([Request Arithmetic](ADVANCED_USAGE.md#request-arithmetic)). It is also
20
20
  a research system: every figure it reports carries a layer and a method.
21
21
 
22
22
  It runs on any runtime that speaks `Request` and `Response`, bridges `node:http` for the rest, and
@@ -24,43 +24,30 @@ has no runtime dependencies. A site with no server code can load it from two fil
24
24
 
25
25
  ---
26
26
 
27
- ## ๐Ÿ“‹ Table of Contents
28
-
29
- - [Why Spool](#-why-spool)
30
- - [Install](#-install)
31
- - [Quick Start](#-quick-start)
32
- - [Page Runtime](#-page-runtime)
33
- - [Runtimes](#-runtimes)
34
- - [Compatibility](#-compatibility)
35
- - [Static Sites](#-static-sites)
36
- - [Strategies](#-strategies)
37
- - [Freshness Contracts](#-freshness-contracts)
38
- - [Datasets](#-datasets)
39
- - [Authenticated Data](#-authenticated-data)
40
- - [Declared Writes](#-declared-writes)
41
- - [Catalogue](#-catalogue)
42
- - [Analytics Batching](#-analytics-batching)
43
- - [Third-Party Files](#-third-party-files)
44
- - [Subresource Handling](#-subresource-handling)
45
- - [Shadow Mode](#-shadow-mode)
46
- - [Proxy Mode](#-proxy-mode)
47
- - [Trimmed Stylesheets](#-trimmed-stylesheets)
48
- - [Images and Video](#-images-and-video)
49
- - [Capacity Reports](#-capacity-reports)
50
- - [Security](#-security)
51
- - [Energy Reporting](#-energy-reporting)
52
- - [Prior Art](#-prior-art)
53
- - [Out of Scope](#-out-of-scope)
54
- - [Browser Support](#-browser-support)
55
- - [Subpath Exports](#-subpath-exports)
56
- - [Testing](#-testing)
57
- - [Documentation](#-documentation)
58
- - [Related Repositories](#-related-repositories)
59
- - [License](#-license)
27
+ ## Table of Contents
28
+
29
+ - [Why Spool](#why-spool)
30
+ - [Install](#install)
31
+ - [Quick Start](#quick-start)
32
+ - [Page Runtime](#page-runtime)
33
+ - [Runtimes](#runtimes)
34
+ - [Compatibility](#compatibility)
35
+ - [Static Sites](#static-sites)
36
+ - [Advanced Usage](#advanced-usage)
37
+ - [Security](#security)
38
+ - [Energy Reporting](#energy-reporting)
39
+ - [Prior Art](#prior-art)
40
+ - [Out of Scope](#out-of-scope)
41
+ - [Browser Support](#browser-support)
42
+ - [Subpath Exports](#subpath-exports)
43
+ - [Testing](#testing)
44
+ - [Documentation](#documentation)
45
+ - [Related Repositories](#related-repositories)
46
+ - [License](#license)
60
47
 
61
48
  ---
62
49
 
63
- ## ๐ŸŽฏ Why Spool
50
+ ## Why Spool
64
51
 
65
52
  A local view removes, in order, a request that would have reached the Worker, the server work
66
53
  behind it (rendering, queries, Durable Object and database reads), and the bytes of a document the
@@ -77,14 +64,14 @@ the worker cannot answer goes to the network, so adopting spool changes no page
77
64
  declared.
78
65
 
79
66
  A collection of thousands of rows, such as a directory or a course catalogue, can be held as a
80
- [dataset](#-datasets). The device seeds it once and then receives only the rows that changed.
67
+ [dataset](ADVANCED_USAGE.md#datasets). The device seeds it once and then receives only the rows that changed.
81
68
 
82
69
  [docs/impact.md](docs/impact.md) holds the request, cost and energy figures and the measurements
83
- behind them. See [Energy Reporting](#-energy-reporting).
70
+ behind them. See [Energy Reporting](#energy-reporting).
84
71
 
85
72
  ---
86
73
 
87
- ## ๐Ÿ“ฅ Install
74
+ ## Install
88
75
 
89
76
  ### Package
90
77
 
@@ -110,17 +97,17 @@ notes list the hash for each version.
110
97
  ></script>
111
98
  ```
112
99
 
113
- [Page Runtime](#-page-runtime) shows how `createSpool` injects the same tag.
100
+ [Page Runtime](#page-runtime) shows how `createSpool` injects the same tag.
114
101
 
115
102
  ### Direct Files
116
103
 
117
104
  `spool-sw.js`, `client.js` and `manifest.json` are attached to every GitHub release and published
118
105
  under `https://cdn.drupflare.com/spool/<version>/`. A site that serves them as plain files needs no
119
- server library. See [Static Sites](#-static-sites).
106
+ server library. See [Static Sites](#static-sites).
120
107
 
121
108
  ---
122
109
 
123
- ## ๐Ÿš€ Quick Start
110
+ ## Quick Start
124
111
 
125
112
  Every runtime below makes the same three moves:
126
113
 
@@ -130,6 +117,9 @@ Every runtime below makes the same three moves:
130
117
  every HTML page the application returns, and `spool.decorate` does the same where there is no
131
118
  Fetch response to wrap.
132
119
 
120
+ The [Getting Started](docs/getting-started.md) page walks through the same steps with the install
121
+ options, a run and the checks that confirm the setup.
122
+
133
123
  ### Declare Routes
134
124
 
135
125
  A route has a path pattern, a strategy, the tags it depends on and a `load` function that returns
@@ -191,7 +181,7 @@ holding a page that depends on them fetch a fresh copy before they show it again
191
181
 
192
182
  ---
193
183
 
194
- ## ๐Ÿ“ฆ Page Runtime
184
+ ## Page Runtime
195
185
 
196
186
  `handle` and `decorate` inject one script tag into each HTML page. By default the script is
197
187
  served from the same origin at `/.spool/<version>/client.js`, which suits a strict
@@ -216,8 +206,26 @@ script fails to load, the page works as it did before.
216
206
 
217
207
  The script registers `/spool-sw.js` and exposes `window.spool.ready`, which resolves to the client
218
208
  handle or to `null` where service workers do not exist. It also exposes `window.spool.activity`,
219
- described below, and the write calls of [Declared Writes](#-declared-writes). To bundle the runtime
220
- yourself, import `register` from `@drupflare/spool/client`.
209
+ described below, and the write calls of [Declared Writes](ADVANCED_USAGE.md#declared-writes). To bundle the runtime
210
+ yourself, import `register` from `@drupflare/spool/client`. The `window.spool` types come with that
211
+ path too, so a project that imports it gets them.
212
+
213
+ `createSpool({ workerMode: 'never' })` serves a site with no service worker. The page installs its
214
+ pack and answers held reads itself, and `/spool-sw.js` answers `404`. A declared write goes through
215
+ `window.spool.fetch` in place of `fetch`. The answer updates the held pack from the delta or digest it
216
+ carries, and any other URL goes to `fetch` unchanged. `spool.write` still queues a value for the
217
+ worker, so without a worker it answers `unavailable`. A public HTML page carries the declarations in
218
+ its ring, or in a `spool-writes` element when it has no ring. On a site with the bootstrap gate, every
219
+ public HTML page carries the ring.
220
+
221
+ ```ts
222
+ // a page with no worker
223
+ const response = await window.spool?.fetch('/courses/101', {
224
+ method: 'PATCH',
225
+ headers: { 'content-type': 'application/json' },
226
+ body: JSON.stringify({ title: 'Biology 101' })
227
+ });
228
+ ```
221
229
 
222
230
  ### Activity Hooks
223
231
 
@@ -254,462 +262,68 @@ measured, the off-screen pause lowered power for a page whose frame loop kept ru
254
262
  hidden-page pause showed no saving for a once-a-second poll. Both readings are in
255
263
  [docs/impact.md](docs/impact.md#metered-figures-beside-the-ledger).
256
264
 
257
- ---
258
-
259
- ## ๐Ÿงฉ Runtimes
260
-
261
- Each example uses the `routes`, `page` and `spool` from [Quick Start](#-quick-start). The
262
- [compatibility table](#-compatibility) lists what each framework hands a handler.
263
-
264
- ### Cloudflare Workers
265
-
266
- Bind the Durable Object that holds the version vector, then pass its namespace to
267
- `durableObjectStore`.
268
-
269
- ```jsonc
270
- {
271
- "durable_objects": { "bindings": [{ "name": "SPOOL_VERSIONS", "class_name": "SpoolVersions" }] },
272
- "migrations": [{ "tag": "v1", "new_sqlite_classes": ["SpoolVersions"] }]
273
- }
274
- ```
275
-
276
- ```ts
277
- import { createSpool } from '@drupflare/spool/server';
278
- import { durableObjectStore } from '@drupflare/spool/stores/durable-object';
279
- import { page, routes } from './routes';
280
-
281
- export { SpoolVersions } from '@drupflare/spool/stores/durable-object';
282
-
283
- export default {
284
- fetch(request: Request, env: Cloudflare.Env) {
285
- const spool = createSpool({ routes, store: durableObjectStore(env.SPOOL_VERSIONS) });
286
- return spool.handle(request, page);
287
- }
288
- };
289
- ```
290
-
291
- ### Cloudflare Pages Functions
292
-
293
- A function takes a context with the request and a `next` that runs the next function or the
294
- static asset server.
295
-
296
- ```ts
297
- import { spool } from './spool';
298
-
299
- export const onRequest = (context: { request: Request; next(): Promise<Response> }) =>
300
- spool.handle(context.request, () => context.next());
301
- ```
302
-
303
- ### Hono
304
-
305
- Hono ignores the response a middleware returns once a route has answered, so assign it to
306
- `c.res`.
265
+ ### In-Page Navigation
307
266
 
308
- ```ts
309
- import { Hono } from 'hono';
310
- import { page } from './routes';
311
- import { spool } from './spool';
312
-
313
- const app = new Hono();
314
-
315
- app.use('*', async (c, next) => {
316
- c.res = await spool.handle(c.req.raw, async () => {
317
- await next();
318
- return c.res;
319
- });
320
- });
321
- app.get('/node/:id', (c) => page(c.req.raw));
322
-
323
- export default app;
324
- ```
325
-
326
- ### Elysia
327
-
328
- `onRequest` answers the spool paths and returns nothing for the rest. The hook has to be an
329
- `async` function, because Elysia does not await a promise that a plain function returns.
330
-
331
- ```ts
332
- import { Elysia } from 'elysia';
333
- import { page } from './routes';
334
- import { spool } from './spool';
335
-
336
- export const app = new Elysia()
337
- .onRequest(async ({ request }) => {
338
- const { pathname } = new URL(request.url);
339
- return pathname === '/spool-sw.js' ? spool.stub() : await spool.respond(request);
340
- })
341
- .get('/node/:id', ({ request }) => spool.decorate(request, page(request)));
342
- ```
343
-
344
- ### itty-router
345
-
346
- A handler that returns `undefined` passes the request to the next one, which is what
347
- `spool.respond` returns for a path that is not spool's.
348
-
349
- ```ts
350
- import { IttyRouter } from 'itty-router';
351
- import { page } from './routes';
352
- import { spool } from './spool';
353
-
354
- const router = IttyRouter();
355
-
356
- router
357
- .all('*', (request) => spool.respond(request))
358
- .get('/spool-sw.js', () => spool.stub())
359
- .get('/node/:id', (request) => spool.decorate(request, page(request)));
360
-
361
- export default { fetch: router.fetch };
362
- ```
363
-
364
- ### Bun
365
-
366
- Bun serves a default export that has a `fetch` function.
367
-
368
- ```ts
369
- import { page } from './routes';
370
- import { spool } from './spool';
371
-
372
- export default {
373
- port: 3000,
374
- fetch: (request: Request) => spool.handle(request, page)
375
- };
376
- ```
377
-
378
- ### Deno
379
-
380
- Run it with `deno serve`.
381
-
382
- ```ts
383
- import { page } from './routes';
384
- import { spool } from './spool';
385
-
386
- export default {
387
- fetch: (request: Request) => spool.handle(request, page)
388
- };
389
- ```
390
-
391
- ### Netlify Functions
392
-
393
- A function takes a `Request` and returns a `Response`. `config.path` sends the spool paths and the
394
- pages to it.
395
-
396
- ```ts
397
- import type { Config, Context } from '@netlify/functions';
398
- import { page } from './routes';
399
- import { spool } from './spool';
400
-
401
- export default (request: Request, context: Context) => spool.handle(request, page);
402
-
403
- export const config: Config = { path: ['/node/*', '/spool-sw.js', '/.spool/*'] };
404
- ```
405
-
406
- ### Netlify Edge Functions
407
-
408
- An edge function runs in front of the site. `context.next()` returns the response the site would
409
- have sent, which `handle` decorates.
410
-
411
- ```ts
412
- import type { Config, Context } from '@netlify/edge-functions';
413
- import { spool } from './spool';
414
-
415
- export default (request: Request, context: Context) => spool.handle(request, () => context.next());
416
-
417
- export const config: Config = { path: '/*' };
418
- ```
419
-
420
- ### Vercel Functions
421
-
422
- A function under `api/` takes a default export with `fetch`. Send every path to it from
423
- `vercel.json`, so the function sees `/.spool/*` and `/spool-sw.js` as well as the pages. Function
424
- instances do not share memory, so `store` has to be a shared `VersionStore`.
425
-
426
- ```json
427
- { "rewrites": [{ "source": "/(.*)", "destination": "/api" }] }
428
- ```
429
-
430
- ```ts
431
- import { page } from './routes';
432
- import { spool } from './spool';
433
-
434
- export default {
435
- fetch: (request: Request) => spool.handle(request, page)
436
- };
437
- ```
438
-
439
- ### Next.js
440
-
441
- Next.js route handlers return `Response` objects, so `handle` wraps one directly. The spool
442
- endpoints and the worker script go through the proxy, which returns `undefined` for every other
443
- request. Next.js 16 calls the file `proxy.ts` and the export `proxy`. Earlier versions call them
444
- `middleware.ts` and `middleware`. The proxy and the route handlers need the same shared store.
445
-
446
- ```ts
447
- import { spool } from './spool';
448
-
449
- export function proxy(request: Request) {
450
- const { pathname } = new URL(request.url);
451
- return pathname === '/spool-sw.js' ? spool.stub() : spool.respond(request);
452
- }
453
- ```
454
-
455
- ```ts
456
- import { page } from './routes';
457
- import { spool } from './spool';
458
-
459
- export const GET = (request: Request) => spool.handle(request, page);
460
- ```
461
-
462
- ### Nuxt and Nitro
463
-
464
- h3 turns an event into a standard request with `toWebRequest`, and it sends a `Response` that a
465
- handler returns. A server middleware that returns nothing passes the request on. Put the first
466
- file in `server/middleware/` and the second in `server/routes/node/[id].ts`. Both share one store
467
- from `./store`.
468
-
469
- ```ts
470
- import { defineEventHandler, toWebRequest } from 'h3';
471
- import { spool } from './spool';
472
-
473
- export default defineEventHandler((event) => {
474
- const request = toWebRequest(event);
475
- return new URL(request.url).pathname === '/spool-sw.js' ? spool.stub() : spool.respond(request);
476
- });
477
- ```
478
-
479
- ```ts
480
- import { defineEventHandler, toWebRequest } from 'h3';
481
- import { page } from './routes';
482
- import { spool } from './spool';
483
-
484
- export default defineEventHandler((event) => spool.handle(toWebRequest(event), page));
485
- ```
486
-
487
- ### AWS Lambda
488
-
489
- Lambda gives a function a JSON event and expects a JSON result, so spool needs an adapter. Hono's
490
- `hono/aws-lambda` converts Function URL, API Gateway HTTP API and ALB events. Save the Hono app
491
- above as `hono.ts` and wrap it.
492
-
493
- ```ts
494
- import { handle } from 'hono/aws-lambda';
495
- import app from './hono';
496
-
497
- export const handler = handle(app);
498
- ```
499
-
500
- ### Azure Functions
501
-
502
- An Azure v4 HTTP function takes an `HttpRequest`, which follows the Fetch standard, and returns an
503
- `HttpResponseInit`. Route every path to it with `{*path}`.
504
-
505
- ```ts
506
- import { app, type HttpRequest, type HttpResponseInit } from '@azure/functions';
507
- import { page } from './routes';
508
- import { spool } from './spool';
509
-
510
- export async function handler(request: HttpRequest): Promise<HttpResponseInit> {
511
- const hasBody = request.method !== 'GET' && request.method !== 'HEAD';
512
- const response = await spool.handle(
513
- new Request(request.url, {
514
- method: request.method,
515
- headers: [...request.headers],
516
- body: hasBody ? await request.arrayBuffer() : undefined
517
- }),
518
- page
519
- );
520
- return {
521
- status: response.status,
522
- headers: [...response.headers],
523
- body: await response.arrayBuffer()
524
- };
525
- }
267
+ The page answers a link or a GET form in place when its content sits in a swap region. The region is the first element marked `data-spool-swap`, or else the first `<main>`:
526
268
 
527
- app.http('spool', {
528
- route: '{*path}',
529
- methods: ['GET', 'POST'],
530
- authLevel: 'anonymous',
531
- handler
532
- });
533
- ```
534
-
535
- ### Node
536
-
537
- `toNodeListener` turns a Fetch-style handler into a `node:http` listener. Its `toRequest` and
538
- `writeResponse` parts are what the frameworks below use to cross between Node and Fetch.
539
-
540
- ```ts
541
- import { createServer } from 'node:http';
542
- import { toNodeListener } from '@drupflare/spool/node';
543
- import { page } from './routes';
544
- import { spool } from './spool';
545
-
546
- export const server = createServer(toNodeListener((request) => spool.handle(request, page)));
547
- ```
548
-
549
- ### Express
550
-
551
- Express responds through its own `res`, so it has no Fetch response to wrap. `spool.respond`
552
- answers `/.spool/*` and returns `undefined` for everything else, which sends the request on to the
553
- next Express handler. A route passes its page through `spool.decorate` to get the envelope and the
554
- script tag.
555
-
556
- ```ts
557
- import express from 'express';
558
- import { toNodeListener, toRequest, writeResponse } from '@drupflare/spool/node';
559
- import { page } from './routes';
560
- import { spool } from './spool';
561
-
562
- export const app = express();
563
-
564
- app.use(toNodeListener((request) => spool.respond(request)));
565
- app.get('/spool-sw.js', (req, res) => writeResponse(res, spool.stub()));
566
- app.get('/node/:id', async (req, res) => {
567
- const request = toRequest(req, res);
568
- await writeResponse(res, await spool.decorate(request, page(request)));
569
- });
570
- ```
571
-
572
- ### Fastify
573
-
574
- Fastify's `reply.send` takes a Fetch `Response`. An `onRequest` hook answers the spool paths from
575
- the raw request before Fastify reads the body.
576
-
577
- ```ts
578
- import Fastify from 'fastify';
579
- import { toRequest } from '@drupflare/spool/node';
580
- import { page } from './routes';
581
- import { spool } from './spool';
582
-
583
- export const app = Fastify();
584
-
585
- app.addHook('onRequest', async (req, reply) => {
586
- const response = await spool.respond(toRequest(req.raw, reply.raw));
587
- if (response) return reply.send(response);
588
- });
589
- app.get('/spool-sw.js', (req, reply) => reply.send(spool.stub()));
590
- app.get('/node/:id', async (req, reply) => {
591
- const request = toRequest(req.raw, reply.raw);
592
- return reply.send(await spool.decorate(request, page(request)));
593
- });
594
- ```
595
-
596
- ### Koa
597
-
598
- Koa exposes the Node objects as `ctx.req` and `ctx.res`. Setting `ctx.respond` to `false` stops
599
- Koa from writing a second response.
600
-
601
- ```ts
602
- import Koa from 'koa';
603
- import { toRequest, writeResponse } from '@drupflare/spool/node';
604
- import { page } from './routes';
605
- import { spool } from './spool';
606
-
607
- export const app = new Koa();
608
-
609
- app.use(async (ctx, next) => {
610
- const request = toRequest(ctx.req, ctx.res);
611
- const response = ctx.path === '/spool-sw.js' ? spool.stub() : await spool.respond(request);
612
- if (!response) return next();
613
- ctx.respond = false;
614
- await writeResponse(ctx.res, response);
615
- });
616
- app.use(async (ctx) => {
617
- if (!ctx.path.startsWith('/node/')) return;
618
- const request = toRequest(ctx.req, ctx.res);
619
- ctx.respond = false;
620
- await writeResponse(ctx.res, await spool.decorate(request, page(request)));
621
- });
622
- ```
623
-
624
- ### hapi
625
-
626
- hapi keeps the Node objects at `request.raw`. Returning `h.abandon` tells it the response is
627
- already written.
628
-
629
- ```ts
630
- import Hapi from '@hapi/hapi';
631
- import { toRequest, writeResponse } from '@drupflare/spool/node';
632
- import { page } from './routes';
633
- import { spool } from './spool';
634
-
635
- export const server = Hapi.server();
636
-
637
- server.ext('onRequest', async (request, h) => {
638
- const { req, res } = request.raw;
639
- const spoolRequest = toRequest(req, res);
640
- const response =
641
- request.path === '/spool-sw.js' ? spool.stub() : await spool.respond(spoolRequest);
642
- if (!response) return h.continue;
643
- await writeResponse(res, response);
644
- return h.abandon;
645
- });
646
- server.route({
647
- method: 'GET',
648
- path: '/node/{id}',
649
- handler: async (request, h) => {
650
- const { req, res } = request.raw;
651
- const spoolRequest = toRequest(req, res);
652
- await writeResponse(res, await spool.decorate(spoolRequest, page(spoolRequest)));
653
- return h.abandon;
654
- }
655
- });
656
- ```
657
-
658
- ### NestJS
659
-
660
- On the Express adapter, `app.use` takes spool's listener and a controller method that injects
661
- `@Req()` and `@Res()` writes the page. The example needs `experimentalDecorators` in
662
- `tsconfig.json`.
663
-
664
- ```ts
665
- import { Controller, Get, Module, Req, Res } from '@nestjs/common';
666
- import { NestFactory } from '@nestjs/core';
667
- import type { Request, Response } from 'express';
668
- import 'reflect-metadata';
669
- import { toNodeListener, toRequest, writeResponse } from '@drupflare/spool/node';
670
- import { page } from './routes';
671
- import { spool } from './spool';
672
-
673
- @Controller()
674
- class NodeController {
675
- @Get('node/:id')
676
- async show(@Req() req: Request, @Res() res: Response) {
677
- const request = toRequest(req, res);
678
- await writeResponse(res, await spool.decorate(request, page(request)));
679
- }
680
- }
681
-
682
- @Module({ controllers: [NodeController] })
683
- class AppModule {}
684
-
685
- export async function bootstrap() {
686
- const app = await NestFactory.create(AppModule, { logger: false });
687
- app.use(toNodeListener((request) => spool.respond(request)));
688
- app.use('/spool-sw.js', (req: Request, res: Response) => writeResponse(res, spool.stub()));
689
- return app;
690
- }
691
- ```
692
-
693
- ### Google Cloud Functions
694
-
695
- The Functions Framework gives an HTTP function Express-style `req` and `res`, so the Node listener
696
- fits as it is.
697
-
698
- ```ts
699
- import { http } from '@google-cloud/functions-framework';
700
- import { toNodeListener } from '@drupflare/spool/node';
701
- import { page } from './routes';
702
- import { spool } from './spool';
703
-
704
- http(
705
- 'spool',
706
- toNodeListener((request) => spool.handle(request, page))
707
- );
708
- ```
709
-
710
- ---
711
-
712
- ## ๐Ÿ“Š Compatibility
269
+ ```html
270
+ <body>
271
+ <header>
272
+ <a href="/">Site</a>
273
+ </header>
274
+ <main data-spool-swap>
275
+ <h1>Course 101</h1>
276
+ </main>
277
+ </body>
278
+ ```
279
+
280
+ The page replaces the region, the element marked `data-spool-swap-sidebar` when both pages have one, and the title. It answers only public routes declared with `family`, from the rows the device holds ([Dataset Routes](ADVANCED_USAGE.md#dataset-routes)). Modified clicks, links with `target` or `download`, cross-origin links and non-GET forms go to the browser, and so does any route the device does not hold. The page writes at most 90 history entries in any 10 seconds. Past that count it swaps the content and leaves the address bar on the previous URL until the window clears.
281
+
282
+ A page with no swap element navigates by document, and no error is shown.
283
+
284
+ ---
285
+
286
+ ## Runtimes
287
+
288
+ Each runtime has its own page, with the framework context and a minimal example. The
289
+ [Adapters](docs/adapters/README.md) index lists every page with its import path.
290
+
291
+ | Runtime | Page |
292
+ | -------------------------- | ---------------------------------------------------------------------------- |
293
+ | Cloudflare Workers | [cloudflare-workers.md](docs/adapters/cloudflare-workers.md) |
294
+ | Cloudflare Pages Functions | [cloudflare-pages-functions.md](docs/adapters/cloudflare-pages-functions.md) |
295
+ | Hono | [hono.md](docs/adapters/hono.md) |
296
+ | Elysia | [elysia.md](docs/adapters/elysia.md) |
297
+ | itty-router | [itty-router.md](docs/adapters/itty-router.md) |
298
+ | Bun | [bun.md](docs/adapters/bun.md) |
299
+ | Deno | [deno.md](docs/adapters/deno.md) |
300
+ | Netlify Functions | [netlify-functions.md](docs/adapters/netlify-functions.md) |
301
+ | Netlify Edge Functions | [netlify-edge-functions.md](docs/adapters/netlify-edge-functions.md) |
302
+ | Vercel Functions | [vercel-functions.md](docs/adapters/vercel-functions.md) |
303
+ | Next.js | [nextjs.md](docs/adapters/nextjs.md) |
304
+ | Nuxt and Nitro | [nuxt-and-nitro.md](docs/adapters/nuxt-and-nitro.md) |
305
+ | Nuxt Module | [nuxt-module.md](docs/adapters/nuxt-module.md) |
306
+ | React Router | [react-router.md](docs/adapters/react-router.md) |
307
+ | Next Adapter | [next-adapter.md](docs/adapters/next-adapter.md) |
308
+ | Vite | [vite.md](docs/adapters/vite.md) |
309
+ | Vue | [vue.md](docs/adapters/vue.md) |
310
+ | Angular | [angular.md](docs/adapters/angular.md) |
311
+ | AWS Lambda | [aws-lambda.md](docs/adapters/aws-lambda.md) |
312
+ | Azure Functions | [azure-functions.md](docs/adapters/azure-functions.md) |
313
+ | Node | [node.md](docs/adapters/node.md) |
314
+ | Express | [express.md](docs/adapters/express.md) |
315
+ | Fastify | [fastify.md](docs/adapters/fastify.md) |
316
+ | Koa | [koa.md](docs/adapters/koa.md) |
317
+ | hapi | [hapi.md](docs/adapters/hapi.md) |
318
+ | NestJS | [nestjs.md](docs/adapters/nestjs.md) |
319
+ | Google Cloud Functions | [google-cloud-functions.md](docs/adapters/google-cloud-functions.md) |
320
+
321
+ Pick the page for your framework. A runtime with a Fetch `fetch(request)` handler needs only
322
+ `spool.handle`, as the [Quick Start](#quick-start) shows.
323
+
324
+ ---
325
+
326
+ ## Compatibility
713
327
 
714
328
  The gate column says how far each row is checked. **runs** means the fixture above drives the
715
329
  framework's own package. **stand-in** means a fixture calls the platform's handler with a plain
@@ -718,14 +332,14 @@ from the framework's documentation and nothing here has run it.
718
332
 
719
333
  | Framework or host | What a handler receives | Spool entry | Gate |
720
334
  | ------------------------------------ | -------------------------------------------------------------- | ------------------------------------------------------------ | -------------------------------------------- |
721
- | Express | Node `req`, `res` | `toNodeListener`, `respond`, `decorate` | runs |
335
+ | Express | Node `req`, `res` | `spoolExpress` | runs |
722
336
  | Fastify | Node `req.raw`, `reply.raw`; `reply.send` takes a `Response` | `toRequest`, `respond`, `decorate` | runs |
723
337
  | Koa | Node `ctx.req`, `ctx.res` | `toRequest`, `writeResponse` | runs |
724
338
  | hapi | Node `request.raw.req`, `request.raw.res` | `toRequest`, `writeResponse` | runs |
725
339
  | NestJS, Express adapter | Node, through Express | `app.use` with `toNodeListener`; `toRequest` in a controller | runs |
726
340
  | NestJS, Fastify adapter | Node, through Fastify | the Fastify entry | not tested |
727
341
  | AdonisJS | Node at `ctx.request.request` and `ctx.response.response` | `toRequest`, `writeResponse` in a middleware | not tested |
728
- | Hono | Fetch | `handle` | runs |
342
+ | Hono | Fetch | `spoolHono` | runs |
729
343
  | Elysia | Fetch | `respond`, `decorate` | runs |
730
344
  | itty-router | Fetch | `respond`, `decorate` | runs |
731
345
  | Oak | Fetch, through `app.handle`; published on JSR only | `handle` | not tested |
@@ -745,8 +359,8 @@ from the framework's documentation and nothing here has run it.
745
359
  | Deno, Deno Deploy | Fetch, through `Deno.serve` or `deno serve` | `handle` | stand-in |
746
360
  | Bun | Fetch | `handle` | runs, under `Bun.serve` |
747
361
  | Vercel Functions | Fetch | `handle` | stand-in |
748
- | AWS Lambda, Function URL or HTTP API | JSON event version 2.0 | `hono/aws-lambda` around a Hono app | stand-in event |
749
- | AWS Lambda, API Gateway REST API | JSON event version 1.0 | `hono/aws-lambda` around a Hono app | not tested |
362
+ | AWS Lambda, Function URL or HTTP API | JSON event version 2.0 | `@hono/aws-lambda` around a Hono app | stand-in event |
363
+ | AWS Lambda, API Gateway REST API | JSON event version 1.0 | `@hono/aws-lambda` around a Hono app | not tested |
750
364
  | Lambda@Edge | CloudFront event with header arrays | none; a converter to `Request` is needed | not tested |
751
365
  | Fastly Compute | Fetch, through `event.request` in a `fetch` listener | `event.respondWith(spool.handle(event.request, next))` | not tested |
752
366
  | Azure Functions v4 | `HttpRequest`, which follows Fetch; returns `HttpResponseInit` | convert, then `handle` | runs, with a real `HttpRequest` |
@@ -754,11 +368,11 @@ from the framework's documentation and nothing here has run it.
754
368
  | Google Cloud Run | any HTTP server on `PORT` | the Node example | not tested |
755
369
 
756
370
  A host that serves static files and runs no code, such as an object store or a plain web server, uses
757
- [Static Sites](#-static-sites).
371
+ [Static Sites](#static-sites).
758
372
 
759
373
  ---
760
374
 
761
- ## ๐Ÿงฑ Static Sites
375
+ ## Static Sites
762
376
 
763
377
  A site with no spool server can still answer repeat views from the device. It serves three files
764
378
  from one directory:
@@ -819,1241 +433,52 @@ importScripts('https://cdn.drupflare.com/spool/<version>/spool-sw.js');
819
433
 
820
434
  `importScripts` has no integrity check, so this form trusts the CDN host with the worker.
821
435
 
822
- ---
823
-
824
- ## ๐Ÿ”€ Strategies
825
-
826
- A route is answered locally in one of five ways. A `template` route stores JSON and renders it
827
- through a compiled template. An `assembly` route stores a shell with named holes, fragment markup
828
- for the holes and values for named slots. A `static` route stores a file as the server sent it. A
829
- `data` route stores JSON and answers a GET for it. A `projection` route stores JSON and renders a
830
- page from it with a function your site bundles into its own worker.
831
-
832
- ```ts
833
- import { placeholder, slot } from '@drupflare/spool/protocol';
834
- import { defineRoute } from '@drupflare/spool/server';
835
-
836
- export const about = defineRoute('/about', {
837
- strategy: 'assembly',
838
- depends: () => ['menu'],
839
- load: () => ({
840
- shell: `<!doctype html><html><head><title>About</title></head><body>${placeholder('menu')}<main>Updated ${slot('updated')}</main></body></html>`,
841
- fragments: { menu: '<nav><a href="/">Home</a></nav>' },
842
- slots: { updated: '2026-10-19' }
843
- })
844
- });
845
- ```
846
-
847
- The worker leaves a hole it cannot fill in place and refuses to render a page with a gap, so the
848
- request goes to the network instead.
849
-
850
- A page the worker renders keeps the site's security headers. The worker remembers
851
- `Content-Security-Policy` (and its report-only form), `X-Frame-Options`, `X-Content-Type-Options`,
852
- `Referrer-Policy`, `Permissions-Policy` and the three `Cross-Origin-*` policies from the network
853
- answer to each path, and sends them with every local render of it. A path with no answer of its own
854
- uses the latest set from any navigation. A page with neither goes to the network until the worker
855
- has seen one, so a local view never goes out without the site's policy.
856
-
857
- ### Records and Projections
858
-
859
- Pages that read the same data share one stored copy. `ctx.record(id, data)` stores an
860
- authoritative value once under its id, and `ctx.projection(id, data)` stores a value derived from
861
- records. Both return a reference to put in the page's data. A record's id is its invalidation tag.
862
- Invalidating `course:101` refreshes the records and projections that read it, and no others.
863
-
864
- ```ts
865
- import { defineRoute } from '@drupflare/spool/server';
866
-
867
- const courses: Record<string, string> = { '101': 'Intro', '102': 'Advanced' };
868
-
869
- export const course = defineRoute('/courses/:id', {
870
- strategy: 'template',
871
- template: {
872
- source: '<h1>{{title}}</h1><nav>{{#each menu}}<a>{{this}}</a>{{/each}}</nav>'
873
- },
874
- load: (ctx) => {
875
- const id = ctx.params['id'] ?? '';
876
- return {
877
- title: ctx.record(`course:${id}`, courses[id]),
878
- menu: ctx.projection(
879
- 'menu',
880
- Object.keys(courses).map((key) => ctx.record(`course:${key}`, courses[key]))
881
- )
882
- };
883
- }
884
- });
885
- ```
886
-
887
- The server sends only the records and projections the device does not already hold at a current
888
- version. A sync for the next page transfers the entries that page reads and that changed.
889
-
890
- ### Templates
891
-
892
- Templates use Mustache-shaped syntax with no expressions, so the worker needs no `eval`.
893
-
894
- | Syntax | Meaning |
895
- | ------------------------------------ | -------------------------------------------------------- |
896
- | `{{field}}` | the field, HTML-escaped; `a.b` reaches into objects |
897
- | `{{{field}}}` | the field as markup; allowed only for names in `trusted` |
898
- | `{{#each list}}...{{/each}}` | repeats for each item; `{{this}}` and `{{@index}}` work |
899
- | `{{#if field}}...{{else}}...{{/if}}` | renders a branch on whether the field is truthy |
900
- | `{{! note }}` | a comment that renders nothing |
901
-
902
- ```ts
903
- import { defineRoute } from '@drupflare/spool/server';
904
-
905
- export const news = defineRoute('/news/:slug', {
906
- strategy: 'template',
907
- template: {
908
- source:
909
- '<article><h1>{{title}}</h1>{{#if byline}}<p>{{byline}}</p>{{else}}<p>Staff</p>{{/if}}<div>{{{body}}}</div><ul>{{#each tags}}<li>{{@index}} {{this}}</li>{{/each}}</ul></article>',
910
- trusted: ['body']
911
- },
912
- depends: (ctx) => [`news:${ctx.params['slug']}`],
913
- load: () => ({
914
- title: 'Hello',
915
- byline: null,
916
- body: '<p>Trusted markup</p>',
917
- tags: ['a', 'b']
918
- })
919
- });
920
- ```
921
-
922
- A template that names a field it does not trust for markup fails when the route is declared.
923
-
924
- A public `template` route can carry its own data in the page. `inlineMaxBytes` on `createSpool`
925
- sets the largest artifact, in bytes, that `handle` and `decorate` write into the HTML as an
926
- escaped `<script type="application/spool+json">` element. The worker reads it from the response
927
- and skips the `/.spool/artifact` request. It is 0, which is off, because every network view of the
928
- page carries the element, used or not. The page the visitor sees and the shadow comparison ignore
929
- it.
930
-
931
- ### Static Files
932
-
933
- A `static` route has no `load` and no template. The application serves the file as it always did,
934
- and `handle` or `decorate` marks the response for the worker, which keeps the bytes. A tag in
935
- `depends` ties the file to a deploy, and `spool.invalidate` after one makes devices fetch the
936
- file again. A static route is always public and is always answered from the device, so it takes no
937
- `scope`, `persist` or `mode`. A query string makes a different file: `/app.js?v=2` is not
938
- answered with the copy of `/app.js`, and a query the worker has not stored goes to the network.
939
-
940
- ```ts
941
- import { defineRoute } from '@drupflare/spool/server';
942
-
943
- export const about = defineRoute('/about', {
944
- strategy: 'static',
945
- depends: () => ['site'],
946
- freshness: { maxStale: 3600, mode: 'soft' }
947
- });
948
- ```
949
-
950
- ### Projection and Data Routes
951
-
952
- A `data` route answers a GET for JSON, such as an API read, from the stored value, so the page's
953
- `fetch` makes no request. The worker answers with `JSON.stringify` of what `load` returned, which
954
- means the application's own handler has to return the same JSON. A navigation to the URL still goes
955
- to the network.
956
-
957
- A `projection` route needs a renderer. Spool ships no bundler: write a worker with `serve` from
958
- `@drupflare/spool/sw`, bundle it with your own toolchain, and pass the bundle to `createSpool` as
959
- `worker`. The server puts the config in front of your bundle as `self.__SPOOL_CONFIG__`, and
960
- `spool.config` is the same object as JSON if you would rather inline it at build time. A route with
961
- no renderer is refused by the worker and the network answers.
962
-
963
- ```ts
964
- import { defineRoute } from '@drupflare/spool/server';
965
-
966
- export const course = defineRoute('/api/courses/:id', {
967
- strategy: 'data',
968
- depends: (ctx) => [`course:${ctx.params['id']}`],
969
- load: (ctx) => ({ id: ctx.params['id'], title: 'Intro' }),
970
- provision: { reuse: 0.8, futureViews: 6 }
971
- });
972
-
973
- export const roster = defineRoute('/rosters/:id', {
974
- strategy: 'projection',
975
- depends: (ctx) => [`roster:${ctx.params['id']}`],
976
- load: (ctx) => ({ id: ctx.params['id'], names: ['Ada', 'Bo'] })
977
- });
978
- ```
979
-
980
- ```ts
981
- import { escapeHtml } from '@drupflare/spool/protocol';
982
- import { serve } from '@drupflare/spool/sw';
983
-
984
- declare const CONFIG: string;
985
-
986
- serve(JSON.parse(CONFIG), {
987
- '/rosters/:id': (artifact) => {
988
- const { names } = (artifact.body as { data: { names: string[] } }).data;
989
- return `<!doctype html><html><head><title>Roster</title></head><body><ul>${names.map((name) => `<li>${escapeHtml(name)}</li>`).join('')}</ul></body></html>`;
990
- }
991
- });
992
- ```
993
-
994
- The server also decides whether to send an artifact at all. It weighs the bytes it would send,
995
- records included, against what a local view saves: reuse times future views times the bytes a view
996
- avoids, less the cost of the request. When that is not positive it answers `204` with
997
- `Spool-Provision: declined`, and the worker leaves that path to the network until it restarts.
998
- The priors are scenario assumptions: set them from your own telemetry in `provision`, or per route
999
- as above. A `data` route weighs against its own size, so its reuse times future views must exceed
1000
- one. The defaults of 0.5 and 2 give exactly one, which breaks even, and the route is left out.
1001
-
1002
- ### Query Routes
1003
-
1004
- A URL with a query string goes to the network. A public `template`, `projection` or `data` route
1005
- can declare the query strings it answers, and the device then answers a new filter, sort or page of
1006
- a collection it already holds with no request.
1007
-
1008
- ```ts
1009
- import { datasetProof, defineRoute } from '@drupflare/spool/server';
1010
-
1011
- const people = [
1012
- { name: 'Ada', department: 'math', age: 36 },
1013
- { name: 'Bo', department: 'cs', age: 25 }
1014
- ];
1015
-
1016
- export const directory = defineRoute('/people', {
1017
- strategy: 'template',
1018
- template: {
1019
- source:
1020
- '<h1>{{#if query}}Page {{query.page}} of {{query.pages}}{{else}}Everyone{{/if}}</h1><ul>{{#each people}}<li>{{name}}</li>{{/each}}</ul>'
1021
- },
1022
- depends: () => ['people'],
1023
- load: async () => ({ people, proof: await datasetProof(people) }),
1024
- query: {
1025
- list: 'people',
1026
- filter: { department: 'string', age: 'number' },
1027
- sort: ['name', 'age'],
1028
- pageSize: 20
1029
- },
1030
- provision: { reuse: 0.8, futureViews: 6 }
1031
- });
1032
- ```
1033
-
1034
- `list` names the key of the loaded data that holds the whole collection, an array of flat objects.
1035
- Each key of `filter` is a query parameter that keeps the items whose field of the same name equals
1036
- it, read as a `string` or a `number`. A repeated parameter matches any of its values, and different
1037
- parameters combine with and. `sort` lists the fields the `sort` parameter may name, and `-name`
1038
- sorts descending. Ties keep the stored order. Strings compare by UTF-16 code unit, with no locale
1039
- and no case folding, so the application's own sort has to agree. A database that compares bytes,
1040
- such as SQLite by default, agrees except for characters above U+FFFF, and a case-insensitive or
1041
- locale collation does not. `pageSize` sets the items per page and `page` counts from 1; without it
1042
- the route is one page of everything. A route with `static` or `assembly` strategy cannot declare
1043
- `query`. A route with `principal` scope can declare it only with `query.dataset`, which
1044
- [Datasets](#-datasets) describes.
1045
-
1046
- `load` returns the array at `list` and a `proof` made by `datasetProof`. The server refuses to
1047
- build the artifact when the proof is missing or does not match the array. The worker computes the
1048
- proof again from the stored copy before it answers a query. The proof shows that the device holds
1049
- the array the server sent. That the array is the whole collection is the route's declaration:
1050
- `list` has to hold every item the application's own answer could draw from, or a filter returns
1051
- fewer rows than the server would.
1052
-
1053
- The template, or the renderer of a `projection` route, receives the stored data with the array
1054
- replaced by the page's items and one added key, `query`, holding `{ total, page, pages }`. `total`
1055
- counts the items after filtering. A `data` route answers with that same object as JSON. The URL
1056
- without a query string renders the stored data as it is and has no `query` key, which is what
1057
- `{{#if query}}` tests above.
1058
-
1059
- The worker answers locally only when the artifact for the path is held and fresh under the route's
1060
- freshness contract, the proof matches and the query is one the route declares. Everything else goes
1061
- to the network as before: a parameter the route does not name, a number filter that is not a
1062
- finite number, a `sort` given twice or not listed, a `page` that is not a positive integer or
1063
- lies past the last page, and an item that is not an object. A copy that fails its proof is dropped
1064
- and fetched whole again. The worker never stores anything under a URL with a query string, and in
1065
- shadow mode it compares its render with the network answer for the same URL.
1066
-
1067
- A query route's artifact is the whole collection, so it is large. The provisioning rule weighs it
1068
- like any other artifact, and `provision` on the route sets the priors. A collection repays its
1069
- transfer after `entryBreakEven(bytes, documentBytes)` local answers per device: a 40 KB collection
1070
- that replaces 17 KB documents needs 2.3 (L2). Pages of one visit come from one held copy until a
1071
- sync replaces it, so paging never mixes two versions on a device.
1072
-
1073
- ---
1074
-
1075
- ## โณ Freshness Contracts
1076
-
1077
- Each route declares how long a stored page may go without validation. `maxStale` is in seconds.
1078
-
1079
- - **hard:** once `maxStale` passes, the worker validates the page with the server before it
1080
- shows it. Offline, the route's `offline` rule decides.
1081
- - **soft:** the worker shows the stored page at once and syncs afterward. A soft bound is a
1082
- stale-while-revalidate window and promises nothing about how fresh the shown page is.
1083
-
1084
- Public routes default to 15 minutes soft, and routes with `scope: 'principal'` to five minutes
1085
- hard; [Authenticated Data](#-authenticated-data) covers how their pages are stored. `offline`
1086
- defaults to `serve` for public routes and `refuse` for principal routes. A soft route may set `sync: 'deferred'` to queue its refresh
1087
- for the next network request that happens anyway.
1088
-
1089
- ```ts
1090
- import { defineRoute } from '@drupflare/spool/server';
1091
-
1092
- export const stock = defineRoute('/stock/:sku', {
1093
- strategy: 'template',
1094
- template: { source: '<p>{{sku}}: {{count}}</p>' },
1095
- depends: (ctx) => [`stock:${ctx.params['sku']}`],
1096
- load: (ctx) => ({ sku: ctx.params['sku'], count: 3 }),
1097
- freshness: { maxStale: 60, mode: 'hard' },
1098
- offline: 'refuse'
1099
- });
1100
-
1101
- export const notice = defineRoute('/notices/:id', {
1102
- strategy: 'template',
1103
- template: { source: '<p>{{text}}</p>' },
1104
- depends: (ctx) => [`notice:${ctx.params['id']}`],
1105
- load: () => ({ text: 'Closed on Friday' }),
1106
- freshness: { maxStale: 3600, mode: 'soft' },
1107
- sync: 'deferred'
1108
- });
1109
- ```
1110
-
1111
- The worker syncs the smallest set of changed dependencies the next page needs. One worker
1112
- leads the sync for every tab of an origin. A response to a request that happens anyway can carry
1113
- invalidations, so a sync does not always need a request of its own.
1114
-
1115
- ---
1116
-
1117
- ## ๐Ÿ’พ Datasets
1118
-
1119
- A route that lists thousands of rows, such as a staff directory, makes a device fetch the whole
1120
- collection again after each change. A dataset lets the device hold the rows and receive only the
1121
- rows that changed. The device seeds a dataset once, then asks for a patch from the cursor it holds.
1122
- A patch carries the rows to put, the keys to delete and a proof the device checks before it stores
1123
- anything.
1124
-
1125
- `defineDataset(name, declaration)` from `@drupflare/spool/server` declares a dataset, and
1126
- `createSpool({ datasets })` serves it at `GET /.spool/dataset`. The declaration gives the
1127
- functions `key`, `seed`, `read`, `changes`, `horizon` and `head`, a `schema` string, and the
1128
- `maxRows` and `maxBytes` of a seed. A dataset has its own integer cursor, which your backend holds.
1129
- It does not use the site's version vector. `changes(since)` returns the net change after a cursor
1130
- and decides which rows are in scope. `read(ids)` is a keyed read that never runs a route's `load`.
1131
- A device whose cursor is below `horizon`, or whose `schema` differs, seeds again.
1132
-
1133
- The proof is the number of rows and the sum of every row's hash. A sum does not depend on order, so
1134
- a patch moves it by the rows it touches. The device applies a seed or a patch to a staging copy,
1135
- compares the result with the proof, and writes the rows and their metadata in one transaction. A
1136
- failed proof clears that dataset and seeds it once more. The proof catches corruption and a
1137
- half-applied patch. It does not defend against a server that sends rows and a digest that agree.
1138
-
1139
- A route reads a dataset in two ways. `query.dataset` names it in place of `list`: the worker takes
1140
- the held rows in key order and applies the route's filter, sort and page, and goes to the network
1141
- while the dataset is not held. `ctx.row(dataset, key)` puts a reference to one row in the route's
1142
- data. The reference resolves when the page renders, so a changed row needs no new artifact.
1143
-
1144
- A route with `query.dataset` may also declare `query.plan`, a read-only plan written as JSON with
1145
- joins, conditions, groups and aggregates over held datasets. It replaces `filter`, `sort` and
1146
- `pageSize`. The worker compares values the way SQLite compares a column with no declared type, and
1147
- sends the query to the network when a plan could give another answer than SQLite would. The server
1148
- never evaluates a plan.
1149
-
1150
- A dataset declared with `provision: 'auto'` reaches a device only while it beats two controls: the
1151
- held page under v0.3, and one plain resource that a browser cache revalidates with an `ETag`.
1152
- Otherwise the seed request gets `409` with `Spool-Provision: declined`, and the worker holds
1153
- nothing and does not ask again in that session. The default, `'always'`, answers every seed. The worker also
1154
- refuses a seed whose rows would not fit the headroom that `navigator.storage.estimate()` reports,
1155
- and the page runtime asks the browser for persistent storage once the first dataset is held.
1156
-
1157
- On synthetic rows (L1, no real site measured), one changed row of 500 costs 228 bytes as a patch and
1158
- 19,981 bytes as a v0.3 list refetch, and the first load costs 867 bytes more than under v0.3
1159
- ([docs/impact.md](docs/impact.md#dataset-patches)). [docs/protocol.md](docs/protocol.md#datasets)
1160
- has the declaration rules, the wire format, query plans and the device store.
1161
-
1162
- ### Principal Datasets
1163
-
1164
- Principal datasets are off by default and unreviewed: no external review of this feature has
1165
- happened. They hold rows that differ for each signed-in principal. Two switches turn them on, and
1166
- both are needed. `defineDataset(name, declaration, { principalDatasets: true })` accepts
1167
- `scope: 'principal'`, and `createSpool({ principalDatasets: true, identity })` serves the dataset.
1168
- A principal dataset declares `maxStaleMs`, which has no default, and cannot use
1169
- `provision: 'auto'`.
1170
-
1171
- The server takes the identity from the request and passes it to every dataset function as
1172
- `ctx.principal`. Which rows a principal may see is your code, and spool does not check what the
1173
- functions return. The device keeps the rows sealed in the principal's partition, as
1174
- [Authenticated Data](#-authenticated-data) describes, and wipes them with it. Held rows answer a
1175
- query for `maxStaleMs` after the last verified sync, so a principal removed on the server can read
1176
- held rows for up to that long. A principal route can declare `query` only with `query.dataset` and
1177
- no plan, and a route and its dataset need the same scope.
1178
- [docs/security.md](docs/security.md#principal-datasets) lists the threat modes and the test that
1179
- holds each one.
1180
-
1181
- ---
1182
-
1183
- ## ๐Ÿ” Authenticated Data
1184
-
1185
- Set `scope: 'principal'` on a route that depends on who is signed in, and give `createSpool` an
1186
- `identity` option. The server sends `Spool-Partition`, an HMAC of the principal under your secret,
1187
- so the browser can tell users apart without learning who they are. The worker keeps each partition
1188
- in its own IndexedDB database with the artifacts, records, projections and sync state sealed with
1189
- AES-GCM. Each value is bound to its partition, kind and id, so a row copied into another slot does
1190
- not decrypt.
1191
-
1192
- `retention` picks the key. `persistent` is a non-extractable key in IndexedDB that survives a
1193
- restart and is deleted on logout or a change of partition. `session`, the default, is a key held in
1194
- the worker's memory, which the browser discards when it stops the worker. Deleting a key makes what
1195
- it sealed unreadable, which is crypto-shredding of data at rest. It does not protect against script
1196
- running on your origin, which can ask the worker for whatever the page can. The ids and versions
1197
- that serve as storage keys stay readable, so use opaque ids for anything sensitive.
1198
-
1199
- A principal route defaults to a five-minute hard bound and `offline: 'refuse'`. Revocation rides
1200
- the same bound: tag a page with the permission it needs, call `spool.invalidate` when the
1201
- permission goes, and the first required sync past the bound finds the page invalid.
1202
-
1203
- A different `Spool-Partition` wipes the partition before it. A login or account-switch handler has
1204
- to send the new value itself with `spool.partition(principal)`, because the request it answers
1205
- still carries the old identity. A sync answer carries `Spool-Partition` for the request's identity
1206
- too, so a sign-in the worker never saw ends at the first required sync, no later than the route's
1207
- `maxStale` plus the next view. Paths in `logout` are wiped when the request reaches the worker,
1208
- before it is sent, so logout empties every tab even with the network down. The worker first sends
1209
- the same-origin posts it holds in `beacons`, while they still carry the session that is ending,
1210
- and drops any that do not get through. The page runtime
1211
- dispatches a `spool:cleared` event on `window` in every tab when that happens, so a page can blank
1212
- what it shows. A response can also send
1213
- `Spool-Clear`: `*` wipes everything, a partition id wipes one. `decorate(request, response, { clear:
1214
- ['*'], hardReset: true })` adds `Clear-Site-Data: "storage"` as well, the browser's own reset
1215
- of everything the origin holds, service worker included. Its Safari support is unverified, so rely
1216
- on the worker's wipe.
1217
-
1218
- ```ts
1219
- import { createSpool, defineRoute } from '@drupflare/spool/server';
1220
- import { memoryStore } from '@drupflare/spool/stores';
1221
-
1222
- export const grades = defineRoute('/grades', {
1223
- strategy: 'template',
1224
- scope: 'principal',
1225
- retention: 'persistent',
1226
- template: { source: '<h1>Grades for {{owner}}</h1>' },
1227
- depends: (ctx) => [`perm:${ctx.identity?.principal}`, `grades:${ctx.identity?.principal}`],
1228
- load: (ctx) => ({ owner: ctx.identity?.principal })
1229
- });
1230
-
1231
- export const spool = createSpool({
1232
- routes: [grades],
1233
- store: memoryStore(),
1234
- identity: {
1235
- secret: 'a long random value kept on the server',
1236
- resolve: (request) => {
1237
- const user = /user=(\w+)/.exec(request.headers.get('cookie') ?? '')?.[1];
1238
- return user ? { principal: user } : null;
1239
- }
1240
- },
1241
- logout: ['/logout']
1242
- });
1243
-
1244
- export async function login(user: string): Promise<Response> {
1245
- return new Response('Signed in', {
1246
- headers: {
1247
- 'set-cookie': `user=${user}; Path=/`,
1248
- 'spool-partition': await spool.partition(user)
1249
- }
1250
- });
1251
- }
1252
- ```
1253
-
1254
- A sync that asks for everything changed lists the tags of every change, so tags for principal data
1255
- should be opaque too. The worker syncs a principal page with `need`, which names only the tags
1256
- that page reads.
1257
-
1258
- ---
1259
-
1260
- ## ๐Ÿ“ Declared Writes
1261
-
1262
- Declared writes are off by default and unreviewed: no external review of this feature has
1263
- happened. A declared write queues one kind of edit on the device and sends it at a network moment.
1264
- The last value for a key wins. `createSpool({ identity, writes: { enabled: true, classes } })`
1265
- turns them on. With the flag off the worker queues nothing, the page calls answer `writes-off`, and
1266
- `PUT /.spool/write` answers `403`.
1267
-
1268
- `defineWrite` from `@drupflare/spool/server` declares a class. Its `scope` has to be `principal`,
1269
- and its `path` a declared principal route. `key` says which key this principal may write, and
1270
- `read` gives that key's current version. `apply` writes the value only when the key is still at the
1271
- version the device edited from, in one atomic step, and returns the new version or the conflict.
1272
- `maxBytes` and `maxPending` bound one value and the queue. The only `kind` is `last-value`, so a
1273
- declaration cannot queue a comment, a cart, a payment, a counter, an inventory change or an audit
1274
- record.
1275
-
1276
- The page calls `window.spool.write(name, key, value, version)`, `window.spool.flush()`,
1277
- `window.spool.writes()` and `window.spool.resolve(name, key, 'local' | 'server')`. Each answers
1278
- `{ ok: true }` with its result or `{ ok: false, code, message }`, and none throws. Edits of one key
1279
- replace each other, so several edits cost one request. A flush sends `PUT /.spool/write` with the
1280
- version in `If-Match` and a `Spool-Write-Id` that stays the same across retries. The page runtime
1281
- flushes when the page is hidden. A stale version gets `412`, and the record waits as `conflicted`
1282
- until `resolve` settles it.
1283
-
1284
- The queue is sealed in the principal's partition and deleted with it. A value that no flush has
1285
- sent is lost at logout, so call `window.spool.flush()` before the logout navigation. The server
1286
- remembers recent write ids in memory, 10,000 by default, so a site with several instances stores
1287
- `ctx.writeId` with the value in `apply`. [docs/protocol.md](docs/protocol.md#declared-writes) has
1288
- the request and its answers, and [docs/security.md](docs/security.md#declared-writes) the limits.
1289
-
1290
- ---
1291
-
1292
- ## ๐Ÿ“š Catalogue
1293
-
1294
- A catalogue sends the data of pages a visitor may open next, so that opening one makes no
1295
- request. An entry costs its bytes whether or not the page is opened, and an opened one saves a
1296
- document request, so an entry pays only when enough visitors open it. The break-even share is
1297
- `k * E / (k * D + o)`, where `E` is the entry's compressed bytes, `D` the bytes of the document
1298
- it replaces, `k` the joules to move a byte and `o` the origin request a view avoids. An entry the
1299
- size of the document needs 96% of visitors to open it, which no page gets. An entry that is only
1300
- the data for a template the device already holds is a few hundred bytes to a few kilobytes and
1301
- pays at a few percent. On the `payload` reading, the rendered fields of a node were a median 473
1302
- bytes compressed on 19 pages from two sites (L1, `measurements/payload`), against a document of
1303
- about 17,000 bytes. References such as media and terms are left out of that figure, so a real entry
1304
- is larger. An entry is data for a `template`, `projection` or `data` route and never HTML.
1305
-
1306
- The site supplies the candidates. A route's `catalogue` returns the pages a visitor on it may open
1307
- next, each with the share expected to. Take the share from your own `nextShare` over the sink
1308
- (each `spool.view` names the route the visitor came from), from link position, or from a fixed
1309
- prior. Pass `{ first: true }` to `nextShare` to count only first views, which is what an entry
1310
- saves: the share over all views ran 1.5 to 6.6 times high in the bins that decide an entry in a replay
1311
- of public visitors ([docs/impact.md](docs/impact.md#next-page-prediction)). `documentBytes` on the entry's route says what a hit saves, and the provisioning prior is
1312
- used when it is absent.
1313
-
1314
- A share taken from counts can carry `n`, the observations behind it. `catalogueConfidence` on
1315
- `createSpool` is a number of standard deviations, and with it above 0 an entry whose guess has an
1316
- `n` is judged on the lower confidence bound of its `p`, so a share seen on 5 visits stops counting
1317
- as one seen on 5,000. It defaults to 0, which judges every guess on its `p`. `NavModel` in
1318
- `@drupflare/spool/protocol` keeps first-order counts with decay and a prior and returns a `p` with
1319
- the weight to pass as `n`. In a replay of 989,818 public visitors, a model like that earned more
1320
- per provisioned kilobyte than the most popular page
1321
- ([docs/impact.md](docs/impact.md#next-page-prediction)).
1322
-
1323
- ```ts
1324
- import { createSpool, defineRoute } from '@drupflare/spool/server';
1325
- import { memoryStore } from '@drupflare/spool/stores';
1326
-
1327
- export const home = defineRoute('/', {
1328
- strategy: 'template',
1329
- template: { source: '<h1>Courses</h1>' },
1330
- load: () => ({}),
1331
- catalogue: () => [
1332
- { path: '/courses/101', p: 0.3 },
1333
- { path: '/courses/102', p: 0.1 }
1334
- ]
1335
- });
1336
-
1337
- export const course = defineRoute('/courses/:id', {
1338
- strategy: 'template',
1339
- template: { source: '<h1>{{title}}</h1>' },
1340
- depends: (ctx) => [`course:${ctx.params['id']}`],
1341
- load: (ctx) => ({ title: `Course ${ctx.params['id']}` }),
1342
- documentBytes: 17_000
1343
- });
1344
-
1345
- export const spool = createSpool({
1346
- routes: [home, course],
1347
- store: memoryStore(),
1348
- catalogueBytes: 16_384
1349
- });
1350
- ```
1351
-
1352
- The server sends an entry only when its share reaches its own break-even, best expected net first,
1353
- until `catalogueBytes` is spent. The cap counts compressed bytes and defaults to 32 KiB. It builds
1354
- at most 32 pages for one request, measures each entry with Brotli where the runtime has it and
1355
- gzip where it does not, and skips a `static` or `assembly` route. A request with `Save-Data: on`
1356
- gets no entries.
1357
-
1358
- The ask rides the beacon batch the worker already sends. A due ask also makes the next network
1359
- moment send that batch, when a view is held to carry it, so a visit of two or three pages gets its
1360
- entries without waiting for 20 views or an hour. That costs one `POST /.spool/beacon` for each
1361
- catalogue page, device and day, sent at a request that happens anyway. A local view never sends
1362
- it, and a moment with nothing held sends nothing. The worker stores the entries like provisioned
1363
- pages, and the first view of an unopened one is reported with the reason `catalogue`. `catalogueFigures` reads the views and the worker's ledger for the entries sent
1364
- and used, the hit share, and the bytes sent against the bytes saved. Until shadow views measure
1365
- the shares, the saving is a scenario.
1366
-
1367
- A hit saves the document request and nothing else. On the Drupal pages measured (75 first views of
1368
- internal pages on 10 sites, L1, `measurements/trim`), a view of a page still fetched that page's
1369
- new images and stylesheet and script aggregates: only 3% of those first views moved nothing but
1370
- the document. A catalogue view takes the document off the wire, and the rest of the page is
1371
- the site's own HTTP cache and [asset trimming](#-trimmed-stylesheets).
1372
-
1373
- A `principal` route can be an entry. The worker then names every catalogue entry no page has
1374
- opened at each sync of a principal page, so a revoked one is dropped at the first required sync.
1375
- The request that asks for a catalogue names the path of the page the visitor is on, including one
1376
- the worker showed locally, so a route that should not tell the server leaves `catalogue` off.
1377
- [docs/security.md](docs/security.md#catalogue-entries) covers both.
1378
-
1379
- ---
1380
-
1381
- ## ๐Ÿ“ก Analytics Batching
1382
-
1383
- A page that keeps analytics scripts posting to an endpoint makes a request on every view, which
1384
- defeats a local view. List the same-origin POST paths in `beacons`. The worker answers each post
1385
- with a `204` and queues it in IndexedDB. At the next network request that happens anyway it sends
1386
- the post to the URL the page meant, with its method, content type, query and body, so the vendor
1387
- receives what the page sent. Spool's own events go to `/.spool/beacon` in one batch, where a
1388
- `sink` receives them. A held post goes out on its own at most once every 24 hours, when the oldest
1389
- is a day old. A body over 64 KiB, counted in bytes, or a binary body goes to the network at once.
1390
- Firefox does not route `fetch` calls made with `keepalive: true` through the worker, so a script
1391
- that posts that way is not queued there.
1392
-
1393
- ```ts
1394
- import { createSpool, jsonLinesSink } from '@drupflare/spool/server';
1395
- import { memoryStore } from '@drupflare/spool/stores';
1396
- import { routes } from './routes';
1397
-
1398
- export const spool = createSpool({
1399
- routes,
1400
- store: memoryStore(),
1401
- beacons: ['/cdn-cgi/rum'],
1402
- sink: jsonLinesSink((line) => console.log(line))
1403
- });
1404
- ```
1405
-
1406
- `jsonLinesSink` writes one JSON line per event. `analyticsEngineSink` writes to a Workers
1407
- Analytics Engine dataset, splitting a long body across data points so nothing is cut off, and
1408
- `memorySink` keeps events for tests. The worker also queues one
1409
- `spool.view` event per page view with the tier, the reason, the staleness and the route the
1410
- visitor came from (a pattern or `external`, never a URL). `viewsOf` reads
1411
- those events back. Set `views: false` to turn that off. `workSeries` records how long each route's
1412
- handler took, for peak-work reports.
1413
-
1414
- ### Collectors
1415
-
1416
- A `collectors` entry holds requests to another origin, such as an analytics host. The page gets an
1417
- empty answer at once and the request leaves at a network moment, inside `maxAgeMs`. An event past
1418
- the cap is dropped, never sent late, and each drop is counted and written to the sink as
1419
- `/.spool/dropped`. With `delivery: 'replay'` the worker sends the held request to the vendor with no
1420
- cookies, so identity has to be in the payload, and `stamp` writes the event's time into a query
1421
- parameter for a vendor that takes one. With `delivery: 'sink'` the server half forwards it.
1422
-
1423
- GA4's browser collector takes no backdating. The Measurement Protocol accepts events with their
1424
- own timestamps up to 72 hours old and wants them within 48 hours to attribute them, and it needs an
1425
- API secret the page must not hold. `GA4_COLLECTOR` holds the `g/collect` hit for 48 hours and
1426
- `ga4Sink` sends it as a Measurement Protocol request carrying the original `timestamp_micros`.
1427
- `ga4Sink` checks its requests against the rules spool knows (`mpViolations`); set `debug: true` to
1428
- send them to `/debug/mp/collect` with your own measurement id and secret. Spool holds events
1429
- whether or not the visitor consented, so gate the analytics tag itself, and ask your counsel
1430
- whether holding an event in IndexedDB needs consent where you operate. Programmatic ads are never
1431
- held, replayed or answered locally.
1432
-
1433
- ```ts
1434
- import { GA4_COLLECTOR } from '@drupflare/spool/protocol';
1435
- import { combineSinks, createSpool, ga4Sink, jsonLinesSink } from '@drupflare/spool/server';
1436
- import { memoryStore } from '@drupflare/spool/stores';
1437
- import { routes } from './routes';
1438
-
1439
- export const spool = createSpool({
1440
- routes,
1441
- store: memoryStore(),
1442
- collectors: [
1443
- GA4_COLLECTOR,
1444
- {
1445
- vendor: 'matomo',
1446
- match: 'https://stats.example.edu/matomo.php',
1447
- maxAgeMs: 23 * 3_600_000,
1448
- delivery: 'replay',
1449
- stamp: { param: 'cdt', kind: 'datetime-utc' }
1450
- }
1451
- ],
1452
- sink: combineSinks(
1453
- jsonLinesSink((line) => console.log(line)),
1454
- ga4Sink({ measurementId: 'G-XXXXXXXXXX', apiSecret: 'kept on the server' })
1455
- )
1456
- });
1457
- ```
1458
-
1459
- ---
1460
-
1461
- ## ๐Ÿงท Third-Party Files
1462
-
1463
- A view that loads a file from another origin is not zero-network. The page reports, when it is
1464
- hidden, how many cross-origin resources it loaded and from which origins (origins only, never
1465
- paths). `thirdPartyReport` reads those reports from `viewsOf` and ranks the origins by the views
1466
- each one alone blocks, with a class and a remedy from a short table you can replace. The counts
1467
- are an upper bound on network requests, since a cross-origin timing entry says nothing about cache
1468
- hits without `Timing-Allow-Origin`.
1469
-
1470
- The opt-in `thirdParty` cache removes the request for files that name their own bytes. List the
1471
- origins in `allow`. The worker learns a file from the first view and keeps it only when the URL
1472
- carries a hash, a version or a `/v12/` segment, or the origin marks it immutable. A file versioned
1473
- only by a query parameter is checked again at a network moment once it is a day old. It keeps
1474
- nothing else, and nothing it cannot read: an opaque response counts about 7 MB against quota in
1475
- Chromium, so `allowOpaque` is off by default. WebKit does not send font requests from a stylesheet
1476
- through the worker, so fonts stay on the network there.
1477
-
1478
- ```ts
1479
- import { thirdPartyReport } from '@drupflare/spool/report';
1480
- import { createSpool, memorySink, viewsOf } from '@drupflare/spool/server';
1481
- import { memoryStore } from '@drupflare/spool/stores';
1482
- import { routes } from './routes';
1483
-
1484
- const sink = memorySink();
1485
-
1486
- export const spool = createSpool({
1487
- routes,
1488
- store: memoryStore(),
1489
- sink,
1490
- thirdParty: {
1491
- allow: ['https://fonts.gstatic.com', 'https://cdn.example.edu'],
1492
- maxEntries: 32
1493
- }
1494
- });
1495
-
1496
- export const report = () => thirdPartyReport(viewsOf(sink.events));
1497
- ```
1498
-
1499
- ---
1500
-
1501
- ## โšก Subresource Handling
1502
-
1503
- By default the worker answers an immutable same-origin asset from CacheStorage. `assets` changes
1504
- that. `passthrough` leaves assets to the browser's cache and never touches them. `static-routing`
1505
- asks the browser through `InstallEvent.addRoutes` to send only navigations, beacons and routed
1506
- paths to the worker and the rest of the origin to the network, and acts as `passthrough` where the
1507
- API does not exist: Chromium has it, Playwright's WebKit has it on macOS and not on Linux, and
1508
- Firefox does not. Static Routing keeps the requests it does not route out of the worker's fetch
1509
- handler, and neither option changes server-side Worker invocations. Neither is measured to save
1510
- energy, and [docs/impact.md](docs/impact.md) holds no reading for them. Under Static Routing a same-origin write no longer reaches the worker,
1511
- so it is not a network moment and its response cannot carry invalidations; navigations still are.
1512
-
1513
- ```ts
1514
- import { createSpool } from '@drupflare/spool/server';
1515
- import { memoryStore } from '@drupflare/spool/stores';
1516
- import { routes } from './routes';
1517
-
1518
- export const spool = createSpool({
1519
- routes,
1520
- store: memoryStore(),
1521
- assets: 'static-routing'
1522
- });
1523
- ```
1524
-
1525
- ---
1526
-
1527
- ## ๐Ÿ” Shadow Mode
1528
-
1529
- In shadow mode the worker decides whether it could serve a page locally, renders it, still fetches
1530
- the page from the server, compares the two with `normalise` and reports the result. It never
1531
- serves the local page. Use it to measure eligibility and mismatches on real traffic before turning
1532
- serving on. A `static` route has no shadow comparison.
1533
-
1534
- ```ts
1535
- import { createSpool } from '@drupflare/spool/server';
1536
- import { memoryStore } from '@drupflare/spool/stores';
1537
- import { routes } from './routes';
1538
-
1539
- export const shadowed = createSpool({ routes, store: memoryStore(), mode: 'shadow' });
1540
- export const canary = createSpool({ routes, store: memoryStore(), shadowSample: 0.05 });
1541
- ```
1542
-
1543
- `mode: 'shadow'` applies to every route that does not name its own `mode`. `shadowSample` keeps a
1544
- share of views in shadow while the rest are served, which gives a standing canary. A mismatch
1545
- names its route, so a route whose template and application disagree shows up before a visitor
1546
- sees it. The worker records the shadow comparison for navigations only. Subresources and other
1547
- requests keep the browser's own path.
1548
-
1549
- ---
1550
-
1551
- ## ๐Ÿ”Œ Proxy Mode
1552
-
1553
- `spoolProxy` fronts an origin that cannot load the library, such as PHP, Drupal with no module,
1554
- or anything else that speaks HTTP. It returns a Fetch handler. The proxy answers `/.spool/*` and
1555
- `/spool-sw.js`. Every other request goes to the origin with its method, headers and body, and a
1556
- redirect is passed on, not followed. A page the proxy may keep is decorated like a `createSpool`
1557
- response, so the worker stores it and answers later views with no request. A cold visit costs one
1558
- origin fetch: the proxy names a hash of the page in `Spool-Artifact`, and the worker keeps the
1559
- page it already received as the artifact when its own hash matches. On any mismatch it asks
1560
- `/.spool/artifact` as before. In the first-payload reading (L1, a synthetic 11,145-byte page) a
1561
- cold visit made 1 request and 1 origin fetch where it made 2 and 2 before.
1562
-
1563
- `spool` takes the options `createSpool` takes, minus `routes`. The proxy adds one public
1564
- `assembly` route that matches every path (`/**`) and uses the origin's finished HTML as the
1565
- artifact. `routes` lists overrides, such as a shorter freshness bound for one pattern. They
1566
- match first and take the same fields as `defineRoute`, without `load`.
1567
-
1568
- On Node the handler goes through `toNodeListener`. `origin` is the address of the real server
1569
- and has no path. `fetch` replaces the global `fetch` the proxy calls the origin with, such as a
1570
- Workers service binding.
1571
-
1572
- ```ts
1573
- import { createServer } from 'node:http';
1574
- import { toNodeListener } from '@drupflare/spool/node';
1575
- import { spoolProxy } from '@drupflare/spool/proxy';
1576
- import { memoryStore } from '@drupflare/spool/stores';
1577
-
1578
- const proxy = spoolProxy({
1579
- origin: process.env['ORIGIN'] as string,
1580
- spool: { store: memoryStore() },
1581
- purgeSecret: process.env['PURGE_SECRET']
1582
- });
1583
-
1584
- export const server = createServer(toNodeListener(proxy));
1585
- ```
1586
-
1587
- On Workers the proxy is built once per isolate and keeps the version vector in a Durable Object,
1588
- bound as in [Cloudflare Workers](#cloudflare-workers). Set the purge secret with
1589
- `wrangler secret put PURGE_SECRET`.
1590
-
1591
- ```ts
1592
- import { spoolProxy, type SpoolProxy } from '@drupflare/spool/proxy';
1593
- import { durableObjectStore } from '@drupflare/spool/stores/durable-object';
1594
-
1595
- export { SpoolVersions } from '@drupflare/spool/stores/durable-object';
1596
-
1597
- let proxy: SpoolProxy | undefined;
1598
-
1599
- export default {
1600
- fetch(request: Request, env: Cloudflare.Env) {
1601
- proxy ??= spoolProxy({
1602
- origin: 'https://origin.example.edu',
1603
- spool: { store: durableObjectStore(env.SPOOL_VERSIONS) },
1604
- purgeSecret: env.PURGE_SECRET
1605
- });
1606
- return proxy(request);
1607
- }
1608
- };
1609
- ```
1610
-
1611
- ### Tags
1612
-
1613
- The tags the origin sends become the page's dependencies. The proxy reads `Cache-Tag` (comma
1614
- separated), `Surrogate-Key` and `X-Drupal-Cache-Tags` (space separated) and `xkey` (spaces or
1615
- commas), then removes them from the response unless `exposeTags` is true. A page with no tags
1616
- depends on one tag derived from its path.
1617
-
1618
- ### Eligibility
1619
-
1620
- The proxy keeps a response only when all of these hold:
1621
-
1622
- - the request is a GET or HEAD, has no query string and no `Authorization` header, and carries
1623
- no cookie outside `publicCookies`, which is empty by default
1624
- - the response is a 200 `text/html`, or the answer of a `data` route
1625
- - the page is UTF-8, or declares no charset in its `Content-Type` or in a `<meta>` within its
1626
- first 1024 bytes
1627
- - the response sets no cookie outside `publicCookies`, has no `private` or `no-store` in
1628
- `Cache-Control`, a `Vary` that names only `Accept-Encoding` and `Cookie`, no more than
1629
- `maxTags` tags and no more than `maxBytes` bytes
1630
-
1631
- Everything else is forwarded untouched. A page the proxy declines tells the worker to drop its
1632
- copy of that path, and a response that sets a cookie outside `publicCookies` tells the worker to
1633
- clear everything it holds, so a login leaves no anonymous copy to show. Put cookies that carry no
1634
- identity on the list, such as a bot-check or load-balancer cookie set on every response; otherwise
1635
- proxy mode never keeps a page on that site. Writes always go to the origin and are never
1636
- answered locally.
1637
-
1638
- Principal routes are off unless `spool.identity` is set. With it, a route declared
1639
- `scope: 'principal'` follows the partitions in [Authenticated Data](#-authenticated-data). The
1640
- cookie, `Authorization` and `private` rules do not apply to it. `Set-Cookie` and `no-store`
1641
- still do.
1642
-
1643
- ### Invalidation
1644
-
1645
- Two signals reach the proxy. When a page it forwards differs from the last body it saw, it bumps
1646
- that page's tags. The proxy hashes the body after `normalise`, and `ignore` takes patterns to
1647
- blank, such as a per-render form token. `POST /.spool/purge` bumps the tags it is given, so a CMS
1648
- hook or Drupal's purge module can call it. It needs `Authorization: Bearer <purgeSecret>` and a
1649
- JSON body, and it refuses every call when no secret is set.
1650
-
1651
- ```sh
1652
- curl -X POST https://example.edu/.spool/purge \
1653
- -H "Authorization: Bearer $PURGE_SECRET" \
1654
- -H "Content-Type: application/json" \
1655
- -d '{"tags": ["node:1", "config:system.menu.main"]}'
1656
- ```
1657
-
1658
- ### Limits
1659
-
1660
- The proxy sees finished HTML, not the database behind it. An artifact is a whole page, the
1661
- replica holds no records, and two pages never share a fragment. A page stays on a device until its
1662
- route's `maxStale` passes and the next sync reports a change, so freshness is bounded by
1663
- `maxStale` plus the time a purge takes to arrive.
1664
-
1665
- A tag shared by many pages invalidates all of them when one changes. The remembered bodies live
1666
- in the instance: a restart forgets them, and each instance of a scaled-out proxy keeps its own.
1667
- A whole page is a large artifact, and the provisioning rule declines one that costs more than the
1668
- priors say it saves, so set `provision` on the spool for large pages. A `Content-Security-Policy`
1669
- that allows only nonce or hash scripts blocks the injected client tag. A URL with a query string
1670
- always goes to the origin, because the proxy holds finished HTML and has no collection to filter;
1671
- [Query Routes](#query-routes) apply to `createSpool` only. [Trimmed Stylesheets](#-trimmed-stylesheets) covers the `trim` option
1672
- and [Images and Video](#-images-and-video) the `images` and `video` options.
1673
-
1674
- ---
1675
-
1676
- ## ๐Ÿงน Trimmed Stylesheets
1677
-
1678
- A new page of a Drupal site usually loads a stylesheet aggregate the visitor has not seen, and
1679
- most of its rules match nothing on the page. `@drupflare/spool/trim` builds a manifest that removes
1680
- the rules no page of the site can match, and `spoolProxy` serves the trimmed text in place of the
1681
- origin's file. Each file is trimmed against every page and script of the site, never against one
1682
- page, so it keeps its URL and its one cached copy. Rules are only removed, so the cascade of what
1683
- stays is unchanged.
1684
-
1685
- ```ts
1686
- import { spoolProxy } from '@drupflare/spool/proxy';
1687
- import { memoryStore } from '@drupflare/spool/stores';
1688
- import { buildTrimManifest } from '@drupflare/spool/trim';
1689
-
1690
- export const manifest = await buildTrimManifest({
1691
- pages: ['<body class="hero"><a class="button">Apply</a></body>'],
1692
- scripts: [{ url: '/js/site.js', text: 'menu.classList.add("open")' }],
1693
- styles: [{ url: '/css/site.css', text: '.hero{a:b}.button{c:d}.open{e:f}.print-only{g:h}' }]
1694
- });
1695
-
1696
- export const proxy = spoolProxy({
1697
- origin: 'https://origin.example.edu',
1698
- spool: { store: memoryStore() },
1699
- trim: manifest
1700
- });
1701
- ```
1702
-
1703
- `pages` is HTML, `scripts` and `styles` are the same-origin files the pages load, and `url` is the
1704
- address the page links. Crawl with a browser and pass the page's DOM after load and after a
1705
- scroll as well as the origin's HTML: a class that a script builds from parts, such as
1706
- `'fs-carousel' + '-enabled'`, is only in the rendered page. Pass the scripts of other origins that
1707
- the pages load as `foreignScripts`, since a widget that builds markup adds classes too.
1708
-
1709
- Two checks protect a visitor. The proxy serves a trimmed file only when the origin's bytes still
1710
- hash to the manifest's `source`, and otherwise sends the origin's file and counts a mismatch. It
1711
- also reads every HTML page before it leaves: a page with a token the trim removed a rule for, or a
1712
- same-origin script the manifest was not built with, links the full file through `spool-trim=0`.
1713
- `proxy.trim` counts the files served, the mismatches and the guard firings, and the sink gets
1714
- each mismatch and firing as an event. [docs/security.md](docs/security.md#trimmed-stylesheets) names
1715
- what the guard cannot see: a class a vendor script adds after the crawl, markup injected after load
1716
- from a page the crawl never requested, and a class name built at runtime that no crawl rendered.
1717
-
1718
- The proxy trims CSS and leaves JavaScript alone. Coverage at load cannot see what a click or a
1719
- scroll will need: the used share of same-origin JavaScript rose from 36% at load to 40% after one
1720
- scroll on the pages measured (L1), so no load-time figure says which of the rest is dead. The same
1721
- saving from a build step that runs PurgeCSS with a safelist needs no proxy. The proxy gets it
1722
- without a change to the origin. The saving applies to a network view of a page the visitor has not
1723
- seen, never to a repeat or local view, and it is not part of the per-view headline;
1724
- [docs/impact.md](docs/impact.md#asset-trimming) has the bytes, joules and prices.
1725
-
1726
- ---
1727
-
1728
- ## ๐ŸŽฌ Images and Video
1729
-
1730
- A browser that has fetched an image cannot make it smaller, so conversion happens where the file is
1731
- served. `@drupflare/spool/media` gives `spoolProxy` three options, each off until set. `images`
1732
- converts a JPEG or PNG response to WebP for a browser whose `Accept` lists it. `video` rewrites
1733
- the `<video>` tags of an HTML response, such as adding `preload="none"`. `embeds` swaps a YouTube
1734
- or Vimeo iframe for a link that loads the player on click.
1735
-
1736
- ```ts
1737
- import { type ImageEncoder } from '@drupflare/spool/media';
1738
- import { spoolProxy } from '@drupflare/spool/proxy';
1739
- import { memoryStore } from '@drupflare/spool/stores';
1740
-
1741
- export function mediaProxy(encoder: ImageEncoder) {
1742
- return spoolProxy({
1743
- origin: 'https://origin.example.edu',
1744
- spool: { store: memoryStore() },
1745
- images: { encoder, width: { fallback: 1600 } },
1746
- video: { preload: 'none', autoplay: 'defer', when: 'slow' }
1747
- });
1748
- }
1749
- ```
1750
-
1751
- An `ImageEncoder` takes the origin's bytes and `{ format: 'webp', quality, width }` and returns
1752
- `{ bytes }`. It never upscales, so `width` is a ceiling, and it throws on input it cannot decode.
1753
- spool has no runtime dependencies and ships an adapter for [tinyimg](https://tinyimg.gmitch215.dev),
1754
- which decodes and encodes inside WebAssembly:
1755
-
1756
- ```js
1757
- import { readFile } from 'node:fs/promises';
1758
- import { TinyImgModule, transform } from '@gmitch215/tinyimg';
1759
- import { tinyimgEncoder } from '@drupflare/spool/media';
1760
-
1761
- const wasm = new URL(import.meta.resolve('@gmitch215/tinyimg/tinyimg.wasm'));
1762
- const module = await TinyImgModule.loadBytes(await readFile(wasm));
1763
-
1764
- export const encoder = tinyimgEncoder({ module, transform });
1765
- ```
1766
-
1767
- Use tinyimg 1.1.1 or later. Version 1.1.0 and earlier drop the EXIF orientation tag without
1768
- rotating the pixels, so a phone photo comes out sideways. `tinyimgEncoder` throws when the module
1769
- reports an older version.
1770
-
1771
- Quality is 80, 65 for `Save-Data: on` and 70 for an `ECT` of `slow-2g`, `2g` or `3g`. The width
1772
- comes from `Sec-CH-Width`, then `Sec-CH-Viewport-Width` times `Sec-CH-DPR`, then
1773
- `width.fallback`. The proxy converts a `GET` that matches `paths` (JPEG and PNG names by default),
1774
- has no `Range` or `Authorization`, and gets a `200` JPEG or PNG without a cookie between
1775
- `minBytes` (4096) and `maxBytes` (15,000,000). Anything else, and any file the encoder cannot
1776
- shrink, goes out as the origin sent it. The proxy fetches the origin for every request, so a
1777
- visitor only gets bytes the origin sent that visitor.
1778
-
1779
- A converted response carries `Vary` for `Accept`, `Save-Data`, `Sec-CH-Width`, `Sec-CH-DPR`,
1780
- `Sec-CH-Viewport-Width` and `ECT`, a weak `ETag` and `X-Spool-Image`. HTML responses gain
1781
- `Accept-CH` for the three size hints and `ECT` unless `hints` is false, and the browser then
1782
- sends them with later image requests to the same origin.
1783
-
1784
- `maxWidth` (2560) is the width for a request that carries no hint, and `false` removes it. `png`
1785
- is `'lossy'` by default. `'lossless'` encodes every PNG without loss, and `'auto'` encodes a PNG
1786
- under 200 KB both ways and serves the smaller file, which suits screenshots and flat graphics.
1787
- Requests for the same file, quality and width share one encoder call while it runs. The proxy
1788
- waits `timeoutMs` (10,000) for the encoder and then serves the origin's file, and the late result
1789
- is cached for the next request. `timeoutMs: false` waits without limit.
1790
-
1791
- Drupal's video formatter has no `preload` setting, and `preload="none"` does not stop `autoplay`.
1792
- `autoplay: 'defer'` removes the attribute, adds `controls` and marks the tag
1793
- `data-spool-autoplay`. With `when: 'always'` the page the spool keeps carries the rewrite. With
1794
- `'save-data'` or `'slow'` the rewritten page is private to that visitor: it gets `private` in
1795
- `Cache-Control` and `Save-Data, ECT` in `Vary`, and the spool does not keep it. A device's stored
1796
- copy of that page keeps the origin's tags.
1797
-
1798
- `proxy.images` counts `converted`, `bytesIn` and `bytesOut` (converted responses only),
1799
- `cacheHits`, `coalesced`, `lazied`, `responsive`, and `skipped` by reason. `proxy.video` counts
1800
- `pages` and `rewritten`. Read the saving as `bytesIn - bytesOut` over real traffic on the site. A
1801
- request for any other path counts under `skipped.path`, so that key follows total traffic.
1802
- [docs/security.md](docs/security.md#images-and-video) covers the decoder.
1803
-
1804
- ### Page Rewrites
1805
-
1806
- Three options change the HTML the origin sends. Each edits start tags only and leaves every other
1807
- byte as it was, and a rewritten page gets a weak `ETag`.
1808
-
1809
- ```ts
1810
- import { type ImageEncoder } from '@drupflare/spool/media';
1811
- import { spoolProxy } from '@drupflare/spool/proxy';
1812
- import { memoryStore } from '@drupflare/spool/stores';
1813
-
1814
- export function rewriteProxy(encoder: ImageEncoder) {
1815
- return spoolProxy({
1816
- origin: 'https://origin.example.edu',
1817
- spool: { store: memoryStore() },
1818
- images: {
1819
- encoder,
1820
- png: 'auto',
1821
- maxWidth: 1920,
1822
- timeoutMs: 5000,
1823
- lazy: { skip: 3 },
1824
- responsive: { widths: [480, 960, 1440], sizes: '(min-width: 60em) 50vw, 100vw' }
1825
- },
1826
- embeds: { providers: ['youtube'], thumbnail: false }
1827
- });
1828
- }
1829
- ```
1830
-
1831
- `lazy` adds `loading="lazy"` and `decoding="async"` to each `<img>` after the first `skip` (2 by
1832
- default), and `iframes: true` does the same for `<iframe>`. It leaves a tag that has `loading`,
1833
- `fetchpriority` or a lazy-loading library's `data-src`, a tag inside a `<header>` or `<noscript>`,
1834
- and an image that a `<link rel="preload" as="image">` names. A browser that defers the largest
1835
- image of a page paints it later, so `skip` is the only guard and the option is off until set. In a
1836
- crawl of 70 pages per profile on 16 sites, the default `skip` of 2 made the largest image lazy on 2
1837
- of 23 desktop pages and 2 of 20 phone pages that had an image as their largest element, and no
1838
- `skip` from 2 to 4 reached zero on both profiles.
1839
-
1840
- `responsive` adds a `srcset` of `?spool-w=<width>` variants to a same-origin `<img>` that has none,
1841
- one per entry of `widths` (320, 480, 640, 960, 1280 and 1920 by default), and a `sizes` of `100vw`
1842
- unless the tag has one. The proxy removes `spool-w` before it asks the origin, answers only widths
1843
- from the list and never sends a file larger than the origin's. Firefox and Safari send no
1844
- `Sec-CH-Width`, and this is how they get a resized file. Name the layout width in `sizes`, since
1845
- `100vw` makes the browser pick the widest candidate it can use. A tag with `srcset` or `data-src`,
1846
- and any tag inside `<picture>`, stays as it is.
1847
-
1848
- `embeds` replaces a server-rendered YouTube or Vimeo `<iframe>` with one whose `srcdoc` holds a
1849
- link to the player. YouTube shows its thumbnail (`thumbnail: false` shows a plain frame) and the
1850
- player loads on click. A page that sends `Content-Security-Policy` is left alone unless
1851
- `ignoreCsp` is true, and so is an embed with `enablejsapi=1`. An embed that plays on its own by
1852
- design stays too: a URL with `background=1`, or with `autoplay=1` beside `mute`, `muted` or `loop`,
1853
- and an iframe whose `class` or `id` names a hero, ambient, background or banner. In a crawl of the
1854
- server-rendered pages of 52 sites (L1, `measurements/embed-markup`), 2 of 4 Vimeo iframes were
1855
- background loops. `proxy.embeds` counts `pages`, `rewritten` and `skipped` by reason. [docs/security.md](docs/security.md#embed-facades) covers the rules.
1856
-
1857
- ### Dictionary Transport
1858
-
1859
- A visitor who holds an earlier stylesheet or script of a directory can fetch the next one as the
1860
- difference from it. `dictionary` implements this for Chromium 130 and later
1861
- ([RFC 9842](https://www.rfc-editor.org/rfc/rfc9842)). The proxy marks public `.css` and `.js`
1862
- responses with `Use-As-Dictionary` and answers a request that names one of them with
1863
- `Content-Encoding: dcz`. spool has no compression library, so the caller supplies the `encoder`,
1864
- and Node's `zlib` does zstd:
1865
-
1866
- ```ts
1867
- import { promisify } from 'node:util';
1868
- import { zstdCompress } from 'node:zlib';
1869
- import { spoolProxy } from '@drupflare/spool/proxy';
1870
- import { memoryStore } from '@drupflare/spool/stores';
1871
-
1872
- const compress = promisify(zstdCompress);
1873
-
1874
- export const proxy = spoolProxy({
1875
- origin: 'https://origin.example.edu',
1876
- spool: { store: memoryStore() },
1877
- dictionary: {
1878
- encoder: async (dictionary, input) => new Uint8Array(await compress(input, { dictionary }))
1879
- }
1880
- });
1881
- ```
1882
-
1883
- The `encoder` gets the dictionary, the new file and the coding (`'dcz'`, or `'dcb'` when `formats`
1884
- lists it) and returns the compressed stream. spool adds the coding's header. An encoder that
1885
- throws, or whose output is no smaller, leaves the plain file. Only responses the proxy would keep
1886
- as public ones, with no cookie, ever become dictionaries. `proxy.dictionary` counts `registered`,
1887
- `encoded`, `bytesIn`, `bytesOut` and `skipped`.
1888
- [docs/security.md](docs/security.md#dictionary-transport) covers the exposure.
1889
-
1890
- `base` names the files that register as dictionaries, as a function of the pathname, a regular
1891
- expression or a list of them. Without it the browser holds the latest file of each directory it
1892
- fetched, and the sizing in [docs/impact.md](docs/impact.md#savings-today-and-at-maximum) shows that
1893
- choice recovers little. With it the browser holds only the files you name, one per directory, and
1894
- every other file of the directory is encoded against the base the request names.
1895
-
1896
- `chooseBases` picks the bases from a crawl. It takes the site's stylesheets and scripts with their
1897
- compressed sizes and a `delta` function that returns the size of one file compressed against
1898
- another. For each directory and file kind it returns the file whose deltas against the others, each
1899
- capped at the other file's plain size, add up smallest. That `held` policy scores a base for a
1900
- visitor who already holds it. Pass `{ policy: 'first-visit' }` to add the base's own plain size
1901
- to the sum, for visitors who load the whole group before they hold anything. Neither policy
1902
- weighs how often a file is requested or how long browsers keep the base, so neither is a proven
1903
- minimum of total transfer. A directory with one file has no base, and a tie goes to the URL that
1904
- sorts first. `baseMatcher` turns the choices into a `base` function.
1905
-
1906
- ```ts
1907
- import { promisify } from 'node:util';
1908
- import { brotliCompressSync, zstdCompress } from 'node:zlib';
1909
- import { baseMatcher, chooseBases, spoolProxy } from '@drupflare/spool/proxy';
1910
- import { memoryStore } from '@drupflare/spool/stores';
1911
-
1912
- const compress = promisify(zstdCompress);
1913
-
1914
- const crawl = [
1915
- { url: 'https://origin.example.edu/css/a.css', kind: 'style', text: '.a{color:red}.b{margin:0}' },
1916
- {
1917
- url: 'https://origin.example.edu/css/b.css',
1918
- kind: 'style',
1919
- text: '.a{color:red}.c{margin:1px}'
1920
- },
1921
- {
1922
- url: 'https://origin.example.edu/css/c.css',
1923
- kind: 'style',
1924
- text: '.a{color:red}.d{margin:2px}'
1925
- }
1926
- ] as const;
1927
-
1928
- const bytes = (index: number) => Buffer.from(crawl[index]?.text ?? '');
1929
-
1930
- const choices = await chooseBases(
1931
- crawl.map(({ url, kind, text }) => ({ url, kind, plain: brotliCompressSync(text).length })),
1932
- async (file, dictionary) =>
1933
- (await compress(bytes(file), { dictionary: bytes(dictionary) })).length
1934
- );
1935
-
1936
- export const proxy = spoolProxy({
1937
- origin: 'https://origin.example.edu',
1938
- spool: { store: memoryStore() },
1939
- dictionary: {
1940
- encoder: async (dictionary, input) => new Uint8Array(await compress(input, { dictionary })),
1941
- base: baseMatcher(choices)
1942
- }
1943
- });
1944
- ```
1945
-
1946
- On the 10 Drupal sites of the reading, a base chosen this way and held from an earlier visit saves
1947
- 29.9% of the brotli 11 bytes of new stylesheets and scripts at the median site with `dcz`, and a
1948
- visit that has to fetch the base first saves 3.9%
1949
- (L1, [docs/impact.md](docs/impact.md#savings-today-and-at-maximum)).
1950
- `scripts/measure/dictionary-size.ts` records the same choice for a site's crawl.
1951
-
1952
- ### Defaults
1953
-
1954
- Every option is absent by default. The table shows what a feature does once its parent option is
1955
- set and how to change it.
1956
-
1957
- | Feature | Option | Default once the parent is set | Leave or enter |
1958
- | ------------------------------------------ | ------------------- | ------------------------------------------- | ----------------------------------------------------- |
1959
- | WebP conversion, `304` and `X-Spool-Image` | `images` | on, needs an `encoder` | leave `images` out |
1960
- | `Accept-CH` hints and `Vary` | `images.hints` | on | `hints: false` |
1961
- | Quality by `Save-Data` and `ECT` | `images.quality` | 80, 65 and 70 | give all three one number |
1962
- | Width ceiling for a request with no width | `images.maxWidth` | 2560 | `maxWidth: false`, or another number |
1963
- | Shared encodes and the encoder time limit | `images.timeoutMs` | 10,000 ms, identical encodes share one call | `timeoutMs: false` removes the limit |
1964
- | PNG mode | `images.png` | `'lossy'` | `'auto'` or `'lossless'` |
1965
- | `loading="lazy"` on images and iframes | `images.lazy` | off | `lazy: {}` or `{ skip, iframes }` |
1966
- | `srcset` variants | `images.responsive` | off | `responsive: {}` or `{ widths, sizes }` |
1967
- | `preload="none"` on video | `video.preload` | `'none'` | `preload: false` |
1968
- | Video autoplay deferral | `video.autoplay` | off | `autoplay: 'defer'` |
1969
- | YouTube and Vimeo facades | `embeds` | off | `embeds: {}` or `{ providers, thumbnail, ignoreCsp }` |
1970
- | Dictionary transport | `dictionary` | off | `dictionary: { encoder }` |
1971
- | Which files become dictionaries | `dictionary.base` | every `.css` and `.js` response | `base: /\/base\.js$/`, a list or a function |
1972
-
1973
- ---
1974
-
1975
- ## ๐Ÿ“ˆ Capacity Reports
1976
-
1977
- `@drupflare/spool/report` turns peak work and an institution's own limits into three separate
1978
- results: potential, financial and physical. None is added to another, and any can be empty.
1979
-
1980
- `capacityPlan` takes intervals of demand per resource with the part spool avoided, and what one
1981
- node serves per second within the latency target. It returns the nodes needed before and after,
1982
- as whole numbers, and how many could retire. `perNode.busyMs` is the milliseconds of handler work
1983
- a node does per second, such as 1000 times its cores times a target utilisation. `spares`, `floor`,
1984
- `regions` and `minPerRegion` raise the answer for redundancy, and a resource with no `avoided`, such
1985
- as memory, does not shrink. `intervalsFromWork` builds the intervals from the `WorkSample`s the
1986
- work series records.
1987
-
1988
- `institutionReport` adds the plan, an `observed` before and after of the same workload from
1989
- production telemetry, an invoice total before and after, and a ledger of retired equipment. The
1990
- plan says what could change, `observed` what changed, the invoices what was paid, and the ledger
1991
- what was removed. The physical result reads only the ledger, and an entry needs the evidence for
1992
- its removal. With no ledger the statement reads `No physical capacity saving demonstrated.` The
1993
- report also returns `economics`, `storage`, `trust` and `research` blocks. Each is null without its
1994
- input: an `institution` profile for `economics` and `research`, `storage` for the held bytes, and
1995
- `views` for `trust`.
1996
- [docs/impact.md](docs/impact.md#an-institutions-figures) prints them for three example
1997
- institutions.
1998
-
1999
- ```ts
2000
- import { capacityPlan, institutionReport } from '@drupflare/spool/report';
2001
-
2002
- const plan = capacityPlan(
2003
- [
2004
- {
2005
- start: Date.UTC(2026, 8, 1, 14),
2006
- seconds: 300,
2007
- demand: { busyMs: 9000 },
2008
- avoided: { busyMs: 6300 }
2009
- },
2010
- {
2011
- start: Date.UTC(2026, 8, 1, 14, 5),
2012
- seconds: 300,
2013
- demand: { busyMs: 7000 },
2014
- avoided: { busyMs: 4900 }
2015
- }
2016
- ],
2017
- { perNode: { busyMs: 700 }, spares: 1, floor: 2 }
2018
- );
2019
-
2020
- export const report = institutionReport({ plan, planLayer: 'L3' });
2021
- ```
2022
-
2023
- The example needs 14 nodes without spool and 5 with it, a potential of 9, at layer L3 because the
2024
- numbers are made up. `channelRows` lists the metered, attributional, capacity and avoided-demand
2025
- channels as separate rows, and nothing in the module sums them. [docs/impact.md](docs/impact.md#channels)
2026
- says what each one counts.
2027
-
2028
- ### Request Arithmetic
2029
-
2030
- On a plan that limits requests, the question is how many views the plan carries. Name three
2031
- inputs, each a scenario (L3) until a site's traffic supplies it:
2032
-
2033
- - `h`, the share of views answered on the device
2034
- - `r`, the requests a network view makes to the platform, and `rho`, the part of them a local view
2035
- removes (one for the document when the page's API calls still leave)
2036
- - `s`, the requests spool sends itself per local view: syncs, beacon flushes, catalogue asks and
2037
- worker script checks
2038
-
2039
- The platform sees `r - h * rho + h * s` requests per view, so a limit `L` carries
2040
- `L / (r - h * rho + h * s)` views. With `r = rho = 1` this is `freeTierCeiling` in
2041
- `@drupflare/spool/report`. Three things bound `h` below 1:
2042
-
2043
- - A device's first view of a site, a view after the browser evicted the replica, and a view past a
2044
- freshness bound all reach the network. A device that makes `v` views in a freshness window
2045
- answers at most `v - 1` of them locally, so `h <= 1 - 1/v` for that device.
2046
- - A client that does not register the worker, such as a crawler, answers none.
2047
- - A request outside every route family always leaves.
2048
-
2049
- A limit on CPU per request, such as the 10 ms of the Workers Free plan, does not change with `h`.
2050
- The docs' free-tier table uses one sync per 20 local views as an illustration. No multiplier
2051
- follows from the formula until a site's `v`, its share of clients that cannot answer locally and
2052
- `s` are measured.
2053
-
2054
- ---
2055
-
2056
- ## ๐Ÿ”’ Security
436
+ `spool.publishStatic(sink)` is a different route to the same tier, for a host that runs no spool server. It writes the page runtime, the worker script, the packs and their deltas, one boot script per pack and a `_headers` file to the sink, and the host answers those files with no handler call. It does not write `spool.json`. With no policy known, the stub names none, and a device renders locally after one network view that teaches it the headers. Pass `policy` to `createSpool` (`{}` for a site with no security headers) to name them. The Vite, Vue, React Router and Nuxt plugins call `publishStatic` at the end of a build by default; `static: false` turns that off. See [Static Tier](docs/protocol.md#static-tier) and the [static example](examples/static/README.md).
437
+
438
+ ---
439
+
440
+ ## Advanced Usage
441
+
442
+ [ADVANCED_USAGE.md](ADVANCED_USAGE.md) covers each declaration and setting past the Quick Start,
443
+ in the order an operator meets them. Each section gives the use, the smallest declaration and the
444
+ limit.
445
+
446
+ - [Strategies](ADVANCED_USAGE.md#strategies): the five ways a route is answered locally, and the
447
+ security headers a local page keeps.
448
+ - [Records and Projections](ADVANCED_USAGE.md#records-and-projections): shared records, stored once
449
+ per id.
450
+ - [Templates](ADVANCED_USAGE.md#templates): the template syntax and the trusted-field rule.
451
+ - [Static Files](ADVANCED_USAGE.md#static-files): files kept as the server sent them.
452
+ - [Projection and Data Routes](ADVANCED_USAGE.md#projection-and-data-routes): JSON answers, bundled
453
+ renderers and the provisioning rule.
454
+ - [Query Routes](ADVANCED_USAGE.md#query-routes): filter, sort and page a held collection with no
455
+ request.
456
+ - [Freshness Contracts](ADVANCED_USAGE.md#freshness-contracts): hard and soft bounds, and the script
457
+ lease.
458
+ - [Datasets](ADVANCED_USAGE.md#datasets): row patches with a proof, plus
459
+ [Dataset Routes](ADVANCED_USAGE.md#dataset-routes) and
460
+ [Principal Datasets](ADVANCED_USAGE.md#principal-datasets).
461
+ - [Authenticated Data](ADVANCED_USAGE.md#authenticated-data): per-principal partitions, sealed
462
+ storage and logout wipes.
463
+ - [Declared Writes](ADVANCED_USAGE.md#declared-writes): queued last-value edits with a version check.
464
+ - [Page Runtime Settings](ADVANCED_USAGE.md#page-runtime-settings): the bootstrap gate, `workerMode`,
465
+ swap elements and the `window.spool` surface.
466
+ - [Catalogue](ADVANCED_USAGE.md#catalogue): data for the pages a visitor may open next.
467
+ - [Analytics Batching](ADVANCED_USAGE.md#analytics-batching) and
468
+ [Collectors](ADVANCED_USAGE.md#collectors): beacons, sinks and third-party collection.
469
+ - [Third-Party Files](ADVANCED_USAGE.md#third-party-files) and
470
+ [Subresource Handling](ADVANCED_USAGE.md#subresource-handling).
471
+ - [Shadow Mode](ADVANCED_USAGE.md#shadow-mode): compare local renders with the network before
472
+ serving.
473
+ - [Proxy Mode](ADVANCED_USAGE.md#proxy-mode): front an origin that cannot load the library.
474
+ - [Trimmed Stylesheets](ADVANCED_USAGE.md#trimmed-stylesheets) and
475
+ [Images and Video](ADVANCED_USAGE.md#images-and-video): proxy options for CSS, images and video.
476
+ - [Capacity Reports](ADVANCED_USAGE.md#capacity-reports): the capacity plan, the institution report
477
+ and request arithmetic.
478
+
479
+ ---
480
+
481
+ ## Security
2057
482
 
2058
483
  Authorisation stays on the server. The device holds only what the server already sent that
2059
484
  principal, and the worker has no access rule. Principal data sits in a per-partition IndexedDB
@@ -2069,7 +494,7 @@ each claim and lists the open limitations in one table. Report a vulnerability a
2069
494
 
2070
495
  ---
2071
496
 
2072
- ## ๐Ÿ”‹ Energy Reporting
497
+ ## Energy Reporting
2073
498
 
2074
499
  Energy is one objective among those spool reports, and the administrator decides how much weight
2075
500
  it gets. The energy of one deployment is small against an institution's total, and small against
@@ -2083,13 +508,15 @@ published intensity over bytes is an allocation of that figure, and the page pri
2083
508
  reading beside it. The savings from trimming, images and dictionaries apply to network views and
2084
509
  never add to the local-view figure.
2085
510
 
511
+ The request counts in [Economics](docs/economics.md#headline-numbers) come from library readings on Chromium and Firefox. WebKit is absent from every library reading. The library's headline covers Chromium and Firefox only. WebKit, which is every browser on iOS, defaults to no worker because of an upstream script check on every navigation; its figures are model rows or single script-check readings, never library readings. "Chromium and Firefox" is well under all of mobile.
512
+
2086
513
  [An Institution's Figures](docs/impact.md#an-institutions-figures) leads with economics, storage
2087
514
  and trust and treats energy as a research figure. [What This Does Not
2088
515
  Claim](docs/impact.md#what-this-does-not-claim) lists what the page leaves unsaid.
2089
516
 
2090
517
  ---
2091
518
 
2092
- ## ๐Ÿงฌ Prior Art
519
+ ## Prior Art
2093
520
 
2094
521
  spool is a server-authoritative replica whose freshness policy lets a provisioned view render with
2095
522
  no per-view validation request, with shadow mode and a fall back to the server for anything it
@@ -2108,7 +535,7 @@ Sync Kit (WWW 2010) also keeps templates and data in the browser to reduce serve
2108
535
 
2109
536
  ---
2110
537
 
2111
- ## ๐Ÿšซ Out of Scope
538
+ ## Out of Scope
2112
539
 
2113
540
  - spool never authorises on the client. The device holds only what the server already sent that
2114
541
  principal.
@@ -2121,17 +548,22 @@ Sync Kit (WWW 2010) also keeps templates and data in the browser to reduce serve
2121
548
 
2122
549
  ---
2123
550
 
2124
- ## ๐ŸŒ Browser Support
551
+ ## Browser Support
2125
552
 
2126
- | Engine | Controlled navigation |
2127
- | -------- | ------------------------------------------------------------------------------------------------------------------ |
2128
- | Chromium | no request, new tabs included |
2129
- | Firefox | no request, new tabs included |
2130
- | WebKit | one request to re-check `/spool-sw.js`, and one refetch of each cacheable subresource on the first controlled load |
553
+ | Engine | Controlled navigation |
554
+ | -------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
555
+ | Chromium | no request, new tabs included |
556
+ | Firefox | no request, new tabs included |
557
+ | WebKit | with `workerMode: 'always'`, on macOS, one request to re-check `/spool-sw.js` on each document navigation (none on Linux while the worker runs); with the default `'auto'`, no worker on a page with a ring, so no script request; one refetch of each cacheable subresource on the first controlled load |
2131
558
 
2132
559
  The worker registers with `updateViaCache: 'all'` and the script is served at
2133
- `max-age=86400`, which is what lets Chromium and Firefox skip the update check. A deploy changes
2134
- the script bytes and reaches a browser within that day. A browser without service workers gets the
560
+ `max-age=86400`, which is what lets Chromium and Firefox skip the update check. WebKit on macOS
561
+ re-checks the script regardless, so with `workerMode: 'always'` a document navigation there costs one
562
+ request whatever the site does ([docs/impact.md](docs/impact.md#demand-per-client)). With the default
563
+ `workerMode: 'auto'`, WebKit registers no worker on a page that carries a ring, so the script is not
564
+ requested; the page runtime still answers held reads and in-page navigation
565
+ ([WebKit Default](docs/protocol.md#webkit-default)). A deploy changes the script bytes and reaches a
566
+ browser within that day. A browser without service workers gets the
2135
567
  page it always got.
2136
568
 
2137
569
  A page that loads a resource from another origin is not a zero-request view. spool counts those
@@ -2139,7 +571,7 @@ loads per view and reports the view as such.
2139
571
 
2140
572
  ---
2141
573
 
2142
- ## ๐Ÿงญ Subpath Exports
574
+ ## Subpath Exports
2143
575
 
2144
576
  | Import | What is in it |
2145
577
  | ---------------------------------------- | ------------------------------------------------------------------------------------------------------------- |
@@ -2155,10 +587,18 @@ loads per view and reports the view as such.
2155
587
  | `@drupflare/spool/report` | the energy model, capacity plans and institution reports |
2156
588
  | `@drupflare/spool/trim` | `buildTrimManifest`, `trimCss` and `guardPage` |
2157
589
  | `@drupflare/spool/media` | `tinyimgEncoder`, `imageVariant`, `rewriteVideo` and `rewriteEmbeds` |
590
+ | `@drupflare/spool/express` | `spoolExpress` |
591
+ | `@drupflare/spool/hono` | `spoolHono` |
592
+ | `@drupflare/spool/vite` | `spoolVite` |
593
+ | `@drupflare/spool/vue` | `spoolVue` and `vuePages` |
594
+ | `@drupflare/spool/angular` | `spoolAngular` |
595
+ | `@drupflare/spool/nuxt` | the Nuxt module (default export) and `#spool/pages` |
596
+ | `@drupflare/spool/react-router` | `spoolReactRouter`, `reactRouterPages` and `flattenRoutes` |
597
+ | `@drupflare/spool/next` | the Next adapter (default export), `spoolTag`, `spoolRouteHandler`, `nextPages` and `SPOOL_PAGES_FILE` |
2158
598
 
2159
599
  ---
2160
600
 
2161
- ## ๐Ÿงช Testing
601
+ ## Testing
2162
602
 
2163
603
  ```sh
2164
604
  bun run typecheck
@@ -2171,6 +611,11 @@ bun run docs:build
2171
611
  bun run build
2172
612
  ```
2173
613
 
614
+ `typecheck` also compiles every `ts` fence in this page and the guides as written, and it fails with
615
+ the page and line of the first fence that does not compile. A fence that cannot stand alone takes
616
+ `ts no-check` after the opening backticks and is skipped. Samples that tests run live under
617
+ `tests/node/readme/`.
618
+
2174
619
  `impact:check` fails when `docs/impact.md` differs from the readings in `measurements/`. A node
2175
620
  test fails when `src/server/embedded.ts` differs from the source under `src/protocol`, `src/sw`
2176
621
  and `src/client`, and `bun run build:sw` regenerates it. `test` runs the unit project in workerd and the node project, which includes a test that packs the
@@ -2180,17 +625,20 @@ firefox webkit` fetches the browsers.
2180
625
 
2181
626
  ---
2182
627
 
2183
- ## ๐Ÿ“– Documentation
628
+ ## Documentation
2184
629
 
630
+ - [docs/configuration.md](docs/configuration.md) covers the settings a first deployment decides.
2185
631
  - [docs/protocol.md](docs/protocol.md) holds the paths, headers, JSON bodies and limits for anyone
2186
632
  who writes an adapter or a second server. A test pins what `Spool-Protocol: 1` freezes, and the
2187
633
  page lists what may change without a new number.
2188
634
  - [docs/security.md](docs/security.md) holds the threat model and the open limitations.
2189
635
  - [docs/impact.md](docs/impact.md) holds the energy model, the readings and what they leave unsaid.
636
+ - [docs/economics.md](docs/economics.md) holds the bandwidth and Worker invocation projections.
637
+ - [examples/README.md](examples/README.md) lists the example sites and runtimes, and the command that runs each one.
2190
638
 
2191
639
  ---
2192
640
 
2193
- ## ๐Ÿ”— Related Repositories
641
+ ## Related Repositories
2194
642
 
2195
643
  | Repository | What it is |
2196
644
  | --------------------------------------------------------------- | -------------------------------------------------------- |
@@ -2201,6 +649,8 @@ firefox webkit` fetches the browsers.
2201
649
 
2202
650
  ---
2203
651
 
2204
- ## ๐Ÿ“„ License
652
+ ## License
2205
653
 
2206
654
  MIT (c) Gregory Mitchell 2026. See [LICENSE](LICENSE).
655
+
656
+ The documentation favicons in `docs/` come from Google's [Noto Emoji](https://github.com/googlefonts/noto-emoji) set, licensed under [Apache 2.0](https://www.apache.org/licenses/LICENSE-2.0).