@drupflare/spool 0.3.0 โ†’ 1.0.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (206) hide show
  1. package/README.md +279 -1607
  2. package/dist/angular/index.d.ts +13 -0
  3. package/dist/angular/index.js +25 -0
  4. package/dist/angular/pages.d.ts +28 -0
  5. package/dist/angular/pages.js +85 -0
  6. package/dist/cdn/client.js +1 -1
  7. package/dist/cdn/location.d.ts +6 -0
  8. package/dist/cdn/location.js +8 -0
  9. package/dist/cdn/manifest.json +7 -7
  10. package/dist/cdn/spool-sw.js +1 -1
  11. package/dist/client/bootstrap.d.ts +57 -0
  12. package/dist/client/bootstrap.js +307 -0
  13. package/dist/client/contact.d.ts +24 -0
  14. package/dist/client/contact.js +66 -0
  15. package/dist/client/declared.d.ts +23 -0
  16. package/dist/client/declared.js +68 -0
  17. package/dist/client/entry.js +123 -2
  18. package/dist/client/gate.d.ts +1 -0
  19. package/dist/client/gate.js +148 -0
  20. package/dist/client/index.d.ts +25 -0
  21. package/dist/client/index.js +1 -0
  22. package/dist/client/live.d.ts +66 -0
  23. package/dist/client/live.js +225 -0
  24. package/dist/client/navigate-entry.d.ts +1 -0
  25. package/dist/client/navigate-entry.js +4 -0
  26. package/dist/client/navigate.d.ts +54 -0
  27. package/dist/client/navigate.js +272 -0
  28. package/dist/client/persist.d.ts +2 -0
  29. package/dist/client/persist.js +9 -0
  30. package/dist/client/register.d.ts +34 -2
  31. package/dist/client/register.js +64 -7
  32. package/dist/express/index.d.ts +11 -0
  33. package/dist/express/index.js +209 -0
  34. package/dist/hono/index.d.ts +19 -0
  35. package/dist/hono/index.js +49 -0
  36. package/dist/next/index.d.ts +33 -0
  37. package/dist/next/index.js +41 -0
  38. package/dist/next/pages.d.ts +22 -0
  39. package/dist/next/pages.js +39 -0
  40. package/dist/node/listener.d.ts +3 -0
  41. package/dist/node/listener.js +57 -2
  42. package/dist/nuxt/index.d.ts +22 -0
  43. package/dist/nuxt/index.js +104 -0
  44. package/dist/nuxt/pages.d.ts +26 -0
  45. package/dist/nuxt/pages.js +48 -0
  46. package/dist/nuxt/runtime/index.d.ts +24 -0
  47. package/dist/nuxt/runtime/index.js +58 -0
  48. package/dist/plugins/coverage.d.ts +14 -0
  49. package/dist/plugins/coverage.js +27 -0
  50. package/dist/plugins/mode.d.ts +19 -0
  51. package/dist/plugins/mode.js +20 -0
  52. package/dist/plugins/path.d.ts +16 -0
  53. package/dist/plugins/path.js +139 -0
  54. package/dist/plugins/report.d.ts +22 -0
  55. package/dist/plugins/report.js +18 -0
  56. package/dist/plugins/routes.d.ts +38 -0
  57. package/dist/plugins/routes.js +52 -0
  58. package/dist/plugins/static.d.ts +42 -0
  59. package/dist/plugins/static.js +97 -0
  60. package/dist/protocol/bundle.d.ts +111 -0
  61. package/dist/protocol/bundle.js +428 -0
  62. package/dist/protocol/config.d.ts +80 -1
  63. package/dist/protocol/config.js +31 -1
  64. package/dist/protocol/dataset.d.ts +189 -0
  65. package/dist/protocol/dataset.js +212 -0
  66. package/dist/protocol/envelope.d.ts +34 -1
  67. package/dist/protocol/envelope.js +16 -1
  68. package/dist/protocol/errors.d.ts +1 -1
  69. package/dist/protocol/graph.d.ts +23 -4
  70. package/dist/protocol/graph.js +34 -3
  71. package/dist/protocol/index-rows.d.ts +22 -0
  72. package/dist/protocol/index-rows.js +53 -0
  73. package/dist/protocol/index.d.ts +5 -0
  74. package/dist/protocol/index.js +5 -0
  75. package/dist/protocol/normalise.d.ts +4 -4
  76. package/dist/protocol/normalise.js +9 -5
  77. package/dist/protocol/plan.d.ts +100 -0
  78. package/dist/protocol/plan.js +661 -0
  79. package/dist/protocol/provision.d.ts +46 -0
  80. package/dist/protocol/provision.js +38 -0
  81. package/dist/protocol/query.d.ts +66 -5
  82. package/dist/protocol/query.js +103 -4
  83. package/dist/protocol/ring.d.ts +58 -0
  84. package/dist/protocol/ring.js +6 -0
  85. package/dist/protocol/route.d.ts +63 -13
  86. package/dist/protocol/route.js +126 -6
  87. package/dist/protocol/tag.d.ts +22 -2
  88. package/dist/protocol/tag.js +25 -4
  89. package/dist/protocol/view.d.ts +2 -0
  90. package/dist/protocol/view.js +8 -2
  91. package/dist/protocol/views.d.ts +6 -0
  92. package/dist/protocol/views.js +15 -0
  93. package/dist/protocol/write.d.ts +234 -0
  94. package/dist/protocol/write.js +343 -0
  95. package/dist/proxy/eligible.d.ts +22 -1
  96. package/dist/proxy/eligible.js +62 -15
  97. package/dist/proxy/proxy.js +28 -18
  98. package/dist/react-router/index.d.ts +20 -0
  99. package/dist/react-router/index.js +60 -0
  100. package/dist/react-router/pages.d.ts +34 -0
  101. package/dist/react-router/pages.js +83 -0
  102. package/dist/report/dataset.d.ts +48 -0
  103. package/dist/report/dataset.js +93 -0
  104. package/dist/report/demand.d.ts +49 -0
  105. package/dist/report/demand.js +60 -0
  106. package/dist/report/derive.d.ts +285 -0
  107. package/dist/report/derive.js +676 -1
  108. package/dist/report/economics.d.ts +66 -1
  109. package/dist/report/economics.js +208 -1
  110. package/dist/report/headline.d.ts +158 -0
  111. package/dist/report/headline.js +473 -0
  112. package/dist/report/index.d.ts +4 -0
  113. package/dist/report/index.js +4 -0
  114. package/dist/report/inputs.d.ts +176 -0
  115. package/dist/report/inputs.js +248 -0
  116. package/dist/report/institution.d.ts +188 -1
  117. package/dist/report/institution.js +146 -0
  118. package/dist/report/loops.d.ts +73 -0
  119. package/dist/report/loops.js +101 -0
  120. package/dist/report/measurement.d.ts +1 -1
  121. package/dist/report/measurement.js +7 -1
  122. package/dist/report/model.d.ts +2 -0
  123. package/dist/report/model.js +4 -0
  124. package/dist/report/thirdparty.d.ts +57 -0
  125. package/dist/report/thirdparty.js +111 -0
  126. package/dist/report/traffic.d.ts +8 -0
  127. package/dist/report/traffic.js +7 -0
  128. package/dist/report/trim.d.ts +56 -0
  129. package/dist/report/trim.js +83 -1
  130. package/dist/server/bundle.d.ts +55 -0
  131. package/dist/server/bundle.js +172 -0
  132. package/dist/server/dataset.d.ts +54 -0
  133. package/dist/server/dataset.js +355 -0
  134. package/dist/server/embedded.d.ts +7 -3
  135. package/dist/server/embedded.js +7 -3
  136. package/dist/server/index.d.ts +6 -0
  137. package/dist/server/index.js +6 -0
  138. package/dist/server/live-sockets.d.ts +54 -0
  139. package/dist/server/live-sockets.js +131 -0
  140. package/dist/server/live.d.ts +146 -0
  141. package/dist/server/live.js +228 -0
  142. package/dist/server/ring.d.ts +77 -0
  143. package/dist/server/ring.js +311 -0
  144. package/dist/server/spool.d.ts +158 -3
  145. package/dist/server/spool.js +578 -41
  146. package/dist/server/static.d.ts +42 -0
  147. package/dist/server/static.js +29 -0
  148. package/dist/server/stub.d.ts +43 -5
  149. package/dist/server/stub.js +126 -8
  150. package/dist/server/write.d.ts +49 -0
  151. package/dist/server/write.js +223 -0
  152. package/dist/stores/durable-object.d.ts +46 -0
  153. package/dist/stores/durable-object.js +114 -0
  154. package/dist/sw/analytics.d.ts +7 -1
  155. package/dist/sw/analytics.js +16 -2
  156. package/dist/sw/coverage.d.ts +6 -0
  157. package/dist/sw/coverage.js +34 -0
  158. package/dist/sw/dataset-sync.d.ts +76 -0
  159. package/dist/sw/dataset-sync.js +149 -0
  160. package/dist/sw/dataset.d.ts +91 -0
  161. package/dist/sw/dataset.js +318 -0
  162. package/dist/sw/family.d.ts +15 -0
  163. package/dist/sw/family.js +48 -0
  164. package/dist/sw/graph/index.d.ts +31 -1
  165. package/dist/sw/graph/index.js +57 -5
  166. package/dist/sw/graph/invalidate.d.ts +15 -3
  167. package/dist/sw/graph/invalidate.js +42 -4
  168. package/dist/sw/idb.d.ts +2 -0
  169. package/dist/sw/idb.js +80 -11
  170. package/dist/sw/index.d.ts +2 -0
  171. package/dist/sw/index.js +1 -0
  172. package/dist/sw/pack/store.d.ts +128 -0
  173. package/dist/sw/pack/store.js +737 -0
  174. package/dist/sw/partition.d.ts +6 -1
  175. package/dist/sw/partition.js +21 -2
  176. package/dist/sw/policy.d.ts +2 -0
  177. package/dist/sw/policy.js +11 -0
  178. package/dist/sw/principal-dataset.d.ts +64 -0
  179. package/dist/sw/principal-dataset.js +254 -0
  180. package/dist/sw/probe.d.ts +16 -0
  181. package/dist/sw/probe.js +25 -0
  182. package/dist/sw/render.d.ts +7 -2
  183. package/dist/sw/render.js +19 -5
  184. package/dist/sw/replica.d.ts +12 -3
  185. package/dist/sw/replica.js +40 -5
  186. package/dist/sw/router.d.ts +3 -2
  187. package/dist/sw/router.js +15 -2
  188. package/dist/sw/runtime.d.ts +90 -2
  189. package/dist/sw/runtime.js +857 -61
  190. package/dist/sw/scheduler.d.ts +6 -1
  191. package/dist/sw/scheduler.js +22 -5
  192. package/dist/sw/seal.d.ts +13 -0
  193. package/dist/sw/seal.js +13 -4
  194. package/dist/sw/static.d.ts +18 -0
  195. package/dist/sw/static.js +84 -1
  196. package/dist/sw/storage.d.ts +51 -1
  197. package/dist/sw/storage.js +33 -1
  198. package/dist/sw/write.d.ts +147 -0
  199. package/dist/sw/write.js +625 -0
  200. package/dist/version.d.ts +1 -1
  201. package/dist/version.js +1 -2
  202. package/dist/vite/index.d.ts +26 -0
  203. package/dist/vite/index.js +66 -0
  204. package/dist/vue/index.d.ts +36 -0
  205. package/dist/vue/index.js +53 -0
  206. package/package.json +65 -4
package/README.md CHANGED
@@ -1,68 +1,77 @@
1
1
  # ๐Ÿงต spool
2
2
 
3
- > An incremental application replica for the browser
3
+ > A browser replica that removes repeated demand from your server
4
4
 
5
5
  [![Build](https://github.com/drupflare/spool/actions/workflows/build.yml/badge.svg)](https://github.com/drupflare/spool/actions/workflows/build.yml)
6
6
  [![Browser](https://github.com/drupflare/spool/actions/workflows/browser.yml/badge.svg)](https://github.com/drupflare/spool/actions/workflows/browser.yml)
7
7
  [![Prettier](https://github.com/drupflare/spool/actions/workflows/prettier.yml/badge.svg)](https://github.com/drupflare/spool/actions/workflows/prettier.yml)
8
8
  [![codecov](https://codecov.io/gh/drupflare/spool/branch/master/graph/badge.svg)](https://codecov.io/gh/drupflare/spool)
9
- [![License](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE)
9
+ [![License](https://img.shields.io/badge/license-MIT-blue.svg)](https://github.com/drupflare/spool/blob/master/LICENSE)
10
10
 
11
- spool keeps a replica of what a page needs on the visitor's device. The server sends templates
12
- and authorised, versioned data. A service worker stores them and computes eligible pages on the
13
- device, so a repeat view makes no request and costs the server no work.
11
+ spool removes repeated execution demand from your server. It keeps a replica of what a page needs
12
+ on the visitor's device. The server sends templates and authorised, versioned data, and a service
13
+ worker computes eligible pages from them. A page answered there is a request that never reaches
14
+ your Worker or origin, and server work that never runs. The contract is a count: a test asserts
15
+ the server hits of a local view, and in Chromium and Firefox the count is zero.
16
+
17
+ spool has three uses. In a drupflare deployment it takes eligible public views off the Worker.
18
+ For a team on a free or constrained plan it lowers the requests the plan has to carry, by an
19
+ amount that depends on the site's traffic ([Request Arithmetic](ADVANCED_USAGE.md#request-arithmetic)). It is also
20
+ a research system: every figure it reports carries a layer and a method.
14
21
 
15
22
  It runs on any runtime that speaks `Request` and `Response`, bridges `node:http` for the rest, and
16
23
  has no runtime dependencies. A site with no server code can load it from two files and a manifest.
17
24
 
18
25
  ---
19
26
 
20
- ## ๐Ÿ“‹ Table of Contents
21
-
22
- - [Why Spool](#-why-spool)
23
- - [Install](#-install)
24
- - [Quick Start](#-quick-start)
25
- - [Page Runtime](#-page-runtime)
26
- - [Runtimes](#-runtimes)
27
- - [Compatibility](#-compatibility)
28
- - [Static Sites](#-static-sites)
29
- - [Strategies](#-strategies)
30
- - [Freshness Contracts](#-freshness-contracts)
31
- - [Authenticated Data](#-authenticated-data)
32
- - [Catalogue](#-catalogue)
33
- - [Analytics Batching](#-analytics-batching)
34
- - [Third-Party Files](#-third-party-files)
35
- - [Subresource Handling](#-subresource-handling)
36
- - [Shadow Mode](#-shadow-mode)
37
- - [Proxy Mode](#-proxy-mode)
38
- - [Trimmed Stylesheets](#-trimmed-stylesheets)
39
- - [Images and Video](#-images-and-video)
40
- - [Capacity Reports](#-capacity-reports)
41
- - [Out of Scope](#-out-of-scope)
42
- - [Browser Support](#-browser-support)
43
- - [Subpath Exports](#-subpath-exports)
44
- - [Testing](#-testing)
45
- - [Related Repositories](#-related-repositories)
46
- - [License](#-license)
27
+ ## Table of Contents
28
+
29
+ - [Why Spool](#why-spool)
30
+ - [Install](#install)
31
+ - [Quick Start](#quick-start)
32
+ - [Page Runtime](#page-runtime)
33
+ - [Runtimes](#runtimes)
34
+ - [Compatibility](#compatibility)
35
+ - [Static Sites](#static-sites)
36
+ - [Advanced Usage](#advanced-usage)
37
+ - [Security](#security)
38
+ - [Energy Reporting](#energy-reporting)
39
+ - [Prior Art](#prior-art)
40
+ - [Out of Scope](#out-of-scope)
41
+ - [Browser Support](#browser-support)
42
+ - [Subpath Exports](#subpath-exports)
43
+ - [Testing](#testing)
44
+ - [Documentation](#documentation)
45
+ - [Related Repositories](#related-repositories)
46
+ - [License](#license)
47
47
 
48
48
  ---
49
49
 
50
- ## ๐ŸŽฏ Why Spool
50
+ ## Why Spool
51
+
52
+ A local view removes, in order, a request that would have reached the Worker, the server work
53
+ behind it (rendering, queries, Durable Object and database reads), and the bytes of a document the
54
+ device already holds. Energy follows from those and is one objective an administrator may weigh.
51
55
 
52
56
  A response is a deterministic transformation of a set of authorised, versioned dependencies. The
53
57
  server half names those dependencies and sends the snapshot the visitor may hold. The worker keeps
54
58
  a graph of records, projections and responses, with their versions, and syncs only what the next
55
59
  page needs.
56
60
 
57
- Your application keeps every write, every access decision and every page it cannot reproduce
58
- safely on the device. A route the worker cannot answer goes to the network, so adopting spool
59
- changes no page that was not declared.
61
+ Your application keeps every access decision and every page it cannot reproduce safely on the
62
+ device. It keeps every write too, except the last-value writes you declare, which ship off. A route
63
+ the worker cannot answer goes to the network, so adopting spool changes no page that was not
64
+ declared.
60
65
 
61
- [docs/impact.md](docs/impact.md) holds the energy model and the measurements behind it.
66
+ A collection of thousands of rows, such as a directory or a course catalogue, can be held as a
67
+ [dataset](ADVANCED_USAGE.md#datasets). The device seeds it once and then receives only the rows that changed.
68
+
69
+ [docs/impact.md](docs/impact.md) holds the request, cost and energy figures and the measurements
70
+ behind them. See [Energy Reporting](#energy-reporting).
62
71
 
63
72
  ---
64
73
 
65
- ## ๐Ÿ“ฅ Install
74
+ ## Install
66
75
 
67
76
  ### Package
68
77
 
@@ -88,17 +97,17 @@ notes list the hash for each version.
88
97
  ></script>
89
98
  ```
90
99
 
91
- [Page Runtime](#-page-runtime) shows how `createSpool` injects the same tag.
100
+ [Page Runtime](#page-runtime) shows how `createSpool` injects the same tag.
92
101
 
93
102
  ### Direct Files
94
103
 
95
104
  `spool-sw.js`, `client.js` and `manifest.json` are attached to every GitHub release and published
96
105
  under `https://cdn.drupflare.com/spool/<version>/`. A site that serves them as plain files needs no
97
- server library. See [Static Sites](#-static-sites).
106
+ server library. See [Static Sites](#static-sites).
98
107
 
99
108
  ---
100
109
 
101
- ## ๐Ÿš€ Quick Start
110
+ ## Quick Start
102
111
 
103
112
  Every runtime below makes the same three moves:
104
113
 
@@ -108,6 +117,9 @@ Every runtime below makes the same three moves:
108
117
  every HTML page the application returns, and `spool.decorate` does the same where there is no
109
118
  Fetch response to wrap.
110
119
 
120
+ The [Getting Started](docs/getting-started.md) page walks through the same steps with the install
121
+ options, a run and the checks that confirm the setup.
122
+
111
123
  ### Declare Routes
112
124
 
113
125
  A route has a path pattern, a strategy, the tags it depends on and a `load` function that returns
@@ -169,7 +181,7 @@ holding a page that depends on them fetch a fresh copy before they show it again
169
181
 
170
182
  ---
171
183
 
172
- ## ๐Ÿ“ฆ Page Runtime
184
+ ## Page Runtime
173
185
 
174
186
  `handle` and `decorate` inject one script tag into each HTML page. By default the script is
175
187
  served from the same origin at `/.spool/<version>/client.js`, which suits a strict
@@ -193,8 +205,27 @@ it against the hash, so a changed file never runs. Release files are never overw
193
205
  script fails to load, the page works as it did before.
194
206
 
195
207
  The script registers `/spool-sw.js` and exposes `window.spool.ready`, which resolves to the client
196
- handle or to `null` where service workers do not exist. To bundle the runtime yourself, import
197
- `register` from `@drupflare/spool/client`.
208
+ handle or to `null` where service workers do not exist. It also exposes `window.spool.activity`,
209
+ described below, and the write calls of [Declared Writes](ADVANCED_USAGE.md#declared-writes). To bundle the runtime
210
+ yourself, import `register` from `@drupflare/spool/client`. The `window.spool` types come with that
211
+ path too, so a project that imports it gets them.
212
+
213
+ `createSpool({ workerMode: 'never' })` serves a site with no service worker. The page installs its
214
+ pack and answers held reads itself, and `/spool-sw.js` answers `404`. A declared write goes through
215
+ `window.spool.fetch` in place of `fetch`. The answer updates the held pack from the delta or digest it
216
+ carries, and any other URL goes to `fetch` unchanged. `spool.write` still queues a value for the
217
+ worker, so without a worker it answers `unavailable`. A public HTML page carries the declarations in
218
+ its ring, or in a `spool-writes` element when it has no ring. On a site with the bootstrap gate, every
219
+ public HTML page carries the ring.
220
+
221
+ ```ts
222
+ // a page with no worker
223
+ const response = await window.spool?.fetch('/courses/101', {
224
+ method: 'PATCH',
225
+ headers: { 'content-type': 'application/json' },
226
+ body: JSON.stringify({ title: 'Biology 101' })
227
+ });
228
+ ```
198
229
 
199
230
  ### Activity Hooks
200
231
 
@@ -231,462 +262,68 @@ measured, the off-screen pause lowered power for a page whose frame loop kept ru
231
262
  hidden-page pause showed no saving for a once-a-second poll. Both readings are in
232
263
  [docs/impact.md](docs/impact.md#metered-figures-beside-the-ledger).
233
264
 
234
- ---
235
-
236
- ## ๐Ÿงฉ Runtimes
237
-
238
- Each example uses the `routes`, `page` and `spool` from [Quick Start](#-quick-start). The
239
- [compatibility table](#-compatibility) lists what each framework hands a handler.
240
-
241
- ### Cloudflare Workers
242
-
243
- Bind the Durable Object that holds the version vector, then pass its namespace to
244
- `durableObjectStore`.
245
-
246
- ```jsonc
247
- {
248
- "durable_objects": { "bindings": [{ "name": "SPOOL_VERSIONS", "class_name": "SpoolVersions" }] },
249
- "migrations": [{ "tag": "v1", "new_sqlite_classes": ["SpoolVersions"] }]
250
- }
251
- ```
252
-
253
- ```ts
254
- import { createSpool } from '@drupflare/spool/server';
255
- import { durableObjectStore } from '@drupflare/spool/stores/durable-object';
256
- import { page, routes } from './routes';
257
-
258
- export { SpoolVersions } from '@drupflare/spool/stores/durable-object';
259
-
260
- export default {
261
- fetch(request: Request, env: Cloudflare.Env) {
262
- const spool = createSpool({ routes, store: durableObjectStore(env.SPOOL_VERSIONS) });
263
- return spool.handle(request, page);
264
- }
265
- };
266
- ```
267
-
268
- ### Cloudflare Pages Functions
269
-
270
- A function takes a context with the request and a `next` that runs the next function or the
271
- static asset server.
272
-
273
- ```ts
274
- import { spool } from './spool';
275
-
276
- export const onRequest = (context: { request: Request; next(): Promise<Response> }) =>
277
- spool.handle(context.request, () => context.next());
278
- ```
279
-
280
- ### Hono
281
-
282
- Hono ignores the response a middleware returns once a route has answered, so assign it to
283
- `c.res`.
284
-
285
- ```ts
286
- import { Hono } from 'hono';
287
- import { page } from './routes';
288
- import { spool } from './spool';
289
-
290
- const app = new Hono();
291
-
292
- app.use('*', async (c, next) => {
293
- c.res = await spool.handle(c.req.raw, async () => {
294
- await next();
295
- return c.res;
296
- });
297
- });
298
- app.get('/node/:id', (c) => page(c.req.raw));
299
-
300
- export default app;
301
- ```
302
-
303
- ### Elysia
304
-
305
- `onRequest` answers the spool paths and returns nothing for the rest. The hook has to be an
306
- `async` function, because Elysia does not await a promise that a plain function returns.
307
-
308
- ```ts
309
- import { Elysia } from 'elysia';
310
- import { page } from './routes';
311
- import { spool } from './spool';
312
-
313
- export const app = new Elysia()
314
- .onRequest(async ({ request }) => {
315
- const { pathname } = new URL(request.url);
316
- return pathname === '/spool-sw.js' ? spool.stub() : await spool.respond(request);
317
- })
318
- .get('/node/:id', ({ request }) => spool.decorate(request, page(request)));
319
- ```
320
-
321
- ### itty-router
322
-
323
- A handler that returns `undefined` passes the request to the next one, which is what
324
- `spool.respond` returns for a path that is not spool's.
325
-
326
- ```ts
327
- import { IttyRouter } from 'itty-router';
328
- import { page } from './routes';
329
- import { spool } from './spool';
330
-
331
- const router = IttyRouter();
332
-
333
- router
334
- .all('*', (request) => spool.respond(request))
335
- .get('/spool-sw.js', () => spool.stub())
336
- .get('/node/:id', (request) => spool.decorate(request, page(request)));
337
-
338
- export default { fetch: router.fetch };
339
- ```
340
-
341
- ### Bun
342
-
343
- Bun serves a default export that has a `fetch` function.
344
-
345
- ```ts
346
- import { page } from './routes';
347
- import { spool } from './spool';
348
-
349
- export default {
350
- port: 3000,
351
- fetch: (request: Request) => spool.handle(request, page)
352
- };
353
- ```
354
-
355
- ### Deno
356
-
357
- Run it with `deno serve`.
358
-
359
- ```ts
360
- import { page } from './routes';
361
- import { spool } from './spool';
362
-
363
- export default {
364
- fetch: (request: Request) => spool.handle(request, page)
365
- };
366
- ```
367
-
368
- ### Netlify Functions
369
-
370
- A function takes a `Request` and returns a `Response`. `config.path` sends the spool paths and the
371
- pages to it.
372
-
373
- ```ts
374
- import type { Config, Context } from '@netlify/functions';
375
- import { page } from './routes';
376
- import { spool } from './spool';
377
-
378
- export default (request: Request, context: Context) => spool.handle(request, page);
379
-
380
- export const config: Config = { path: ['/node/*', '/spool-sw.js', '/.spool/*'] };
381
- ```
382
-
383
- ### Netlify Edge Functions
384
-
385
- An edge function runs in front of the site. `context.next()` returns the response the site would
386
- have sent, which `handle` decorates.
387
-
388
- ```ts
389
- import type { Config, Context } from '@netlify/edge-functions';
390
- import { spool } from './spool';
391
-
392
- export default (request: Request, context: Context) => spool.handle(request, () => context.next());
393
-
394
- export const config: Config = { path: '/*' };
395
- ```
396
-
397
- ### Vercel Functions
398
-
399
- A function under `api/` takes a default export with `fetch`. Send every path to it from
400
- `vercel.json`, so the function sees `/.spool/*` and `/spool-sw.js` as well as the pages. Function
401
- instances do not share memory, so `store` has to be a shared `VersionStore`.
402
-
403
- ```json
404
- { "rewrites": [{ "source": "/(.*)", "destination": "/api" }] }
405
- ```
406
-
407
- ```ts
408
- import { page } from './routes';
409
- import { spool } from './spool';
410
-
411
- export default {
412
- fetch: (request: Request) => spool.handle(request, page)
413
- };
414
- ```
415
-
416
- ### Next.js
417
-
418
- Next.js route handlers return `Response` objects, so `handle` wraps one directly. The spool
419
- endpoints and the worker script go through the proxy, which returns `undefined` for every other
420
- request. Next.js 16 calls the file `proxy.ts` and the export `proxy`. Earlier versions call them
421
- `middleware.ts` and `middleware`. The proxy and the route handlers need the same shared store.
422
-
423
- ```ts
424
- import { spool } from './spool';
425
-
426
- export function proxy(request: Request) {
427
- const { pathname } = new URL(request.url);
428
- return pathname === '/spool-sw.js' ? spool.stub() : spool.respond(request);
429
- }
430
- ```
431
-
432
- ```ts
433
- import { page } from './routes';
434
- import { spool } from './spool';
435
-
436
- export const GET = (request: Request) => spool.handle(request, page);
437
- ```
438
-
439
- ### Nuxt and Nitro
440
-
441
- h3 turns an event into a standard request with `toWebRequest`, and it sends a `Response` that a
442
- handler returns. A server middleware that returns nothing passes the request on. Put the first
443
- file in `server/middleware/` and the second in `server/routes/node/[id].ts`. Both share one store
444
- from `./store`.
445
-
446
- ```ts
447
- import { defineEventHandler, toWebRequest } from 'h3';
448
- import { spool } from './spool';
449
-
450
- export default defineEventHandler((event) => {
451
- const request = toWebRequest(event);
452
- return new URL(request.url).pathname === '/spool-sw.js' ? spool.stub() : spool.respond(request);
453
- });
454
- ```
455
-
456
- ```ts
457
- import { defineEventHandler, toWebRequest } from 'h3';
458
- import { page } from './routes';
459
- import { spool } from './spool';
460
-
461
- export default defineEventHandler((event) => spool.handle(toWebRequest(event), page));
462
- ```
265
+ ### In-Page Navigation
463
266
 
464
- ### AWS Lambda
267
+ The page answers a link or a GET form in place when its content sits in a swap region. The region is the first element marked `data-spool-swap`, or else the first `<main>`:
465
268
 
466
- Lambda gives a function a JSON event and expects a JSON result, so spool needs an adapter. Hono's
467
- `hono/aws-lambda` converts Function URL, API Gateway HTTP API and ALB events. Save the Hono app
468
- above as `hono.ts` and wrap it.
469
-
470
- ```ts
471
- import { handle } from 'hono/aws-lambda';
472
- import app from './hono';
473
-
474
- export const handler = handle(app);
475
- ```
476
-
477
- ### Azure Functions
478
-
479
- An Azure v4 HTTP function takes an `HttpRequest`, which follows the Fetch standard, and returns an
480
- `HttpResponseInit`. Route every path to it with `{*path}`.
481
-
482
- ```ts
483
- import { app, type HttpRequest, type HttpResponseInit } from '@azure/functions';
484
- import { page } from './routes';
485
- import { spool } from './spool';
486
-
487
- export async function handler(request: HttpRequest): Promise<HttpResponseInit> {
488
- const hasBody = request.method !== 'GET' && request.method !== 'HEAD';
489
- const response = await spool.handle(
490
- new Request(request.url, {
491
- method: request.method,
492
- headers: [...request.headers],
493
- body: hasBody ? await request.arrayBuffer() : undefined
494
- }),
495
- page
496
- );
497
- return {
498
- status: response.status,
499
- headers: [...response.headers],
500
- body: await response.arrayBuffer()
501
- };
502
- }
503
-
504
- app.http('spool', {
505
- route: '{*path}',
506
- methods: ['GET', 'POST'],
507
- authLevel: 'anonymous',
508
- handler
509
- });
510
- ```
511
-
512
- ### Node
513
-
514
- `toNodeListener` turns a Fetch-style handler into a `node:http` listener. Its `toRequest` and
515
- `writeResponse` parts are what the frameworks below use to cross between Node and Fetch.
516
-
517
- ```ts
518
- import { createServer } from 'node:http';
519
- import { toNodeListener } from '@drupflare/spool/node';
520
- import { page } from './routes';
521
- import { spool } from './spool';
522
-
523
- export const server = createServer(toNodeListener((request) => spool.handle(request, page)));
524
- ```
525
-
526
- ### Express
527
-
528
- Express responds through its own `res`, so it has no Fetch response to wrap. `spool.respond`
529
- answers `/.spool/*` and returns `undefined` for everything else, which sends the request on to the
530
- next Express handler. A route passes its page through `spool.decorate` to get the envelope and the
531
- script tag.
532
-
533
- ```ts
534
- import express from 'express';
535
- import { toNodeListener, toRequest, writeResponse } from '@drupflare/spool/node';
536
- import { page } from './routes';
537
- import { spool } from './spool';
538
-
539
- export const app = express();
540
-
541
- app.use(toNodeListener((request) => spool.respond(request)));
542
- app.get('/spool-sw.js', (req, res) => writeResponse(res, spool.stub()));
543
- app.get('/node/:id', async (req, res) => {
544
- const request = toRequest(req, res);
545
- await writeResponse(res, await spool.decorate(request, page(request)));
546
- });
547
- ```
548
-
549
- ### Fastify
550
-
551
- Fastify's `reply.send` takes a Fetch `Response`. An `onRequest` hook answers the spool paths from
552
- the raw request before Fastify reads the body.
553
-
554
- ```ts
555
- import Fastify from 'fastify';
556
- import { toRequest } from '@drupflare/spool/node';
557
- import { page } from './routes';
558
- import { spool } from './spool';
559
-
560
- export const app = Fastify();
561
-
562
- app.addHook('onRequest', async (req, reply) => {
563
- const response = await spool.respond(toRequest(req.raw, reply.raw));
564
- if (response) return reply.send(response);
565
- });
566
- app.get('/spool-sw.js', (req, reply) => reply.send(spool.stub()));
567
- app.get('/node/:id', async (req, reply) => {
568
- const request = toRequest(req.raw, reply.raw);
569
- return reply.send(await spool.decorate(request, page(request)));
570
- });
571
- ```
572
-
573
- ### Koa
574
-
575
- Koa exposes the Node objects as `ctx.req` and `ctx.res`. Setting `ctx.respond` to `false` stops
576
- Koa from writing a second response.
577
-
578
- ```ts
579
- import Koa from 'koa';
580
- import { toRequest, writeResponse } from '@drupflare/spool/node';
581
- import { page } from './routes';
582
- import { spool } from './spool';
583
-
584
- export const app = new Koa();
585
-
586
- app.use(async (ctx, next) => {
587
- const request = toRequest(ctx.req, ctx.res);
588
- const response = ctx.path === '/spool-sw.js' ? spool.stub() : await spool.respond(request);
589
- if (!response) return next();
590
- ctx.respond = false;
591
- await writeResponse(ctx.res, response);
592
- });
593
- app.use(async (ctx) => {
594
- if (!ctx.path.startsWith('/node/')) return;
595
- const request = toRequest(ctx.req, ctx.res);
596
- ctx.respond = false;
597
- await writeResponse(ctx.res, await spool.decorate(request, page(request)));
598
- });
599
- ```
600
-
601
- ### hapi
602
-
603
- hapi keeps the Node objects at `request.raw`. Returning `h.abandon` tells it the response is
604
- already written.
605
-
606
- ```ts
607
- import Hapi from '@hapi/hapi';
608
- import { toRequest, writeResponse } from '@drupflare/spool/node';
609
- import { page } from './routes';
610
- import { spool } from './spool';
611
-
612
- export const server = Hapi.server();
613
-
614
- server.ext('onRequest', async (request, h) => {
615
- const { req, res } = request.raw;
616
- const spoolRequest = toRequest(req, res);
617
- const response =
618
- request.path === '/spool-sw.js' ? spool.stub() : await spool.respond(spoolRequest);
619
- if (!response) return h.continue;
620
- await writeResponse(res, response);
621
- return h.abandon;
622
- });
623
- server.route({
624
- method: 'GET',
625
- path: '/node/{id}',
626
- handler: async (request, h) => {
627
- const { req, res } = request.raw;
628
- const spoolRequest = toRequest(req, res);
629
- await writeResponse(res, await spool.decorate(spoolRequest, page(spoolRequest)));
630
- return h.abandon;
631
- }
632
- });
269
+ ```html
270
+ <body>
271
+ <header>
272
+ <a href="/">Site</a>
273
+ </header>
274
+ <main data-spool-swap>
275
+ <h1>Course 101</h1>
276
+ </main>
277
+ </body>
633
278
  ```
634
279
 
635
- ### NestJS
636
-
637
- On the Express adapter, `app.use` takes spool's listener and a controller method that injects
638
- `@Req()` and `@Res()` writes the page. The example needs `experimentalDecorators` in
639
- `tsconfig.json`.
640
-
641
- ```ts
642
- import { Controller, Get, Module, Req, Res } from '@nestjs/common';
643
- import { NestFactory } from '@nestjs/core';
644
- import type { Request, Response } from 'express';
645
- import 'reflect-metadata';
646
- import { toNodeListener, toRequest, writeResponse } from '@drupflare/spool/node';
647
- import { page } from './routes';
648
- import { spool } from './spool';
649
-
650
- @Controller()
651
- class NodeController {
652
- @Get('node/:id')
653
- async show(@Req() req: Request, @Res() res: Response) {
654
- const request = toRequest(req, res);
655
- await writeResponse(res, await spool.decorate(request, page(request)));
656
- }
657
- }
280
+ The page replaces the region, the element marked `data-spool-swap-sidebar` when both pages have one, and the title. It answers only public routes declared with `family`, from the rows the device holds ([Dataset Routes](ADVANCED_USAGE.md#dataset-routes)). Modified clicks, links with `target` or `download`, cross-origin links and non-GET forms go to the browser, and so does any route the device does not hold. The page writes at most 90 history entries in any 10 seconds. Past that count it swaps the content and leaves the address bar on the previous URL until the window clears.
658
281
 
659
- @Module({ controllers: [NodeController] })
660
- class AppModule {}
282
+ A page with no swap element navigates by document, and no error is shown.
661
283
 
662
- export async function bootstrap() {
663
- const app = await NestFactory.create(AppModule, { logger: false });
664
- app.use(toNodeListener((request) => spool.respond(request)));
665
- app.use('/spool-sw.js', (req: Request, res: Response) => writeResponse(res, spool.stub()));
666
- return app;
667
- }
668
- ```
669
-
670
- ### Google Cloud Functions
284
+ ---
671
285
 
672
- The Functions Framework gives an HTTP function Express-style `req` and `res`, so the Node listener
673
- fits as it is.
674
-
675
- ```ts
676
- import { http } from '@google-cloud/functions-framework';
677
- import { toNodeListener } from '@drupflare/spool/node';
678
- import { page } from './routes';
679
- import { spool } from './spool';
680
-
681
- http(
682
- 'spool',
683
- toNodeListener((request) => spool.handle(request, page))
684
- );
685
- ```
286
+ ## Runtimes
287
+
288
+ Each runtime has its own page, with the framework context and a minimal example. The
289
+ [Adapters](docs/adapters/README.md) index lists every page with its import path.
290
+
291
+ | Runtime | Page |
292
+ | -------------------------- | ---------------------------------------------------------------------------- |
293
+ | Cloudflare Workers | [cloudflare-workers.md](docs/adapters/cloudflare-workers.md) |
294
+ | Cloudflare Pages Functions | [cloudflare-pages-functions.md](docs/adapters/cloudflare-pages-functions.md) |
295
+ | Hono | [hono.md](docs/adapters/hono.md) |
296
+ | Elysia | [elysia.md](docs/adapters/elysia.md) |
297
+ | itty-router | [itty-router.md](docs/adapters/itty-router.md) |
298
+ | Bun | [bun.md](docs/adapters/bun.md) |
299
+ | Deno | [deno.md](docs/adapters/deno.md) |
300
+ | Netlify Functions | [netlify-functions.md](docs/adapters/netlify-functions.md) |
301
+ | Netlify Edge Functions | [netlify-edge-functions.md](docs/adapters/netlify-edge-functions.md) |
302
+ | Vercel Functions | [vercel-functions.md](docs/adapters/vercel-functions.md) |
303
+ | Next.js | [nextjs.md](docs/adapters/nextjs.md) |
304
+ | Nuxt and Nitro | [nuxt-and-nitro.md](docs/adapters/nuxt-and-nitro.md) |
305
+ | Nuxt Module | [nuxt-module.md](docs/adapters/nuxt-module.md) |
306
+ | React Router | [react-router.md](docs/adapters/react-router.md) |
307
+ | Next Adapter | [next-adapter.md](docs/adapters/next-adapter.md) |
308
+ | Vite | [vite.md](docs/adapters/vite.md) |
309
+ | Vue | [vue.md](docs/adapters/vue.md) |
310
+ | Angular | [angular.md](docs/adapters/angular.md) |
311
+ | AWS Lambda | [aws-lambda.md](docs/adapters/aws-lambda.md) |
312
+ | Azure Functions | [azure-functions.md](docs/adapters/azure-functions.md) |
313
+ | Node | [node.md](docs/adapters/node.md) |
314
+ | Express | [express.md](docs/adapters/express.md) |
315
+ | Fastify | [fastify.md](docs/adapters/fastify.md) |
316
+ | Koa | [koa.md](docs/adapters/koa.md) |
317
+ | hapi | [hapi.md](docs/adapters/hapi.md) |
318
+ | NestJS | [nestjs.md](docs/adapters/nestjs.md) |
319
+ | Google Cloud Functions | [google-cloud-functions.md](docs/adapters/google-cloud-functions.md) |
320
+
321
+ Pick the page for your framework. A runtime with a Fetch `fetch(request)` handler needs only
322
+ `spool.handle`, as the [Quick Start](#quick-start) shows.
686
323
 
687
324
  ---
688
325
 
689
- ## ๐Ÿ“Š Compatibility
326
+ ## Compatibility
690
327
 
691
328
  The gate column says how far each row is checked. **runs** means the fixture above drives the
692
329
  framework's own package. **stand-in** means a fixture calls the platform's handler with a plain
@@ -695,14 +332,14 @@ from the framework's documentation and nothing here has run it.
695
332
 
696
333
  | Framework or host | What a handler receives | Spool entry | Gate |
697
334
  | ------------------------------------ | -------------------------------------------------------------- | ------------------------------------------------------------ | -------------------------------------------- |
698
- | Express | Node `req`, `res` | `toNodeListener`, `respond`, `decorate` | runs |
335
+ | Express | Node `req`, `res` | `spoolExpress` | runs |
699
336
  | Fastify | Node `req.raw`, `reply.raw`; `reply.send` takes a `Response` | `toRequest`, `respond`, `decorate` | runs |
700
337
  | Koa | Node `ctx.req`, `ctx.res` | `toRequest`, `writeResponse` | runs |
701
338
  | hapi | Node `request.raw.req`, `request.raw.res` | `toRequest`, `writeResponse` | runs |
702
339
  | NestJS, Express adapter | Node, through Express | `app.use` with `toNodeListener`; `toRequest` in a controller | runs |
703
340
  | NestJS, Fastify adapter | Node, through Fastify | the Fastify entry | not tested |
704
341
  | AdonisJS | Node at `ctx.request.request` and `ctx.response.response` | `toRequest`, `writeResponse` in a middleware | not tested |
705
- | Hono | Fetch | `handle` | runs |
342
+ | Hono | Fetch | `spoolHono` | runs |
706
343
  | Elysia | Fetch | `respond`, `decorate` | runs |
707
344
  | itty-router | Fetch | `respond`, `decorate` | runs |
708
345
  | Oak | Fetch, through `app.handle`; published on JSR only | `handle` | not tested |
@@ -722,8 +359,8 @@ from the framework's documentation and nothing here has run it.
722
359
  | Deno, Deno Deploy | Fetch, through `Deno.serve` or `deno serve` | `handle` | stand-in |
723
360
  | Bun | Fetch | `handle` | runs, under `Bun.serve` |
724
361
  | Vercel Functions | Fetch | `handle` | stand-in |
725
- | AWS Lambda, Function URL or HTTP API | JSON event version 2.0 | `hono/aws-lambda` around a Hono app | stand-in event |
726
- | AWS Lambda, API Gateway REST API | JSON event version 1.0 | `hono/aws-lambda` around a Hono app | not tested |
362
+ | AWS Lambda, Function URL or HTTP API | JSON event version 2.0 | `@hono/aws-lambda` around a Hono app | stand-in event |
363
+ | AWS Lambda, API Gateway REST API | JSON event version 1.0 | `@hono/aws-lambda` around a Hono app | not tested |
727
364
  | Lambda@Edge | CloudFront event with header arrays | none; a converter to `Request` is needed | not tested |
728
365
  | Fastly Compute | Fetch, through `event.request` in a `fetch` listener | `event.respondWith(spool.handle(event.request, next))` | not tested |
729
366
  | Azure Functions v4 | `HttpRequest`, which follows Fetch; returns `HttpResponseInit` | convert, then `handle` | runs, with a real `HttpRequest` |
@@ -731,11 +368,11 @@ from the framework's documentation and nothing here has run it.
731
368
  | Google Cloud Run | any HTTP server on `PORT` | the Node example | not tested |
732
369
 
733
370
  A host that serves static files and runs no code, such as an object store or a plain web server, uses
734
- [Static Sites](#-static-sites).
371
+ [Static Sites](#static-sites).
735
372
 
736
373
  ---
737
374
 
738
- ## ๐Ÿงฑ Static Sites
375
+ ## Static Sites
739
376
 
740
377
  A site with no spool server can still answer repeat views from the device. It serves three files
741
378
  from one directory:
@@ -744,7 +381,7 @@ from one directory:
744
381
  - `client.js`, the page script, from the same place.
745
382
  - `spool.json`, the list of files the worker may keep.
746
383
 
747
- Each page loads the script with the tag from [CDN Script](#-install), or from the site's own copy.
384
+ Each page loads the script with the tag from [CDN Script](#cdn-script), or from the site's own copy.
748
385
  The worker has no server to talk to, so it reads `spool.json` from the directory its own script
749
386
  sits in.
750
387
 
@@ -796,1133 +433,137 @@ importScripts('https://cdn.drupflare.com/spool/<version>/spool-sw.js');
796
433
 
797
434
  `importScripts` has no integrity check, so this form trusts the CDN host with the worker.
798
435
 
799
- ---
800
-
801
- ## ๐Ÿ”€ Strategies
802
-
803
- A route is answered locally in one of five ways. A `template` route stores JSON and renders it
804
- through a compiled template. An `assembly` route stores a shell with named holes, fragment markup
805
- for the holes and values for named slots. A `static` route stores a file as the server sent it. A
806
- `data` route stores JSON and answers a GET for it. A `projection` route stores JSON and renders a
807
- page from it with a function your site bundles into its own worker.
808
-
809
- ```ts
810
- import { placeholder, slot } from '@drupflare/spool/protocol';
811
- import { defineRoute } from '@drupflare/spool/server';
812
-
813
- export const about = defineRoute('/about', {
814
- strategy: 'assembly',
815
- depends: () => ['menu'],
816
- load: () => ({
817
- shell: `<!doctype html><html><head><title>About</title></head><body>${placeholder('menu')}<main>Updated ${slot('updated')}</main></body></html>`,
818
- fragments: { menu: '<nav><a href="/">Home</a></nav>' },
819
- slots: { updated: '2026-10-19' }
820
- })
821
- });
822
- ```
823
-
824
- The worker leaves a hole it cannot fill in place and refuses to render a page with a gap, so the
825
- request goes to the network instead.
826
-
827
- A page the worker renders keeps the site's security headers. The worker remembers
828
- `Content-Security-Policy` (and its report-only form), `X-Frame-Options`, `X-Content-Type-Options`,
829
- `Referrer-Policy`, `Permissions-Policy` and the three `Cross-Origin-*` policies from the network
830
- answer to each path, and sends them with every local render of it. A path with no answer of its own
831
- uses the latest set from any navigation. A page with neither goes to the network until the worker
832
- has seen one, so a local view never goes out without the site's policy.
833
-
834
- ### Records and Projections
835
-
836
- Pages that read the same data share one stored copy. `ctx.record(id, data)` stores an
837
- authoritative value once under its id, and `ctx.projection(id, data)` stores a value derived from
838
- records. Both return a reference to put in the page's data. A record's id is its invalidation tag.
839
- Invalidating `course:101` refreshes the records and projections that read it, and no others.
840
-
841
- ```ts
842
- import { defineRoute } from '@drupflare/spool/server';
843
-
844
- const courses: Record<string, string> = { '101': 'Intro', '102': 'Advanced' };
845
-
846
- export const course = defineRoute('/courses/:id', {
847
- strategy: 'template',
848
- template: {
849
- source: '<h1>{{title}}</h1><nav>{{#each menu}}<a>{{this}}</a>{{/each}}</nav>'
850
- },
851
- load: (ctx) => {
852
- const id = ctx.params['id'] ?? '';
853
- return {
854
- title: ctx.record(`course:${id}`, courses[id]),
855
- menu: ctx.projection(
856
- 'menu',
857
- Object.keys(courses).map((key) => ctx.record(`course:${key}`, courses[key]))
858
- )
859
- };
860
- }
861
- });
862
- ```
863
-
864
- The server sends only the records and projections the device does not already hold at a current
865
- version. A sync for the next page transfers the entries that page reads and that changed.
866
-
867
- ### Templates
868
-
869
- Templates use Mustache-shaped syntax with no expressions, so the worker needs no `eval`.
870
-
871
- | Syntax | Meaning |
872
- | ------------------------------------ | -------------------------------------------------------- |
873
- | `{{field}}` | the field, HTML-escaped; `a.b` reaches into objects |
874
- | `{{{field}}}` | the field as markup; allowed only for names in `trusted` |
875
- | `{{#each list}}...{{/each}}` | repeats for each item; `{{this}}` and `{{@index}}` work |
876
- | `{{#if field}}...{{else}}...{{/if}}` | renders a branch on whether the field is truthy |
877
- | `{{! note }}` | a comment that renders nothing |
878
-
879
- ```ts
880
- import { defineRoute } from '@drupflare/spool/server';
881
-
882
- export const news = defineRoute('/news/:slug', {
883
- strategy: 'template',
884
- template: {
885
- source:
886
- '<article><h1>{{title}}</h1>{{#if byline}}<p>{{byline}}</p>{{else}}<p>Staff</p>{{/if}}<div>{{{body}}}</div><ul>{{#each tags}}<li>{{@index}} {{this}}</li>{{/each}}</ul></article>',
887
- trusted: ['body']
888
- },
889
- depends: (ctx) => [`news:${ctx.params['slug']}`],
890
- load: () => ({
891
- title: 'Hello',
892
- byline: null,
893
- body: '<p>Trusted markup</p>',
894
- tags: ['a', 'b']
895
- })
896
- });
897
- ```
898
-
899
- A template that names a field it does not trust for markup fails when the route is declared.
900
-
901
- A public `template` route can carry its own data in the page. `inlineMaxBytes` on `createSpool`
902
- sets the largest artifact, in bytes, that `handle` and `decorate` write into the HTML as an
903
- escaped `<script type="application/spool+json">` element. The worker reads it from the response
904
- and skips the `/.spool/artifact` request. It is 0, which is off, because every network view of the
905
- page carries the element, used or not. The page the visitor sees and the shadow comparison ignore
906
- it.
907
-
908
- ### Static Files
909
-
910
- A `static` route has no `load` and no template. The application serves the file as it always did,
911
- and `handle` or `decorate` marks the response for the worker, which keeps the bytes. A tag in
912
- `depends` ties the file to a deploy, and `spool.invalidate` after one makes devices fetch the
913
- file again. A static route is always public and is always answered from the device, so it takes no
914
- `scope`, `persist` or `mode`. A query string makes a different file: `/app.js?v=2` is not
915
- answered with the copy of `/app.js`, and a query the worker has not stored goes to the network.
916
-
917
- ```ts
918
- import { defineRoute } from '@drupflare/spool/server';
919
-
920
- export const about = defineRoute('/about', {
921
- strategy: 'static',
922
- depends: () => ['site'],
923
- freshness: { maxStale: 3600, mode: 'soft' }
924
- });
925
- ```
926
-
927
- ### Projection and Data Routes
928
-
929
- A `data` route answers a GET for JSON, such as an API read, from the stored value, so the page's
930
- `fetch` makes no request. The worker answers with `JSON.stringify` of what `load` returned, which
931
- means the application's own handler has to return the same JSON. A navigation to the URL still goes
932
- to the network.
933
-
934
- A `projection` route needs a renderer. Spool ships no bundler: write a worker with `serve` from
935
- `@drupflare/spool/sw`, bundle it with your own toolchain, and pass the bundle to `createSpool` as
936
- `worker`. The server puts the config in front of your bundle as `self.__SPOOL_CONFIG__`, and
937
- `spool.config` is the same object as JSON if you would rather inline it at build time. A route with
938
- no renderer is refused by the worker and the network answers.
939
-
940
- ```ts
941
- import { defineRoute } from '@drupflare/spool/server';
942
-
943
- export const course = defineRoute('/api/courses/:id', {
944
- strategy: 'data',
945
- depends: (ctx) => [`course:${ctx.params['id']}`],
946
- load: (ctx) => ({ id: ctx.params['id'], title: 'Intro' }),
947
- provision: { reuse: 0.8, futureViews: 6 }
948
- });
949
-
950
- export const roster = defineRoute('/rosters/:id', {
951
- strategy: 'projection',
952
- depends: (ctx) => [`roster:${ctx.params['id']}`],
953
- load: (ctx) => ({ id: ctx.params['id'], names: ['Ada', 'Bo'] })
954
- });
955
- ```
956
-
957
- ```ts
958
- import { escapeHtml } from '@drupflare/spool/protocol';
959
- import { serve } from '@drupflare/spool/sw';
960
-
961
- declare const CONFIG: string;
962
-
963
- serve(JSON.parse(CONFIG), {
964
- '/rosters/:id': (artifact) => {
965
- const { names } = (artifact.body as { data: { names: string[] } }).data;
966
- return `<!doctype html><html><head><title>Roster</title></head><body><ul>${names.map((name) => `<li>${escapeHtml(name)}</li>`).join('')}</ul></body></html>`;
967
- }
968
- });
969
- ```
970
-
971
- The server also decides whether to send an artifact at all. It weighs the bytes it would send,
972
- records included, against what a local view saves: reuse times future views times the bytes a view
973
- avoids, less the cost of the request. When that is not positive it answers `204` with
974
- `Spool-Provision: declined`, and the worker leaves that path to the network until it restarts.
975
- The priors are scenario assumptions: set them from your own telemetry in `provision`, or per route
976
- as above. A `data` route weighs against its own size, so its reuse times future views must exceed
977
- one. The defaults of 0.5 and 2 give exactly one, which breaks even, and the route is left out.
978
-
979
- ### Query Routes
980
-
981
- A URL with a query string goes to the network. A public `template`, `projection` or `data` route
982
- can declare the query strings it answers, and the device then answers a new filter, sort or page of
983
- a collection it already holds with no request.
984
-
985
- ```ts
986
- import { datasetProof, defineRoute } from '@drupflare/spool/server';
987
-
988
- const people = [
989
- { name: 'Ada', department: 'math', age: 36 },
990
- { name: 'Bo', department: 'cs', age: 25 }
991
- ];
992
-
993
- export const directory = defineRoute('/people', {
994
- strategy: 'template',
995
- template: {
996
- source:
997
- '<h1>{{#if query}}Page {{query.page}} of {{query.pages}}{{else}}Everyone{{/if}}</h1><ul>{{#each people}}<li>{{name}}</li>{{/each}}</ul>'
998
- },
999
- depends: () => ['people'],
1000
- load: async () => ({ people, proof: await datasetProof(people) }),
1001
- query: {
1002
- list: 'people',
1003
- filter: { department: 'string', age: 'number' },
1004
- sort: ['name', 'age'],
1005
- pageSize: 20
1006
- },
1007
- provision: { reuse: 0.8, futureViews: 6 }
1008
- });
1009
- ```
1010
-
1011
- `list` names the key of the loaded data that holds the whole collection, an array of flat objects.
1012
- Each key of `filter` is a query parameter that keeps the items whose field of the same name equals
1013
- it, read as a `string` or a `number`. A repeated parameter matches any of its values, and different
1014
- parameters combine with and. `sort` lists the fields the `sort` parameter may name, and `-name`
1015
- sorts descending. Ties keep the stored order. Strings compare by UTF-16 code unit, with no locale
1016
- and no case folding, so the application's own sort has to agree. A database that compares bytes,
1017
- such as SQLite by default, agrees except for characters above U+FFFF, and a case-insensitive or
1018
- locale collation does not. `pageSize` sets the items per page and `page` counts from 1; without it
1019
- the route is one page of everything. A route with `principal` scope, or any
1020
- other strategy, cannot declare `query`.
1021
-
1022
- `load` returns the array at `list` and a `proof` made by `datasetProof`. The server refuses to
1023
- build the artifact when the proof is missing or does not match the array. The worker computes the
1024
- proof again from the stored copy before it answers a query. The proof shows that the device holds
1025
- the array the server sent. That the array is the whole collection is the route's declaration:
1026
- `list` has to hold every item the application's own answer could draw from, or a filter returns
1027
- fewer rows than the server would.
1028
-
1029
- The template, or the renderer of a `projection` route, receives the stored data with the array
1030
- replaced by the page's items and one added key, `query`, holding `{ total, page, pages }`. `total`
1031
- counts the items after filtering. A `data` route answers with that same object as JSON. The URL
1032
- without a query string renders the stored data as it is and has no `query` key, which is what
1033
- `{{#if query}}` tests above.
1034
-
1035
- The worker answers locally only when the artifact for the path is held and fresh under the route's
1036
- freshness contract, the proof matches and the query is one the route declares. Everything else goes
1037
- to the network as before: a parameter the route does not name, a number filter that is not a
1038
- finite number, a `sort` given twice or not listed, a `page` that is not a positive integer or
1039
- lies past the last page, and an item that is not an object. A copy that fails its proof is dropped
1040
- and fetched whole again. The worker never stores anything under a URL with a query string, and in
1041
- shadow mode it compares its render with the network answer for the same URL.
1042
-
1043
- A query route's artifact is the whole collection, so it is large. The provisioning rule weighs it
1044
- like any other artifact, and `provision` on the route sets the priors. A collection repays its
1045
- transfer after `entryBreakEven(bytes, documentBytes)` local answers per device: a 40 KB collection
1046
- that replaces 17 KB documents needs 2.3 (L2). Pages of one visit come from one held copy until a
1047
- sync replaces it, so paging never mixes two versions on a device.
1048
-
1049
- ---
1050
-
1051
- ## โณ Freshness Contracts
1052
-
1053
- Each route declares how long a stored page may go without validation. `maxStale` is in seconds.
1054
-
1055
- - **hard:** once `maxStale` passes, the worker validates the page with the server before it
1056
- shows it. Offline, the route's `offline` rule decides.
1057
- - **soft:** the worker shows the stored page at once and syncs afterward. A soft bound is a
1058
- stale-while-revalidate window and promises nothing about how fresh the shown page is.
1059
-
1060
- Public routes default to 15 minutes soft, and routes with `scope: 'principal'` to five minutes
1061
- hard; [Authenticated Data](#-authenticated-data) covers how their pages are stored. `offline`
1062
- defaults to `serve` for public routes and `refuse` for principal routes. A soft route may set `sync: 'deferred'` to queue its refresh
1063
- for the next network request that happens anyway.
1064
-
1065
- ```ts
1066
- import { defineRoute } from '@drupflare/spool/server';
1067
-
1068
- export const stock = defineRoute('/stock/:sku', {
1069
- strategy: 'template',
1070
- template: { source: '<p>{{sku}}: {{count}}</p>' },
1071
- depends: (ctx) => [`stock:${ctx.params['sku']}`],
1072
- load: (ctx) => ({ sku: ctx.params['sku'], count: 3 }),
1073
- freshness: { maxStale: 60, mode: 'hard' },
1074
- offline: 'refuse'
1075
- });
1076
-
1077
- export const notice = defineRoute('/notices/:id', {
1078
- strategy: 'template',
1079
- template: { source: '<p>{{text}}</p>' },
1080
- depends: (ctx) => [`notice:${ctx.params['id']}`],
1081
- load: () => ({ text: 'Closed on Friday' }),
1082
- freshness: { maxStale: 3600, mode: 'soft' },
1083
- sync: 'deferred'
1084
- });
1085
- ```
1086
-
1087
- The worker syncs the smallest set of changed dependencies the next page needs. One worker
1088
- leads the sync for every tab of an origin. A response to a request that happens anyway can carry
1089
- invalidations, so a sync does not always need a request of its own.
1090
-
1091
- ---
1092
-
1093
- ## ๐Ÿ” Authenticated Data
1094
-
1095
- Set `scope: 'principal'` on a route that depends on who is signed in, and give `createSpool` an
1096
- `identity` option. The server sends `Spool-Partition`, an HMAC of the principal under your secret,
1097
- so the browser can tell users apart without learning who they are. The worker keeps each partition
1098
- in its own IndexedDB database with the artifacts, records, projections and sync state sealed with
1099
- AES-GCM. Each value is bound to its partition, kind and id, so a row copied into another slot does
1100
- not decrypt.
1101
-
1102
- `retention` picks the key. `persistent` is a non-extractable key in IndexedDB that survives a
1103
- restart and is deleted on logout or a change of partition. `session`, the default, is a key held in
1104
- the worker's memory, which the browser discards when it stops the worker. Deleting a key makes what
1105
- it sealed unreadable, which is crypto-shredding of data at rest. It does not protect against script
1106
- running on your origin, which can ask the worker for whatever the page can. The ids and versions
1107
- that serve as storage keys stay readable, so use opaque ids for anything sensitive.
1108
-
1109
- A principal route defaults to a five-minute hard bound and `offline: 'refuse'`. Revocation rides
1110
- the same bound: tag a page with the permission it needs, call `spool.invalidate` when the
1111
- permission goes, and the first required sync past the bound finds the page invalid.
1112
-
1113
- A different `Spool-Partition` wipes the partition before it. A login or account-switch handler has
1114
- to send the new value itself with `spool.partition(principal)`, because the request it answers
1115
- still carries the old identity. A sync answer carries `Spool-Partition` for the request's identity
1116
- too, so a sign-in the worker never saw ends at the first required sync, no later than the route's
1117
- `maxStale` plus the next view. Paths in `logout` are wiped when the request reaches the worker,
1118
- before it is sent, so logout empties every tab even with the network down. The worker first sends
1119
- the same-origin posts it holds in `beacons`, while they still carry the session that is ending,
1120
- and drops any that do not get through. The page runtime
1121
- dispatches a `spool:cleared` event on `window` in every tab when that happens, so a page can blank
1122
- what it shows. A response can also send
1123
- `Spool-Clear`: `*` wipes everything, a partition id wipes one. `decorate(request, response, { clear:
1124
- ['*'], hardReset: true })` adds `Clear-Site-Data: "storage"` as well, the browser's own reset
1125
- of everything the origin holds, service worker included. Its Safari support is unverified, so rely
1126
- on the worker's wipe.
1127
-
1128
- ```ts
1129
- import { createSpool, defineRoute } from '@drupflare/spool/server';
1130
- import { memoryStore } from '@drupflare/spool/stores';
1131
-
1132
- export const grades = defineRoute('/grades', {
1133
- strategy: 'template',
1134
- scope: 'principal',
1135
- retention: 'persistent',
1136
- template: { source: '<h1>Grades for {{owner}}</h1>' },
1137
- depends: (ctx) => [`perm:${ctx.identity?.principal}`, `grades:${ctx.identity?.principal}`],
1138
- load: (ctx) => ({ owner: ctx.identity?.principal })
1139
- });
1140
-
1141
- export const spool = createSpool({
1142
- routes: [grades],
1143
- store: memoryStore(),
1144
- identity: {
1145
- secret: 'a long random value kept on the server',
1146
- resolve: (request) => {
1147
- const user = /user=(\w+)/.exec(request.headers.get('cookie') ?? '')?.[1];
1148
- return user ? { principal: user } : null;
1149
- }
1150
- },
1151
- logout: ['/logout']
1152
- });
1153
-
1154
- export async function login(user: string): Promise<Response> {
1155
- return new Response('Signed in', {
1156
- headers: {
1157
- 'set-cookie': `user=${user}; Path=/`,
1158
- 'spool-partition': await spool.partition(user)
1159
- }
1160
- });
1161
- }
1162
- ```
1163
-
1164
- A sync that asks for everything changed lists the tags of every change, so tags for principal data
1165
- should be opaque too. The worker syncs a principal page with `need`, which names only the tags
1166
- that page reads.
1167
-
1168
- ---
1169
-
1170
- ## ๐Ÿ“š Catalogue
1171
-
1172
- A catalogue sends the data of pages a visitor may open next, so that opening one makes no
1173
- request. An entry costs its bytes whether or not the page is opened, and an opened one saves a
1174
- document request, so an entry pays only when enough visitors open it. The break-even share is
1175
- `k * E / (k * D + o)`, where `E` is the entry's compressed bytes, `D` the bytes of the document
1176
- it replaces, `k` the joules to move a byte and `o` the origin request a view avoids. An entry the
1177
- size of the document needs 96% of visitors to open it, which no page gets. An entry that is only
1178
- the data for a template the device already holds is a few hundred bytes to a few kilobytes and
1179
- pays at a few percent. On the `payload` reading, the rendered fields of a node were a median 473
1180
- bytes compressed on 19 pages from two sites (L1, `measurements/payload`), against a document of
1181
- about 17,000 bytes. References such as media and terms are left out of that figure, so a real entry
1182
- is larger. An entry is data for a `template`, `projection` or `data` route and never HTML.
1183
-
1184
- The site supplies the candidates. A route's `catalogue` returns the pages a visitor on it may open
1185
- next, each with the share expected to. Take the share from your own `nextShare` over the sink
1186
- (each `spool.view` names the route the visitor came from), from link position, or from a fixed
1187
- prior. Pass `{ first: true }` to `nextShare` to count only first views, which is what an entry
1188
- saves: the share over all views ran 1.5 to 7 times high in the bins that decide an entry in a replay
1189
- of public visitors ([docs/impact.md](docs/impact.md#next-page-prediction)). `documentBytes` on the entry's route says what a hit saves, and the provisioning prior is
1190
- used when it is absent.
1191
-
1192
- A share taken from counts can carry `n`, the observations behind it. `catalogueConfidence` on
1193
- `createSpool` is a number of standard deviations, and with it above 0 an entry whose guess has an
1194
- `n` is judged on the lower confidence bound of its `p`, so a share seen on 5 visits stops counting
1195
- as one seen on 5,000. It defaults to 0, which judges every guess on its `p`. `NavModel` in
1196
- `@drupflare/spool/protocol` keeps first-order counts with decay and a prior and returns a `p` with
1197
- the weight to pass as `n`. In a replay of 989,818 public visitors, a model like that earned more
1198
- per provisioned kilobyte than the most popular page
1199
- ([docs/impact.md](docs/impact.md#next-page-prediction)).
1200
-
1201
- ```ts
1202
- import { createSpool, defineRoute } from '@drupflare/spool/server';
1203
- import { memoryStore } from '@drupflare/spool/stores';
1204
-
1205
- export const home = defineRoute('/', {
1206
- strategy: 'template',
1207
- template: { source: '<h1>Courses</h1>' },
1208
- load: () => ({}),
1209
- catalogue: () => [
1210
- { path: '/courses/101', p: 0.3 },
1211
- { path: '/courses/102', p: 0.1 }
1212
- ]
1213
- });
1214
-
1215
- export const course = defineRoute('/courses/:id', {
1216
- strategy: 'template',
1217
- template: { source: '<h1>{{title}}</h1>' },
1218
- depends: (ctx) => [`course:${ctx.params['id']}`],
1219
- load: (ctx) => ({ title: `Course ${ctx.params['id']}` }),
1220
- documentBytes: 17_000
1221
- });
1222
-
1223
- export const spool = createSpool({
1224
- routes: [home, course],
1225
- store: memoryStore(),
1226
- catalogueBytes: 16_384
1227
- });
1228
- ```
1229
-
1230
- The server sends an entry only when its share reaches its own break-even, best expected net first,
1231
- until `catalogueBytes` is spent. The cap counts compressed bytes and defaults to 32 KiB. It builds
1232
- at most 32 pages for one request, measures each entry with Brotli where the runtime has it and
1233
- gzip where it does not, and skips a `static` or `assembly` route. A request with `Save-Data: on`
1234
- gets no entries.
1235
-
1236
- The ask rides the beacon batch the worker already sends. A due ask also makes the next network
1237
- moment send that batch, when a view is held to carry it, so a visit of two or three pages gets its
1238
- entries without waiting for 20 views or an hour. That costs one `POST /.spool/beacon` for each
1239
- catalogue page, device and day, sent at a request that happens anyway. A local view never sends
1240
- it, and a moment with nothing held sends nothing. The worker stores the entries like provisioned
1241
- pages, and the first view of an unopened one is reported with the reason `catalogue`. `catalogueFigures` reads the views and the worker's ledger for the entries sent
1242
- and used, the hit share, and the bytes sent against the bytes saved. Until shadow views measure
1243
- the shares, the saving is a scenario.
1244
-
1245
- A hit saves the document request and nothing else. On the Drupal pages measured (75 first views of
1246
- internal pages on 10 sites, L1, `measurements/trim`), a view of a page still fetched that page's
1247
- new images and stylesheet and script aggregates: only 3% of those first views moved nothing but
1248
- the document. A catalogue view takes the document off the wire, and the rest of the page is
1249
- the site's own HTTP cache and [asset trimming](#-trimmed-stylesheets).
1250
-
1251
- A `principal` route can be an entry. The worker then names every catalogue entry no page has
1252
- opened at each sync of a principal page, so a revoked one is dropped at the first required sync.
1253
- The request that asks for a catalogue names the path of the page the visitor is on, including one
1254
- the worker showed locally, so a route that should not tell the server leaves `catalogue` off.
1255
- [docs/security.md](docs/security.md#catalogue-entries) covers both.
1256
-
1257
- ---
1258
-
1259
- ## ๐Ÿ“ก Analytics Batching
1260
-
1261
- A page that keeps analytics scripts posting to an endpoint makes a request on every view, which
1262
- defeats a local view. List the same-origin POST paths in `beacons`. The worker answers each post
1263
- with a `204` and queues it in IndexedDB. At the next network request that happens anyway it sends
1264
- the post to the URL the page meant, with its method, content type, query and body, so the vendor
1265
- receives what the page sent. Spool's own events go to `/.spool/beacon` in one batch, where a
1266
- `sink` receives them. A held post goes out on its own at most once every 24 hours, when the oldest
1267
- is a day old. A body over 64 KiB, counted in bytes, or a binary body goes to the network at once.
1268
- Firefox does not route `fetch` calls made with `keepalive: true` through the worker, so a script
1269
- that posts that way is not queued there.
1270
-
1271
- ```ts
1272
- import { createSpool, jsonLinesSink } from '@drupflare/spool/server';
1273
- import { memoryStore } from '@drupflare/spool/stores';
1274
- import { routes } from './routes';
1275
-
1276
- export const spool = createSpool({
1277
- routes,
1278
- store: memoryStore(),
1279
- beacons: ['/cdn-cgi/rum'],
1280
- sink: jsonLinesSink((line) => console.log(line))
1281
- });
1282
- ```
1283
-
1284
- `jsonLinesSink` writes one JSON line per event. `analyticsEngineSink` writes to a Workers
1285
- Analytics Engine dataset, splitting a long body across data points so nothing is cut off, and
1286
- `memorySink` keeps events for tests. The worker also queues one
1287
- `spool.view` event per page view with the tier, the reason, the staleness and the route the
1288
- visitor came from (a pattern or `external`, never a URL). `viewsOf` reads
1289
- those events back. Set `views: false` to turn that off. `workSeries` records how long each route's
1290
- handler took, for peak-work reports.
1291
-
1292
- ### Collectors
1293
-
1294
- A `collectors` entry holds requests to another origin, such as an analytics host. The page gets an
1295
- empty answer at once and the request leaves at a network moment, inside `maxAgeMs`. An event past
1296
- the cap is dropped, never sent late, and each drop is counted and written to the sink as
1297
- `/.spool/dropped`. With `delivery: 'replay'` the worker sends the held request to the vendor with no
1298
- cookies, so identity has to be in the payload, and `stamp` writes the event's time into a query
1299
- parameter for a vendor that takes one. With `delivery: 'sink'` the server half forwards it.
1300
-
1301
- GA4's browser collector takes no backdating. The Measurement Protocol accepts events with their
1302
- own timestamps up to 72 hours old and wants them within 48 hours to attribute them, and it needs an
1303
- API secret the page must not hold. `GA4_COLLECTOR` holds the `g/collect` hit for 48 hours and
1304
- `ga4Sink` sends it as a Measurement Protocol request carrying the original `timestamp_micros`.
1305
- `ga4Sink` checks its requests against the rules spool knows (`mpViolations`); set `debug: true` to
1306
- send them to `/debug/mp/collect` with your own measurement id and secret. Spool holds events
1307
- whether or not the visitor consented, so gate the analytics tag itself, and ask your counsel
1308
- whether holding an event in IndexedDB needs consent where you operate. Programmatic ads are never
1309
- held, replayed or answered locally.
1310
-
1311
- ```ts
1312
- import { GA4_COLLECTOR } from '@drupflare/spool/protocol';
1313
- import { combineSinks, createSpool, ga4Sink, jsonLinesSink } from '@drupflare/spool/server';
1314
- import { memoryStore } from '@drupflare/spool/stores';
1315
- import { routes } from './routes';
1316
-
1317
- export const spool = createSpool({
1318
- routes,
1319
- store: memoryStore(),
1320
- collectors: [
1321
- GA4_COLLECTOR,
1322
- {
1323
- vendor: 'matomo',
1324
- match: 'https://stats.example.edu/matomo.php',
1325
- maxAgeMs: 23 * 3_600_000,
1326
- delivery: 'replay',
1327
- stamp: { param: 'cdt', kind: 'datetime-utc' }
1328
- }
1329
- ],
1330
- sink: combineSinks(
1331
- jsonLinesSink((line) => console.log(line)),
1332
- ga4Sink({ measurementId: 'G-XXXXXXXXXX', apiSecret: 'kept on the server' })
1333
- )
1334
- });
1335
- ```
1336
-
1337
- ---
1338
-
1339
- ## ๐Ÿงท Third-Party Files
1340
-
1341
- A view that loads a file from another origin is not zero-network. The page reports, when it is
1342
- hidden, how many cross-origin resources it loaded and from which origins (origins only, never
1343
- paths). `thirdPartyReport` reads those reports from `viewsOf` and ranks the origins by the views
1344
- each one alone blocks, with a class and a remedy from a short table you can replace. The counts
1345
- are an upper bound on network requests, since a cross-origin timing entry says nothing about cache
1346
- hits without `Timing-Allow-Origin`.
1347
-
1348
- The opt-in `thirdParty` cache removes the request for files that name their own bytes. List the
1349
- origins in `allow`. The worker learns a file from the first view and keeps it only when the URL
1350
- carries a hash, a version or a `/v12/` segment, or the origin marks it immutable. A file versioned
1351
- only by a query parameter is checked again at a network moment once it is a day old. It keeps
1352
- nothing else, and nothing it cannot read: an opaque response counts about 7 MB against quota in
1353
- Chromium, so `allowOpaque` is off by default. WebKit does not send font requests from a stylesheet
1354
- through the worker, so fonts stay on the network there.
1355
-
1356
- ```ts
1357
- import { thirdPartyReport } from '@drupflare/spool/report';
1358
- import { createSpool, memorySink, viewsOf } from '@drupflare/spool/server';
1359
- import { memoryStore } from '@drupflare/spool/stores';
1360
- import { routes } from './routes';
1361
-
1362
- const sink = memorySink();
1363
-
1364
- export const spool = createSpool({
1365
- routes,
1366
- store: memoryStore(),
1367
- sink,
1368
- thirdParty: {
1369
- allow: ['https://fonts.gstatic.com', 'https://cdn.example.edu'],
1370
- maxEntries: 32
1371
- }
1372
- });
1373
-
1374
- export const report = () => thirdPartyReport(viewsOf(sink.events));
1375
- ```
1376
-
1377
- ---
1378
-
1379
- ## โšก Subresource Handling
1380
-
1381
- By default the worker answers an immutable same-origin asset from CacheStorage. `assets` changes
1382
- that. `passthrough` leaves assets to the browser's cache and never touches them. `static-routing`
1383
- asks the browser through `InstallEvent.addRoutes` to send only navigations, beacons and routed
1384
- paths to the worker and the rest of the origin to the network, and acts as `passthrough` where the
1385
- API does not exist: Chromium has it, Playwright's WebKit has it on macOS and not on Linux, and
1386
- Firefox does not. Both options cut the service worker's fetch events (Static Routing took them from
1387
- 32 to 3 per view on a desktop Chromium rig); neither changes server-side Worker invocations.
1388
- Neither is measured to save energy. In the same run, new-tab views drew 2.78 J,
1389
- 2.83 J and 2.84 J of RAPL package energy above idle with CacheStorage, navigation-only handling
1390
- and Static Routing (8 rounds each), and the ranges overlap. Under Static Routing a same-origin write no
1391
- longer reaches the worker, so it is not a network moment and its response cannot carry
1392
- invalidations; navigations still are.
1393
-
1394
- ```ts
1395
- import { createSpool } from '@drupflare/spool/server';
1396
- import { memoryStore } from '@drupflare/spool/stores';
1397
- import { routes } from './routes';
1398
-
1399
- export const spool = createSpool({
1400
- routes,
1401
- store: memoryStore(),
1402
- assets: 'static-routing'
1403
- });
1404
- ```
436
+ `spool.publishStatic(sink)` is a different route to the same tier, for a host that runs no spool server. It writes the page runtime, the worker script, the packs and their deltas, one boot script per pack and a `_headers` file to the sink, and the host answers those files with no handler call. It does not write `spool.json`. With no policy known, the stub names none, and a device renders locally after one network view that teaches it the headers. Pass `policy` to `createSpool` (`{}` for a site with no security headers) to name them. The Vite, Vue, React Router and Nuxt plugins call `publishStatic` at the end of a build by default; `static: false` turns that off. See [Static Tier](docs/protocol.md#static-tier) and the [static example](examples/static/README.md).
1405
437
 
1406
438
  ---
1407
439
 
1408
- ## ๐Ÿ” Shadow Mode
1409
-
1410
- In shadow mode the worker decides whether it could serve a page locally, renders it, still fetches
1411
- the page from the server, compares the two with `normalise` and reports the result. It never
1412
- serves the local page. Use it to measure eligibility and mismatches on real traffic before turning
1413
- serving on. A `static` route has no shadow comparison.
1414
-
1415
- ```ts
1416
- import { createSpool } from '@drupflare/spool/server';
1417
- import { memoryStore } from '@drupflare/spool/stores';
1418
- import { routes } from './routes';
1419
-
1420
- export const shadowed = createSpool({ routes, store: memoryStore(), mode: 'shadow' });
1421
- export const canary = createSpool({ routes, store: memoryStore(), shadowSample: 0.05 });
1422
- ```
1423
-
1424
- `mode: 'shadow'` applies to every route that does not name its own `mode`. `shadowSample` keeps a
1425
- share of views in shadow while the rest are served, which gives a standing canary. A mismatch
1426
- names its route, so a route whose template and application disagree shows up before a visitor
1427
- sees it. The worker records the shadow comparison for navigations only. Subresources and other
1428
- requests keep the browser's own path.
440
+ ## Advanced Usage
441
+
442
+ [ADVANCED_USAGE.md](ADVANCED_USAGE.md) covers each declaration and setting past the Quick Start,
443
+ in the order an operator meets them. Each section gives the use, the smallest declaration and the
444
+ limit.
445
+
446
+ - [Strategies](ADVANCED_USAGE.md#strategies): the five ways a route is answered locally, and the
447
+ security headers a local page keeps.
448
+ - [Records and Projections](ADVANCED_USAGE.md#records-and-projections): shared records, stored once
449
+ per id.
450
+ - [Templates](ADVANCED_USAGE.md#templates): the template syntax and the trusted-field rule.
451
+ - [Static Files](ADVANCED_USAGE.md#static-files): files kept as the server sent them.
452
+ - [Projection and Data Routes](ADVANCED_USAGE.md#projection-and-data-routes): JSON answers, bundled
453
+ renderers and the provisioning rule.
454
+ - [Query Routes](ADVANCED_USAGE.md#query-routes): filter, sort and page a held collection with no
455
+ request.
456
+ - [Freshness Contracts](ADVANCED_USAGE.md#freshness-contracts): hard and soft bounds, and the script
457
+ lease.
458
+ - [Datasets](ADVANCED_USAGE.md#datasets): row patches with a proof, plus
459
+ [Dataset Routes](ADVANCED_USAGE.md#dataset-routes) and
460
+ [Principal Datasets](ADVANCED_USAGE.md#principal-datasets).
461
+ - [Authenticated Data](ADVANCED_USAGE.md#authenticated-data): per-principal partitions, sealed
462
+ storage and logout wipes.
463
+ - [Declared Writes](ADVANCED_USAGE.md#declared-writes): queued last-value edits with a version check.
464
+ - [Page Runtime Settings](ADVANCED_USAGE.md#page-runtime-settings): the bootstrap gate, `workerMode`,
465
+ swap elements and the `window.spool` surface.
466
+ - [Catalogue](ADVANCED_USAGE.md#catalogue): data for the pages a visitor may open next.
467
+ - [Analytics Batching](ADVANCED_USAGE.md#analytics-batching) and
468
+ [Collectors](ADVANCED_USAGE.md#collectors): beacons, sinks and third-party collection.
469
+ - [Third-Party Files](ADVANCED_USAGE.md#third-party-files) and
470
+ [Subresource Handling](ADVANCED_USAGE.md#subresource-handling).
471
+ - [Shadow Mode](ADVANCED_USAGE.md#shadow-mode): compare local renders with the network before
472
+ serving.
473
+ - [Proxy Mode](ADVANCED_USAGE.md#proxy-mode): front an origin that cannot load the library.
474
+ - [Trimmed Stylesheets](ADVANCED_USAGE.md#trimmed-stylesheets) and
475
+ [Images and Video](ADVANCED_USAGE.md#images-and-video): proxy options for CSS, images and video.
476
+ - [Capacity Reports](ADVANCED_USAGE.md#capacity-reports): the capacity plan, the institution report
477
+ and request arithmetic.
1429
478
 
1430
479
  ---
1431
480
 
1432
- ## ๐Ÿ”Œ Proxy Mode
1433
-
1434
- `spoolProxy` fronts an origin that cannot load the library, such as PHP, Drupal with no module,
1435
- or anything else that speaks HTTP. It returns a Fetch handler. The proxy answers `/.spool/*` and
1436
- `/spool-sw.js`. Every other request goes to the origin with its method, headers and body, and a
1437
- redirect is passed on, not followed. A page the proxy may keep is decorated like a `createSpool`
1438
- response, so the worker stores it and answers later views with no request. A cold visit costs one
1439
- origin fetch: the proxy names a hash of the page in `Spool-Artifact`, and the worker keeps the
1440
- page it already received as the artifact when its own hash matches. On any mismatch it asks
1441
- `/.spool/artifact` as before. In the first-payload reading (L1, a synthetic 11,145-byte page) a
1442
- cold visit made 1 request and 1 origin fetch where it made 2 and 2 before.
1443
-
1444
- `spool` takes the options `createSpool` takes, minus `routes`. The proxy adds one public
1445
- `assembly` route that matches every path (`/**`) and uses the origin's finished HTML as the
1446
- artifact. `routes` lists overrides, such as a shorter freshness bound for one pattern. They
1447
- match first and take the same fields as `defineRoute`, without `load`.
481
+ ## Security
1448
482
 
1449
- On Node the handler goes through `toNodeListener`. `origin` is the address of the real server
1450
- and has no path.
1451
-
1452
- ```ts
1453
- import { createServer } from 'node:http';
1454
- import { toNodeListener } from '@drupflare/spool/node';
1455
- import { spoolProxy } from '@drupflare/spool/proxy';
1456
- import { memoryStore } from '@drupflare/spool/stores';
1457
-
1458
- const proxy = spoolProxy({
1459
- origin: process.env['ORIGIN'] as string,
1460
- spool: { store: memoryStore() },
1461
- purgeSecret: process.env['PURGE_SECRET']
1462
- });
1463
-
1464
- export const server = createServer(toNodeListener(proxy));
1465
- ```
483
+ Authorisation stays on the server. The device holds only what the server already sent that
484
+ principal, and the worker has no access rule. Principal data sits in a per-partition IndexedDB
485
+ database sealed with AES-GCM, and deleting the key at logout is crypto-shredding of data at rest.
486
+ That does not protect against script running on your origin. A revoked permission can still show
487
+ for up to a route's `maxStale`, because nothing pushes a revocation to a device. Tag names are
488
+ readable through `/.spool/sync`, so use opaque ids for anything sensitive. Principal datasets and
489
+ declared writes ship off and have had no external review.
1466
490
 
1467
- On Workers the proxy is built once per isolate and keeps the version vector in a Durable Object,
1468
- bound as in [Cloudflare Workers](#cloudflare-workers). Set the purge secret with
1469
- `wrangler secret put PURGE_SECRET`.
1470
-
1471
- ```ts
1472
- import { spoolProxy, type SpoolProxy } from '@drupflare/spool/proxy';
1473
- import { durableObjectStore } from '@drupflare/spool/stores/durable-object';
1474
-
1475
- export { SpoolVersions } from '@drupflare/spool/stores/durable-object';
1476
-
1477
- let proxy: SpoolProxy | undefined;
1478
-
1479
- export default {
1480
- fetch(request: Request, env: Cloudflare.Env) {
1481
- proxy ??= spoolProxy({
1482
- origin: 'https://origin.example.edu',
1483
- spool: { store: durableObjectStore(env.SPOOL_VERSIONS) },
1484
- purgeSecret: env.PURGE_SECRET
1485
- });
1486
- return proxy(request);
1487
- }
1488
- };
1489
- ```
1490
-
1491
- ### Tags
1492
-
1493
- The tags the origin sends become the page's dependencies. The proxy reads `Cache-Tag` (comma
1494
- separated), `Surrogate-Key` and `X-Drupal-Cache-Tags` (space separated) and `xkey` (spaces or
1495
- commas), then removes them from the response unless `exposeTags` is true. A page with no tags
1496
- depends on one tag derived from its path.
1497
-
1498
- ### Eligibility
1499
-
1500
- The proxy keeps a response only when all of these hold:
1501
-
1502
- - the request is a GET or HEAD, has no query string and no `Authorization` header, and carries
1503
- no cookie outside `publicCookies`, which is empty by default
1504
- - the response is a 200 `text/html`, or the answer of a `data` route
1505
- - the page is UTF-8, or declares no charset in its `Content-Type` or in a `<meta>` within its
1506
- first 1024 bytes
1507
- - the response sets no cookie outside `publicCookies`, has no `private` or `no-store` in
1508
- `Cache-Control`, a `Vary` that names only `Accept-Encoding` and `Cookie`, no more than
1509
- `maxTags` tags and no more than `maxBytes` bytes
1510
-
1511
- Everything else is forwarded untouched. A page the proxy declines tells the worker to drop its
1512
- copy of that path, and a response that sets a cookie outside `publicCookies` tells the worker to
1513
- clear everything it holds, so a login leaves no anonymous copy to show. Put cookies that carry no
1514
- identity on the list, such as a bot-check or load-balancer cookie set on every response; otherwise
1515
- proxy mode never keeps a page on that site. Writes always go to the origin and are never
1516
- answered locally.
1517
-
1518
- Principal routes are off unless `spool.identity` is set. With it, a route declared
1519
- `scope: 'principal'` follows the partitions in [Authenticated Data](#-authenticated-data). The
1520
- cookie, `Authorization` and `private` rules do not apply to it. `Set-Cookie` and `no-store`
1521
- still do.
1522
-
1523
- ### Invalidation
1524
-
1525
- Two signals reach the proxy. When a page it forwards differs from the last body it saw, it bumps
1526
- that page's tags. The proxy hashes the body after `normalise`, and `ignore` takes patterns to
1527
- blank, such as a per-render form token. `POST /.spool/purge` bumps the tags it is given, so a CMS
1528
- hook or Drupal's purge module can call it. It needs `Authorization: Bearer <purgeSecret>` and a
1529
- JSON body, and it refuses every call when no secret is set.
1530
-
1531
- ```sh
1532
- curl -X POST https://example.edu/.spool/purge \
1533
- -H "Authorization: Bearer $PURGE_SECRET" \
1534
- -H "Content-Type: application/json" \
1535
- -d '{"tags": ["node:1", "config:system.menu.main"]}'
1536
- ```
1537
-
1538
- ### Limits
1539
-
1540
- The proxy sees finished HTML, not the database behind it. An artifact is a whole page, the
1541
- replica holds no records, and two pages never share a fragment. A page stays on a device until its
1542
- route's `maxStale` passes and the next sync reports a change, so freshness is bounded by
1543
- `maxStale` plus the time a purge takes to arrive.
1544
-
1545
- A tag shared by many pages invalidates all of them when one changes. The remembered bodies live
1546
- in the instance: a restart forgets them, and each instance of a scaled-out proxy keeps its own.
1547
- A whole page is a large artifact, and the provisioning rule declines one that costs more than the
1548
- priors say it saves, so set `provision` on the spool for large pages. A `Content-Security-Policy`
1549
- that allows only nonce or hash scripts blocks the injected client tag. A URL with a query string
1550
- always goes to the origin, because the proxy holds finished HTML and has no collection to filter;
1551
- [Query Routes](#query-routes) apply to `createSpool` only. [Trimmed Stylesheets](#-trimmed-stylesheets) covers the `trim` option
1552
- and [Images and Video](#-images-and-video) the `images` and `video` options.
491
+ [docs/security.md](docs/security.md) is the threat model. It names the file and the test behind
492
+ each claim and lists the open limitations in one table. Report a vulnerability as
493
+ [SECURITY.md](https://github.com/drupflare/spool/blob/master/SECURITY.md) describes.
1553
494
 
1554
495
  ---
1555
496
 
1556
- ## ๐Ÿงน Trimmed Stylesheets
1557
-
1558
- A new page of a Drupal site usually loads a stylesheet aggregate the visitor has not seen, and
1559
- most of its rules match nothing on the page. `@drupflare/spool/trim` builds a manifest that removes
1560
- the rules no page of the site can match, and `spoolProxy` serves the trimmed text in place of the
1561
- origin's file. Each file is trimmed against every page and script of the site, never against one
1562
- page, so it keeps its URL and its one cached copy. Rules are only removed, so the cascade of what
1563
- stays is unchanged.
497
+ ## Energy Reporting
1564
498
 
1565
- ```ts
1566
- import { spoolProxy } from '@drupflare/spool/proxy';
1567
- import { memoryStore } from '@drupflare/spool/stores';
1568
- import { buildTrimManifest } from '@drupflare/spool/trim';
499
+ Energy is one objective among those spool reports, and the administrator decides how much weight
500
+ it gets. The energy of one deployment is small against an institution's total, and small against
501
+ the idle and hosting energy drupflare removes per site. The figures below are research figures.
1569
502
 
1570
- export const manifest = await buildTrimManifest({
1571
- pages: ['<body class="hero"><a class="button">Apply</a></body>'],
1572
- scripts: [{ url: '/js/site.js', text: 'menu.classList.add("open")' }],
1573
- styles: [{ url: '/css/site.css', text: '.hero{a:b}.button{c:d}.open{e:f}.print-only{g:h}' }]
1574
- });
503
+ [docs/impact.md](docs/impact.md) is generated from the readings in `measurements/` by
504
+ `bun run impact`, and `bun run impact:check` fails when the page differs from them. Energy is
505
+ reported in kWh per million views, and nothing sets a target intensity. Every figure carries a
506
+ layer: L0 published, L1 measured here, L2 derived, L3 scenario. A per-view figure that spreads a
507
+ published intensity over bytes is an allocation of that figure, and the page prints the metered
508
+ reading beside it. The savings from trimming, images and dictionaries apply to network views and
509
+ never add to the local-view figure.
1575
510
 
1576
- export const proxy = spoolProxy({
1577
- origin: 'https://origin.example.edu',
1578
- spool: { store: memoryStore() },
1579
- trim: manifest
1580
- });
1581
- ```
511
+ The request counts in [Economics](docs/economics.md#headline-numbers) come from library readings on Chromium and Firefox. WebKit is absent from every library reading. The library's headline covers Chromium and Firefox only. WebKit, which is every browser on iOS, defaults to no worker because of an upstream script check on every navigation; its figures are model rows or single script-check readings, never library readings. "Chromium and Firefox" is well under all of mobile.
1582
512
 
1583
- `pages` is HTML, `scripts` and `styles` are the same-origin files the pages load, and `url` is the
1584
- address the page links. Crawl with a browser and pass the page's DOM after load and after a
1585
- scroll as well as the origin's HTML: a class that a script builds from parts, such as
1586
- `'fs-carousel' + '-enabled'`, is only in the rendered page. Pass the scripts of other origins that
1587
- the pages load as `foreignScripts`, since a widget that builds markup adds classes too.
1588
-
1589
- Two checks protect a visitor. The proxy serves a trimmed file only when the origin's bytes still
1590
- hash to the manifest's `source`, and otherwise sends the origin's file and counts a mismatch. It
1591
- also reads every HTML page before it leaves: a page with a token the trim removed a rule for, or a
1592
- same-origin script the manifest was not built with, links the full file through `spool-trim=0`.
1593
- `proxy.trim` counts the files served, the mismatches and the guard firings, and the sink gets
1594
- each mismatch and firing as an event. [docs/security.md](docs/security.md#trimmed-stylesheets) names
1595
- what the guard cannot see: a class a vendor script adds after the crawl, markup injected after load
1596
- from a page the crawl never requested, and a class name built at runtime that no crawl rendered.
1597
-
1598
- The proxy trims CSS and leaves JavaScript alone. Coverage at load cannot see what a click or a
1599
- scroll will need: the used share of same-origin JavaScript rose from 36% at load to 40% after one
1600
- scroll on the pages measured (L1), so no load-time figure says which of the rest is dead. The same
1601
- saving from a build step that runs PurgeCSS with a safelist needs no proxy. The proxy gets it
1602
- without a change to the origin. The saving applies to a network view of a page the visitor has not
1603
- seen, never to a repeat or local view, and it is not part of the per-view headline;
1604
- [docs/impact.md](docs/impact.md#asset-trimming) has the bytes, joules and prices.
513
+ [An Institution's Figures](docs/impact.md#an-institutions-figures) leads with economics, storage
514
+ and trust and treats energy as a research figure. [What This Does Not
515
+ Claim](docs/impact.md#what-this-does-not-claim) lists what the page leaves unsaid.
1605
516
 
1606
517
  ---
1607
518
 
1608
- ## ๐ŸŽฌ Images and Video
519
+ ## Prior Art
1609
520
 
1610
- A browser that has fetched an image cannot make it smaller, so conversion happens where the file is
1611
- served. `@drupflare/spool/media` gives `spoolProxy` three options, each off until set. `images`
1612
- converts a JPEG or PNG response to WebP for a browser whose `Accept` lists it. `video` rewrites
1613
- the `<video>` tags of an HTML response, such as adding `preload="none"`. `embeds` swaps a YouTube
1614
- or Vimeo iframe for a link that loads the player on click.
521
+ spool is a server-authoritative replica whose freshness policy lets a provisioned view render with
522
+ no per-view validation request, with shadow mode and a fall back to the server for anything it
523
+ cannot answer. HTTP caching, service-worker precaching and earlier replication systems cover parts
524
+ of it. What is shared and what differs:
1615
525
 
1616
- ```ts
1617
- import { type ImageEncoder } from '@drupflare/spool/media';
1618
- import { spoolProxy } from '@drupflare/spool/proxy';
1619
- import { memoryStore } from '@drupflare/spool/stores';
526
+ | System | What it does | Shared with spool | Difference |
527
+ | ------------------ | ------------------------------------------------------------- | ----------------------------------------------- | ----------------------------------------------------------------------------------------- |
528
+ | HTTP caching | reuses a response within `max-age` with no validation | a repeat view with no request | one response at a time; no per-dependency invalidation and no held collections |
529
+ | Workbox precaching | answers a precached navigation from the cache with no request | a worker answering a navigation with no request | the list and its revisions are fixed at build time and refresh when a new worker installs |
530
+ | Fawkes (NSDI 2020) | caches static page templates and sends a patch of DOM updates | a cached template, the server authoritative | the patch is requested on every load, so a cached template never answers a view alone |
531
+ | Speculation Rules | prefetches or prerenders pages the visitor may open next | acting on a predicted next view | the origin answers every speculated request, used or not |
1620
532
 
1621
- export function mediaProxy(encoder: ImageEncoder) {
1622
- return spoolProxy({
1623
- origin: 'https://origin.example.edu',
1624
- spool: { store: memoryStore() },
1625
- images: { encoder, width: { fallback: 1600 } },
1626
- video: { preload: 'none', autoplay: 'defer', when: 'slow' }
1627
- });
1628
- }
1629
- ```
1630
-
1631
- An `ImageEncoder` takes the origin's bytes and `{ format: 'webp', quality, width }` and returns
1632
- `{ bytes }`. It never upscales, so `width` is a ceiling, and it throws on input it cannot decode.
1633
- spool has no runtime dependencies and ships an adapter for [tinyimg](https://tinyimg.gmitch215.dev),
1634
- which decodes and encodes inside WebAssembly:
1635
-
1636
- ```js
1637
- import { readFile } from 'node:fs/promises';
1638
- import { TinyImgModule, transform } from '@gmitch215/tinyimg';
1639
- import { tinyimgEncoder } from '@drupflare/spool/media';
1640
-
1641
- const wasm = new URL(import.meta.resolve('@gmitch215/tinyimg/tinyimg.wasm'));
1642
- const module = await TinyImgModule.loadBytes(await readFile(wasm));
1643
-
1644
- export const encoder = tinyimgEncoder({ module, transform });
1645
- ```
1646
-
1647
- Use tinyimg 1.1.1 or later. Version 1.1.0 and earlier drop the EXIF orientation tag without
1648
- rotating the pixels, so a phone photo comes out sideways. `tinyimgEncoder` throws when the module
1649
- reports an older version.
1650
-
1651
- Quality is 80, 65 for `Save-Data: on` and 70 for an `ECT` of `slow-2g`, `2g` or `3g`. The width
1652
- comes from `Sec-CH-Width`, then `Sec-CH-Viewport-Width` times `Sec-CH-DPR`, then
1653
- `width.fallback`. The proxy converts a `GET` that matches `paths` (JPEG and PNG names by default),
1654
- has no `Range` or `Authorization`, and gets a `200` JPEG or PNG without a cookie between
1655
- `minBytes` (4096) and `maxBytes` (15,000,000). Anything else, and any file the encoder cannot
1656
- shrink, goes out as the origin sent it. The proxy fetches the origin for every request, so a
1657
- visitor only gets bytes the origin sent that visitor.
1658
-
1659
- A converted response carries `Vary` for `Accept`, `Save-Data`, `Sec-CH-Width`, `Sec-CH-DPR`,
1660
- `Sec-CH-Viewport-Width` and `ECT`, a weak `ETag` and `X-Spool-Image`. HTML responses gain
1661
- `Accept-CH` for the three size hints and `ECT` unless `hints` is false, and the browser then
1662
- sends them with later image requests to the same origin.
1663
-
1664
- `maxWidth` (2560) is the width for a request that carries no hint, and `false` removes it. `png`
1665
- is `'lossy'` by default. `'lossless'` encodes every PNG without loss, and `'auto'` encodes a PNG
1666
- under 200 KB both ways and serves the smaller file, which suits screenshots and flat graphics.
1667
- Requests for the same file, quality and width share one encoder call while it runs. The proxy
1668
- waits `timeoutMs` (10,000) for the encoder and then serves the origin's file, and the late result
1669
- is cached for the next request. `timeoutMs: false` waits without limit.
1670
-
1671
- Drupal's video formatter has no `preload` setting, and `preload="none"` does not stop `autoplay`.
1672
- `autoplay: 'defer'` removes the attribute, adds `controls` and marks the tag
1673
- `data-spool-autoplay`. With `when: 'always'` the page the spool keeps carries the rewrite. With
1674
- `'save-data'` or `'slow'` the rewritten page is private to that visitor: it gets `private` in
1675
- `Cache-Control` and `Save-Data, ECT` in `Vary`, and the spool does not keep it. A device's stored
1676
- copy of that page keeps the origin's tags.
1677
-
1678
- `proxy.images` counts `converted`, `bytesIn` and `bytesOut` (converted responses only),
1679
- `cacheHits`, `coalesced`, `lazied`, `responsive`, and `skipped` by reason. `proxy.video` counts
1680
- `pages` and `rewritten`. Read the saving as `bytesIn - bytesOut` over real traffic on the site. A
1681
- request for any other path counts under `skipped.path`, so that key follows total traffic.
1682
- [docs/security.md](docs/security.md#images-and-video) covers the decoder.
1683
-
1684
- ### Page Rewrites
1685
-
1686
- Three options change the HTML the origin sends. Each edits start tags only and leaves every other
1687
- byte as it was, and a rewritten page gets a weak `ETag`.
1688
-
1689
- ```ts
1690
- import { type ImageEncoder } from '@drupflare/spool/media';
1691
- import { spoolProxy } from '@drupflare/spool/proxy';
1692
- import { memoryStore } from '@drupflare/spool/stores';
1693
-
1694
- export function rewriteProxy(encoder: ImageEncoder) {
1695
- return spoolProxy({
1696
- origin: 'https://origin.example.edu',
1697
- spool: { store: memoryStore() },
1698
- images: {
1699
- encoder,
1700
- png: 'auto',
1701
- maxWidth: 1920,
1702
- timeoutMs: 5000,
1703
- lazy: { skip: 3 },
1704
- responsive: { widths: [480, 960, 1440], sizes: '(min-width: 60em) 50vw, 100vw' }
1705
- },
1706
- embeds: { providers: ['youtube'], thumbnail: false }
1707
- });
1708
- }
1709
- ```
1710
-
1711
- `lazy` adds `loading="lazy"` and `decoding="async"` to each `<img>` after the first `skip` (2 by
1712
- default), and `iframes: true` does the same for `<iframe>`. It leaves a tag that has `loading`,
1713
- `fetchpriority` or a lazy-loading library's `data-src`, a tag inside a `<header>` or `<noscript>`,
1714
- and an image that a `<link rel="preload" as="image">` names. A browser that defers the largest
1715
- image of a page paints it later, so `skip` is the only guard and the option is off until set. In a
1716
- crawl of 70 pages per profile on 16 sites, the default `skip` of 2 made the largest image lazy on 2
1717
- of 23 desktop pages and 2 of 20 phone pages that had an image as their largest element, and no
1718
- `skip` from 2 to 4 reached zero on both profiles.
1719
-
1720
- `responsive` adds a `srcset` of `?spool-w=<width>` variants to a same-origin `<img>` that has none,
1721
- one per entry of `widths` (320, 480, 640, 960, 1280 and 1920 by default), and a `sizes` of `100vw`
1722
- unless the tag has one. The proxy removes `spool-w` before it asks the origin, answers only widths
1723
- from the list and never sends a file larger than the origin's. Firefox and Safari send no
1724
- `Sec-CH-Width`, and this is how they get a resized file. Name the layout width in `sizes`, since
1725
- `100vw` makes the browser pick the widest candidate it can use. A tag with `srcset` or `data-src`,
1726
- and any tag inside `<picture>`, stays as it is.
1727
-
1728
- `embeds` replaces a server-rendered YouTube or Vimeo `<iframe>` with one whose `srcdoc` holds a
1729
- link to the player. YouTube shows its thumbnail (`thumbnail: false` shows a plain frame) and the
1730
- player loads on click. A page that sends `Content-Security-Policy` is left alone unless
1731
- `ignoreCsp` is true, and so is an embed with `enablejsapi=1`. An embed that plays on its own by
1732
- design stays too: a URL with `background=1`, or with `autoplay=1` beside `mute`, `muted` or `loop`,
1733
- and an iframe whose `class` or `id` names a hero, ambient, background or banner. In a crawl of the
1734
- server-rendered pages of 52 sites (L1, `measurements/embed-markup`), 2 of 4 Vimeo iframes were
1735
- background loops. `proxy.embeds` counts `pages`, `rewritten` and `skipped` by reason. [docs/security.md](docs/security.md#embed-facades) covers the rules.
1736
-
1737
- ### Dictionary Transport
1738
-
1739
- A visitor who holds an earlier stylesheet or script of a directory can fetch the next one as the
1740
- difference from it. `dictionary` implements this for Chromium 130 and later
1741
- ([RFC 9842](https://www.rfc-editor.org/rfc/rfc9842)). The proxy marks public `.css` and `.js`
1742
- responses with `Use-As-Dictionary` and answers a request that names one of them with
1743
- `Content-Encoding: dcz`. spool has no compression library, so the caller supplies the `encoder`,
1744
- and Node's `zlib` does zstd:
1745
-
1746
- ```ts
1747
- import { promisify } from 'node:util';
1748
- import { zstdCompress } from 'node:zlib';
1749
- import { spoolProxy } from '@drupflare/spool/proxy';
1750
- import { memoryStore } from '@drupflare/spool/stores';
1751
-
1752
- const compress = promisify(zstdCompress);
1753
-
1754
- export const proxy = spoolProxy({
1755
- origin: 'https://origin.example.edu',
1756
- spool: { store: memoryStore() },
1757
- dictionary: {
1758
- encoder: async (dictionary, input) => new Uint8Array(await compress(input, { dictionary }))
1759
- }
1760
- });
1761
- ```
1762
-
1763
- The `encoder` gets the dictionary, the new file and the coding (`'dcz'`, or `'dcb'` when `formats`
1764
- lists it) and returns the compressed stream. spool adds the coding's header. An encoder that
1765
- throws, or whose output is no smaller, leaves the plain file. Only responses the proxy would keep
1766
- as public ones, with no cookie, ever become dictionaries. `proxy.dictionary` counts `registered`,
1767
- `encoded`, `bytesIn`, `bytesOut` and `skipped`.
1768
- [docs/security.md](docs/security.md#dictionary-transport) covers the exposure.
1769
-
1770
- `base` names the files that register as dictionaries, as a function of the pathname, a regular
1771
- expression or a list of them. Without it the browser holds the latest file of each directory it
1772
- fetched, and the sizing in [docs/impact.md](docs/impact.md#savings-today-and-at-maximum) shows that
1773
- choice recovers little. With it the browser holds only the files you name, one per directory, and
1774
- every other file of the directory is encoded against the base the request names.
1775
-
1776
- `chooseBases` picks the bases from a crawl. It takes the site's stylesheets and scripts with their
1777
- compressed sizes and a `delta` function that returns the size of one file compressed against
1778
- another. For each directory and file kind it returns the file whose deltas against the others, each
1779
- capped at the other file's plain size, add up smallest. That `held` policy scores a base for a
1780
- visitor who already holds it. Pass `{ policy: 'first-visit' }` to add the base's own plain size
1781
- to the sum, for visitors who load the whole group before they hold anything. Neither policy
1782
- weighs how often a file is requested or how long browsers keep the base, so neither is a proven
1783
- minimum of total transfer. A directory with one file has no base, and a tie goes to the URL that
1784
- sorts first. `baseMatcher` turns the choices into a `base` function.
1785
-
1786
- ```ts
1787
- import { promisify } from 'node:util';
1788
- import { brotliCompressSync, zstdCompress } from 'node:zlib';
1789
- import { baseMatcher, chooseBases, spoolProxy } from '@drupflare/spool/proxy';
1790
- import { memoryStore } from '@drupflare/spool/stores';
1791
-
1792
- const compress = promisify(zstdCompress);
1793
-
1794
- const crawl = [
1795
- { url: 'https://origin.example.edu/css/a.css', kind: 'style', text: '.a{color:red}.b{margin:0}' },
1796
- {
1797
- url: 'https://origin.example.edu/css/b.css',
1798
- kind: 'style',
1799
- text: '.a{color:red}.c{margin:1px}'
1800
- },
1801
- {
1802
- url: 'https://origin.example.edu/css/c.css',
1803
- kind: 'style',
1804
- text: '.a{color:red}.d{margin:2px}'
1805
- }
1806
- ] as const;
1807
-
1808
- const bytes = (index: number) => Buffer.from(crawl[index]?.text ?? '');
1809
-
1810
- const choices = await chooseBases(
1811
- crawl.map(({ url, kind, text }) => ({ url, kind, plain: brotliCompressSync(text).length })),
1812
- async (file, dictionary) =>
1813
- (await compress(bytes(file), { dictionary: bytes(dictionary) })).length
1814
- );
1815
-
1816
- export const proxy = spoolProxy({
1817
- origin: 'https://origin.example.edu',
1818
- spool: { store: memoryStore() },
1819
- dictionary: {
1820
- encoder: async (dictionary, input) => new Uint8Array(await compress(input, { dictionary })),
1821
- base: baseMatcher(choices)
1822
- }
1823
- });
1824
- ```
1825
-
1826
- On the 10 Drupal sites of the reading, a base chosen this way and already held by the browser saves
1827
- 30% of the stylesheet and script bytes at the median site (L1, [docs/impact.md](docs/impact.md#savings-today-and-at-maximum)).
1828
- `scripts/measure/dictionary-size.ts` records the same choice for a site's crawl.
1829
-
1830
- ### Defaults
1831
-
1832
- Every option is absent by default. The table shows what a feature does once its parent option is
1833
- set and how to change it.
1834
-
1835
- | Feature | Option | Default once the parent is set | Leave or enter |
1836
- | ------------------------------------------ | ------------------- | ------------------------------------------- | ----------------------------------------------------- |
1837
- | WebP conversion, `304` and `X-Spool-Image` | `images` | on, needs an `encoder` | leave `images` out |
1838
- | `Accept-CH` hints and `Vary` | `images.hints` | on | `hints: false` |
1839
- | Quality by `Save-Data` and `ECT` | `images.quality` | 80, 65 and 70 | give all three one number |
1840
- | Width ceiling for a request with no width | `images.maxWidth` | 2560 | `maxWidth: false`, or another number |
1841
- | Shared encodes and the encoder time limit | `images.timeoutMs` | 10,000 ms, identical encodes share one call | `timeoutMs: false` removes the limit |
1842
- | PNG mode | `images.png` | `'lossy'` | `'auto'` or `'lossless'` |
1843
- | `loading="lazy"` on images and iframes | `images.lazy` | off | `lazy: {}` or `{ skip, iframes }` |
1844
- | `srcset` variants | `images.responsive` | off | `responsive: {}` or `{ widths, sizes }` |
1845
- | `preload="none"` on video | `video.preload` | `'none'` | `preload: false` |
1846
- | Video autoplay deferral | `video.autoplay` | off | `autoplay: 'defer'` |
1847
- | YouTube and Vimeo facades | `embeds` | off | `embeds: {}` or `{ providers, thumbnail, ignoreCsp }` |
1848
- | Dictionary transport | `dictionary` | off | `dictionary: { encoder }` |
1849
- | Which files become dictionaries | `dictionary.base` | every `.css` and `.js` response | `base: /\/base\.js$/`, a list or a function |
1850
-
1851
- ---
1852
-
1853
- ## ๐Ÿ“ˆ Capacity Reports
1854
-
1855
- `@drupflare/spool/report` turns peak work and an institution's own limits into three separate
1856
- results: potential, financial and physical. None is added to another, and any can be empty.
1857
-
1858
- `capacityPlan` takes intervals of demand per resource with the part spool avoided, and what one
1859
- node serves per second within the latency target. It returns the nodes needed before and after,
1860
- as whole numbers, and how many could retire. `perNode.busyMs` is the milliseconds of handler work
1861
- a node does per second, such as 1000 times its cores times a target utilisation. `spares`, `floor`,
1862
- `regions` and `minPerRegion` raise the answer for redundancy, and a resource with no `avoided`, such
1863
- as memory, does not shrink. `intervalsFromWork` builds the intervals from the `WorkSample`s the
1864
- work series records.
1865
-
1866
- `institutionReport` adds the plan, an `observed` before and after of the same workload from
1867
- production telemetry, an invoice total before and after, and a ledger of retired equipment. The
1868
- plan says what could change, `observed` what changed, the invoices what was paid, and the ledger
1869
- what was removed. The physical result reads only the ledger, and an entry needs the evidence for
1870
- its removal. With no ledger the statement reads `No physical capacity saving demonstrated.`
1871
-
1872
- ```ts
1873
- import { capacityPlan, institutionReport } from '@drupflare/spool/report';
1874
-
1875
- const plan = capacityPlan(
1876
- [
1877
- {
1878
- start: Date.UTC(2026, 8, 1, 14),
1879
- seconds: 300,
1880
- demand: { busyMs: 9000 },
1881
- avoided: { busyMs: 6300 }
1882
- },
1883
- {
1884
- start: Date.UTC(2026, 8, 1, 14, 5),
1885
- seconds: 300,
1886
- demand: { busyMs: 7000 },
1887
- avoided: { busyMs: 4900 }
1888
- }
1889
- ],
1890
- { perNode: { busyMs: 700 }, spares: 1, floor: 2 }
1891
- );
1892
-
1893
- export const report = institutionReport({ plan, planLayer: 'L3' });
1894
- ```
1895
-
1896
- The example needs 14 nodes without spool and 5 with it, a potential of 9, at layer L3 because the
1897
- numbers are made up. `channelRows` lists the metered, attributional, capacity and avoided-demand
1898
- channels as separate rows, and nothing in the module sums them. [docs/impact.md](docs/impact.md#channels)
1899
- says what each one counts.
533
+ Sync Kit (WWW 2010) also keeps templates and data in the browser to reduce server load. The
534
+ [technical report](TECHNICAL_REPORT.md#executive-summary) states the narrow claim.
1900
535
 
1901
536
  ---
1902
537
 
1903
- ## ๐Ÿšซ Out of Scope
538
+ ## Out of Scope
1904
539
 
1905
540
  - spool never authorises on the client. The device holds only what the server already sent that
1906
541
  principal.
1907
542
  - It never shows a hard route past its bound without validating it first.
1908
- - It never writes. Your application owns every write, and `invalidate` only announces one.
543
+ - It never writes on its own. Your application owns every write, and `invalidate` only announces
544
+ one. The one exception is a declared write class, which is off until you enable it.
1909
545
  - It never infers dependencies from opaque HTML. A route names its tags, or the records it
1910
546
  reads, and in proxy mode the origin names them in its cache tag headers.
1911
547
  - It never replaces a published release file on the CDN.
1912
548
 
1913
549
  ---
1914
550
 
1915
- ## ๐ŸŒ Browser Support
551
+ ## Browser Support
1916
552
 
1917
- | Engine | Controlled navigation |
1918
- | -------- | ------------------------------------------------------------------------------------------------------------------ |
1919
- | Chromium | no request, new tabs included |
1920
- | Firefox | no request, new tabs included |
1921
- | WebKit | one request to re-check `/spool-sw.js`, and one refetch of each cacheable subresource on the first controlled load |
553
+ | Engine | Controlled navigation |
554
+ | -------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
555
+ | Chromium | no request, new tabs included |
556
+ | Firefox | no request, new tabs included |
557
+ | WebKit | with `workerMode: 'always'`, on macOS, one request to re-check `/spool-sw.js` on each document navigation (none on Linux while the worker runs); with the default `'auto'`, no worker on a page with a ring, so no script request; one refetch of each cacheable subresource on the first controlled load |
1922
558
 
1923
559
  The worker registers with `updateViaCache: 'all'` and the script is served at
1924
- `max-age=86400`, which is what lets Chromium and Firefox skip the update check. A deploy changes
1925
- the script bytes and reaches a browser within that day. A browser without service workers gets the
560
+ `max-age=86400`, which is what lets Chromium and Firefox skip the update check. WebKit on macOS
561
+ re-checks the script regardless, so with `workerMode: 'always'` a document navigation there costs one
562
+ request whatever the site does ([docs/impact.md](docs/impact.md#demand-per-client)). With the default
563
+ `workerMode: 'auto'`, WebKit registers no worker on a page that carries a ring, so the script is not
564
+ requested; the page runtime still answers held reads and in-page navigation
565
+ ([WebKit Default](docs/protocol.md#webkit-default)). A deploy changes the script bytes and reaches a
566
+ browser within that day. A browser without service workers gets the
1926
567
  page it always got.
1927
568
 
1928
569
  A page that loads a resource from another origin is not a zero-request view. spool counts those
@@ -1930,26 +571,34 @@ loads per view and reports the view as such.
1930
571
 
1931
572
  ---
1932
573
 
1933
- ## ๐Ÿงญ Subpath Exports
1934
-
1935
- | Import | What is in it |
1936
- | ---------------------------------------- | ----------------------------------------------------------------------------- |
1937
- | `@drupflare/spool` | the protocol and report modules |
1938
- | `@drupflare/spool/protocol` | envelopes, artifacts, patterns, templates, the manifest, view events |
1939
- | `@drupflare/spool/server` | `createSpool`, `defineRoute`, `datasetProof`, sinks, identity and work series |
1940
- | `@drupflare/spool/node` | `toNodeListener`, `toRequest`, `writeResponse` |
1941
- | `@drupflare/spool/proxy` | `spoolProxy`, `chooseBases`, the tag readers and the eligibility verdict |
1942
- | `@drupflare/spool/stores` | `memoryStore` and the `VersionStore` type |
1943
- | `@drupflare/spool/stores/durable-object` | `durableObjectStore` and the `SpoolVersions` class |
1944
- | `@drupflare/spool/sw` | `serve`, for a site that bundles its own worker |
1945
- | `@drupflare/spool/client` | `register` and the page handle |
1946
- | `@drupflare/spool/report` | the energy model, capacity plans and institution reports |
1947
- | `@drupflare/spool/trim` | `buildTrimManifest`, `trimCss` and `guardPage` |
1948
- | `@drupflare/spool/media` | `tinyimgEncoder`, `imageVariant` and `rewriteVideo` |
574
+ ## Subpath Exports
575
+
576
+ | Import | What is in it |
577
+ | ---------------------------------------- | ------------------------------------------------------------------------------------------------------------- |
578
+ | `@drupflare/spool` | the protocol and report modules |
579
+ | `@drupflare/spool/protocol` | envelopes, artifacts, patterns, templates, the manifest, view events |
580
+ | `@drupflare/spool/server` | `createSpool`, `defineRoute`, `defineDataset`, `defineWrite`, `datasetProof`, sinks, identity and work series |
581
+ | `@drupflare/spool/node` | `toNodeListener`, `toRequest`, `writeResponse` |
582
+ | `@drupflare/spool/proxy` | `spoolProxy`, `chooseBases`, the tag readers and the eligibility verdict |
583
+ | `@drupflare/spool/stores` | `memoryStore` and the `VersionStore` type |
584
+ | `@drupflare/spool/stores/durable-object` | `durableObjectStore` and the `SpoolVersions` class |
585
+ | `@drupflare/spool/sw` | `serve`, for a site that bundles its own worker |
586
+ | `@drupflare/spool/client` | `register`, the page handle and `createActivity` |
587
+ | `@drupflare/spool/report` | the energy model, capacity plans and institution reports |
588
+ | `@drupflare/spool/trim` | `buildTrimManifest`, `trimCss` and `guardPage` |
589
+ | `@drupflare/spool/media` | `tinyimgEncoder`, `imageVariant`, `rewriteVideo` and `rewriteEmbeds` |
590
+ | `@drupflare/spool/express` | `spoolExpress` |
591
+ | `@drupflare/spool/hono` | `spoolHono` |
592
+ | `@drupflare/spool/vite` | `spoolVite` |
593
+ | `@drupflare/spool/vue` | `spoolVue` and `vuePages` |
594
+ | `@drupflare/spool/angular` | `spoolAngular` |
595
+ | `@drupflare/spool/nuxt` | the Nuxt module (default export) and `#spool/pages` |
596
+ | `@drupflare/spool/react-router` | `spoolReactRouter`, `reactRouterPages` and `flattenRoutes` |
597
+ | `@drupflare/spool/next` | the Next adapter (default export), `spoolTag`, `spoolRouteHandler`, `nextPages` and `SPOOL_PAGES_FILE` |
1949
598
 
1950
599
  ---
1951
600
 
1952
- ## ๐Ÿงช Testing
601
+ ## Testing
1953
602
 
1954
603
  ```sh
1955
604
  bun run typecheck
@@ -1957,18 +606,39 @@ bun run test
1957
606
  bun run test:coverage
1958
607
  bun run test:browser
1959
608
  bun run prettier:check
609
+ bun run impact:check
1960
610
  bun run docs:build
1961
611
  bun run build
1962
612
  ```
1963
613
 
1964
- `test` runs the unit project in workerd and the node project, which includes a test that packs the
614
+ `typecheck` also compiles every `ts` fence in this page and the guides as written, and it fails with
615
+ the page and line of the first fence that does not compile. A fence that cannot stand alone takes
616
+ `ts no-check` after the opening backticks and is skipped. Samples that tests run live under
617
+ `tests/node/readme/`.
618
+
619
+ `impact:check` fails when `docs/impact.md` differs from the readings in `measurements/`. A node
620
+ test fails when `src/server/embedded.ts` differs from the source under `src/protocol`, `src/sw`
621
+ and `src/client`, and `bun run build:sw` regenerates it. `test` runs the unit project in workerd and the node project, which includes a test that packs the
1965
622
  package and imports every subpath in plain Node. `test:browser` runs Chromium, Firefox and WebKit
1966
623
  against real servers and counts the requests that reach them. `bunx playwright install chromium
1967
624
  firefox webkit` fetches the browsers.
1968
625
 
1969
626
  ---
1970
627
 
1971
- ## ๐Ÿ”— Related Repositories
628
+ ## Documentation
629
+
630
+ - [docs/configuration.md](docs/configuration.md) covers the settings a first deployment decides.
631
+ - [docs/protocol.md](docs/protocol.md) holds the paths, headers, JSON bodies and limits for anyone
632
+ who writes an adapter or a second server. A test pins what `Spool-Protocol: 1` freezes, and the
633
+ page lists what may change without a new number.
634
+ - [docs/security.md](docs/security.md) holds the threat model and the open limitations.
635
+ - [docs/impact.md](docs/impact.md) holds the energy model, the readings and what they leave unsaid.
636
+ - [docs/economics.md](docs/economics.md) holds the bandwidth and Worker invocation projections.
637
+ - [examples/README.md](examples/README.md) lists the example sites and runtimes, and the command that runs each one.
638
+
639
+ ---
640
+
641
+ ## Related Repositories
1972
642
 
1973
643
  | Repository | What it is |
1974
644
  | --------------------------------------------------------------- | -------------------------------------------------------- |
@@ -1979,6 +649,8 @@ firefox webkit` fetches the browsers.
1979
649
 
1980
650
  ---
1981
651
 
1982
- ## ๐Ÿ“„ License
652
+ ## License
1983
653
 
1984
654
  MIT (c) Gregory Mitchell 2026. See [LICENSE](LICENSE).
655
+
656
+ The documentation favicons in `docs/` come from Google's [Noto Emoji](https://github.com/googlefonts/noto-emoji) set, licensed under [Apache 2.0](https://www.apache.org/licenses/LICENSE-2.0).