@drupflare/spool 0.3.0 โ 1.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +279 -1607
- package/dist/angular/index.d.ts +13 -0
- package/dist/angular/index.js +25 -0
- package/dist/angular/pages.d.ts +28 -0
- package/dist/angular/pages.js +85 -0
- package/dist/cdn/client.js +1 -1
- package/dist/cdn/location.d.ts +6 -0
- package/dist/cdn/location.js +8 -0
- package/dist/cdn/manifest.json +7 -7
- package/dist/cdn/spool-sw.js +1 -1
- package/dist/client/bootstrap.d.ts +57 -0
- package/dist/client/bootstrap.js +307 -0
- package/dist/client/contact.d.ts +24 -0
- package/dist/client/contact.js +66 -0
- package/dist/client/declared.d.ts +23 -0
- package/dist/client/declared.js +68 -0
- package/dist/client/entry.js +123 -2
- package/dist/client/gate.d.ts +1 -0
- package/dist/client/gate.js +148 -0
- package/dist/client/index.d.ts +25 -0
- package/dist/client/index.js +1 -0
- package/dist/client/live.d.ts +66 -0
- package/dist/client/live.js +225 -0
- package/dist/client/navigate-entry.d.ts +1 -0
- package/dist/client/navigate-entry.js +4 -0
- package/dist/client/navigate.d.ts +54 -0
- package/dist/client/navigate.js +272 -0
- package/dist/client/persist.d.ts +2 -0
- package/dist/client/persist.js +9 -0
- package/dist/client/register.d.ts +34 -2
- package/dist/client/register.js +64 -7
- package/dist/express/index.d.ts +11 -0
- package/dist/express/index.js +209 -0
- package/dist/hono/index.d.ts +19 -0
- package/dist/hono/index.js +49 -0
- package/dist/next/index.d.ts +33 -0
- package/dist/next/index.js +41 -0
- package/dist/next/pages.d.ts +22 -0
- package/dist/next/pages.js +39 -0
- package/dist/node/listener.d.ts +3 -0
- package/dist/node/listener.js +57 -2
- package/dist/nuxt/index.d.ts +22 -0
- package/dist/nuxt/index.js +104 -0
- package/dist/nuxt/pages.d.ts +26 -0
- package/dist/nuxt/pages.js +48 -0
- package/dist/nuxt/runtime/index.d.ts +24 -0
- package/dist/nuxt/runtime/index.js +58 -0
- package/dist/plugins/coverage.d.ts +14 -0
- package/dist/plugins/coverage.js +27 -0
- package/dist/plugins/mode.d.ts +19 -0
- package/dist/plugins/mode.js +20 -0
- package/dist/plugins/path.d.ts +16 -0
- package/dist/plugins/path.js +139 -0
- package/dist/plugins/report.d.ts +22 -0
- package/dist/plugins/report.js +18 -0
- package/dist/plugins/routes.d.ts +38 -0
- package/dist/plugins/routes.js +52 -0
- package/dist/plugins/static.d.ts +42 -0
- package/dist/plugins/static.js +97 -0
- package/dist/protocol/bundle.d.ts +111 -0
- package/dist/protocol/bundle.js +428 -0
- package/dist/protocol/config.d.ts +80 -1
- package/dist/protocol/config.js +31 -1
- package/dist/protocol/dataset.d.ts +189 -0
- package/dist/protocol/dataset.js +212 -0
- package/dist/protocol/envelope.d.ts +34 -1
- package/dist/protocol/envelope.js +16 -1
- package/dist/protocol/errors.d.ts +1 -1
- package/dist/protocol/graph.d.ts +23 -4
- package/dist/protocol/graph.js +34 -3
- package/dist/protocol/index-rows.d.ts +22 -0
- package/dist/protocol/index-rows.js +53 -0
- package/dist/protocol/index.d.ts +5 -0
- package/dist/protocol/index.js +5 -0
- package/dist/protocol/normalise.d.ts +4 -4
- package/dist/protocol/normalise.js +9 -5
- package/dist/protocol/plan.d.ts +100 -0
- package/dist/protocol/plan.js +661 -0
- package/dist/protocol/provision.d.ts +46 -0
- package/dist/protocol/provision.js +38 -0
- package/dist/protocol/query.d.ts +66 -5
- package/dist/protocol/query.js +103 -4
- package/dist/protocol/ring.d.ts +58 -0
- package/dist/protocol/ring.js +6 -0
- package/dist/protocol/route.d.ts +63 -13
- package/dist/protocol/route.js +126 -6
- package/dist/protocol/tag.d.ts +22 -2
- package/dist/protocol/tag.js +25 -4
- package/dist/protocol/view.d.ts +2 -0
- package/dist/protocol/view.js +8 -2
- package/dist/protocol/views.d.ts +6 -0
- package/dist/protocol/views.js +15 -0
- package/dist/protocol/write.d.ts +234 -0
- package/dist/protocol/write.js +343 -0
- package/dist/proxy/eligible.d.ts +22 -1
- package/dist/proxy/eligible.js +62 -15
- package/dist/proxy/proxy.js +28 -18
- package/dist/react-router/index.d.ts +20 -0
- package/dist/react-router/index.js +60 -0
- package/dist/react-router/pages.d.ts +34 -0
- package/dist/react-router/pages.js +83 -0
- package/dist/report/dataset.d.ts +48 -0
- package/dist/report/dataset.js +93 -0
- package/dist/report/demand.d.ts +49 -0
- package/dist/report/demand.js +60 -0
- package/dist/report/derive.d.ts +285 -0
- package/dist/report/derive.js +676 -1
- package/dist/report/economics.d.ts +66 -1
- package/dist/report/economics.js +208 -1
- package/dist/report/headline.d.ts +158 -0
- package/dist/report/headline.js +473 -0
- package/dist/report/index.d.ts +4 -0
- package/dist/report/index.js +4 -0
- package/dist/report/inputs.d.ts +176 -0
- package/dist/report/inputs.js +248 -0
- package/dist/report/institution.d.ts +188 -1
- package/dist/report/institution.js +146 -0
- package/dist/report/loops.d.ts +73 -0
- package/dist/report/loops.js +101 -0
- package/dist/report/measurement.d.ts +1 -1
- package/dist/report/measurement.js +7 -1
- package/dist/report/model.d.ts +2 -0
- package/dist/report/model.js +4 -0
- package/dist/report/thirdparty.d.ts +57 -0
- package/dist/report/thirdparty.js +111 -0
- package/dist/report/traffic.d.ts +8 -0
- package/dist/report/traffic.js +7 -0
- package/dist/report/trim.d.ts +56 -0
- package/dist/report/trim.js +83 -1
- package/dist/server/bundle.d.ts +55 -0
- package/dist/server/bundle.js +172 -0
- package/dist/server/dataset.d.ts +54 -0
- package/dist/server/dataset.js +355 -0
- package/dist/server/embedded.d.ts +7 -3
- package/dist/server/embedded.js +7 -3
- package/dist/server/index.d.ts +6 -0
- package/dist/server/index.js +6 -0
- package/dist/server/live-sockets.d.ts +54 -0
- package/dist/server/live-sockets.js +131 -0
- package/dist/server/live.d.ts +146 -0
- package/dist/server/live.js +228 -0
- package/dist/server/ring.d.ts +77 -0
- package/dist/server/ring.js +311 -0
- package/dist/server/spool.d.ts +158 -3
- package/dist/server/spool.js +578 -41
- package/dist/server/static.d.ts +42 -0
- package/dist/server/static.js +29 -0
- package/dist/server/stub.d.ts +43 -5
- package/dist/server/stub.js +126 -8
- package/dist/server/write.d.ts +49 -0
- package/dist/server/write.js +223 -0
- package/dist/stores/durable-object.d.ts +46 -0
- package/dist/stores/durable-object.js +114 -0
- package/dist/sw/analytics.d.ts +7 -1
- package/dist/sw/analytics.js +16 -2
- package/dist/sw/coverage.d.ts +6 -0
- package/dist/sw/coverage.js +34 -0
- package/dist/sw/dataset-sync.d.ts +76 -0
- package/dist/sw/dataset-sync.js +149 -0
- package/dist/sw/dataset.d.ts +91 -0
- package/dist/sw/dataset.js +318 -0
- package/dist/sw/family.d.ts +15 -0
- package/dist/sw/family.js +48 -0
- package/dist/sw/graph/index.d.ts +31 -1
- package/dist/sw/graph/index.js +57 -5
- package/dist/sw/graph/invalidate.d.ts +15 -3
- package/dist/sw/graph/invalidate.js +42 -4
- package/dist/sw/idb.d.ts +2 -0
- package/dist/sw/idb.js +80 -11
- package/dist/sw/index.d.ts +2 -0
- package/dist/sw/index.js +1 -0
- package/dist/sw/pack/store.d.ts +128 -0
- package/dist/sw/pack/store.js +737 -0
- package/dist/sw/partition.d.ts +6 -1
- package/dist/sw/partition.js +21 -2
- package/dist/sw/policy.d.ts +2 -0
- package/dist/sw/policy.js +11 -0
- package/dist/sw/principal-dataset.d.ts +64 -0
- package/dist/sw/principal-dataset.js +254 -0
- package/dist/sw/probe.d.ts +16 -0
- package/dist/sw/probe.js +25 -0
- package/dist/sw/render.d.ts +7 -2
- package/dist/sw/render.js +19 -5
- package/dist/sw/replica.d.ts +12 -3
- package/dist/sw/replica.js +40 -5
- package/dist/sw/router.d.ts +3 -2
- package/dist/sw/router.js +15 -2
- package/dist/sw/runtime.d.ts +90 -2
- package/dist/sw/runtime.js +857 -61
- package/dist/sw/scheduler.d.ts +6 -1
- package/dist/sw/scheduler.js +22 -5
- package/dist/sw/seal.d.ts +13 -0
- package/dist/sw/seal.js +13 -4
- package/dist/sw/static.d.ts +18 -0
- package/dist/sw/static.js +84 -1
- package/dist/sw/storage.d.ts +51 -1
- package/dist/sw/storage.js +33 -1
- package/dist/sw/write.d.ts +147 -0
- package/dist/sw/write.js +625 -0
- package/dist/version.d.ts +1 -1
- package/dist/version.js +1 -2
- package/dist/vite/index.d.ts +26 -0
- package/dist/vite/index.js +66 -0
- package/dist/vue/index.d.ts +36 -0
- package/dist/vue/index.js +53 -0
- package/package.json +65 -4
package/README.md
CHANGED
|
@@ -1,68 +1,77 @@
|
|
|
1
1
|
# ๐งต spool
|
|
2
2
|
|
|
3
|
-
>
|
|
3
|
+
> A browser replica that removes repeated demand from your server
|
|
4
4
|
|
|
5
5
|
[](https://github.com/drupflare/spool/actions/workflows/build.yml)
|
|
6
6
|
[](https://github.com/drupflare/spool/actions/workflows/browser.yml)
|
|
7
7
|
[](https://github.com/drupflare/spool/actions/workflows/prettier.yml)
|
|
8
8
|
[](https://codecov.io/gh/drupflare/spool)
|
|
9
|
-
[](LICENSE)
|
|
9
|
+
[](https://github.com/drupflare/spool/blob/master/LICENSE)
|
|
10
10
|
|
|
11
|
-
spool keeps a replica of what a page needs
|
|
12
|
-
|
|
13
|
-
|
|
11
|
+
spool removes repeated execution demand from your server. It keeps a replica of what a page needs
|
|
12
|
+
on the visitor's device. The server sends templates and authorised, versioned data, and a service
|
|
13
|
+
worker computes eligible pages from them. A page answered there is a request that never reaches
|
|
14
|
+
your Worker or origin, and server work that never runs. The contract is a count: a test asserts
|
|
15
|
+
the server hits of a local view, and in Chromium and Firefox the count is zero.
|
|
16
|
+
|
|
17
|
+
spool has three uses. In a drupflare deployment it takes eligible public views off the Worker.
|
|
18
|
+
For a team on a free or constrained plan it lowers the requests the plan has to carry, by an
|
|
19
|
+
amount that depends on the site's traffic ([Request Arithmetic](ADVANCED_USAGE.md#request-arithmetic)). It is also
|
|
20
|
+
a research system: every figure it reports carries a layer and a method.
|
|
14
21
|
|
|
15
22
|
It runs on any runtime that speaks `Request` and `Response`, bridges `node:http` for the rest, and
|
|
16
23
|
has no runtime dependencies. A site with no server code can load it from two files and a manifest.
|
|
17
24
|
|
|
18
25
|
---
|
|
19
26
|
|
|
20
|
-
##
|
|
21
|
-
|
|
22
|
-
- [Why Spool](
|
|
23
|
-
- [Install](
|
|
24
|
-
- [Quick Start](
|
|
25
|
-
- [Page Runtime](
|
|
26
|
-
- [Runtimes](
|
|
27
|
-
- [Compatibility](
|
|
28
|
-
- [Static Sites](
|
|
29
|
-
- [
|
|
30
|
-
- [
|
|
31
|
-
- [
|
|
32
|
-
- [
|
|
33
|
-
- [
|
|
34
|
-
- [
|
|
35
|
-
- [
|
|
36
|
-
- [
|
|
37
|
-
- [
|
|
38
|
-
- [
|
|
39
|
-
- [
|
|
40
|
-
- [Capacity Reports](#-capacity-reports)
|
|
41
|
-
- [Out of Scope](#-out-of-scope)
|
|
42
|
-
- [Browser Support](#-browser-support)
|
|
43
|
-
- [Subpath Exports](#-subpath-exports)
|
|
44
|
-
- [Testing](#-testing)
|
|
45
|
-
- [Related Repositories](#-related-repositories)
|
|
46
|
-
- [License](#-license)
|
|
27
|
+
## Table of Contents
|
|
28
|
+
|
|
29
|
+
- [Why Spool](#why-spool)
|
|
30
|
+
- [Install](#install)
|
|
31
|
+
- [Quick Start](#quick-start)
|
|
32
|
+
- [Page Runtime](#page-runtime)
|
|
33
|
+
- [Runtimes](#runtimes)
|
|
34
|
+
- [Compatibility](#compatibility)
|
|
35
|
+
- [Static Sites](#static-sites)
|
|
36
|
+
- [Advanced Usage](#advanced-usage)
|
|
37
|
+
- [Security](#security)
|
|
38
|
+
- [Energy Reporting](#energy-reporting)
|
|
39
|
+
- [Prior Art](#prior-art)
|
|
40
|
+
- [Out of Scope](#out-of-scope)
|
|
41
|
+
- [Browser Support](#browser-support)
|
|
42
|
+
- [Subpath Exports](#subpath-exports)
|
|
43
|
+
- [Testing](#testing)
|
|
44
|
+
- [Documentation](#documentation)
|
|
45
|
+
- [Related Repositories](#related-repositories)
|
|
46
|
+
- [License](#license)
|
|
47
47
|
|
|
48
48
|
---
|
|
49
49
|
|
|
50
|
-
##
|
|
50
|
+
## Why Spool
|
|
51
|
+
|
|
52
|
+
A local view removes, in order, a request that would have reached the Worker, the server work
|
|
53
|
+
behind it (rendering, queries, Durable Object and database reads), and the bytes of a document the
|
|
54
|
+
device already holds. Energy follows from those and is one objective an administrator may weigh.
|
|
51
55
|
|
|
52
56
|
A response is a deterministic transformation of a set of authorised, versioned dependencies. The
|
|
53
57
|
server half names those dependencies and sends the snapshot the visitor may hold. The worker keeps
|
|
54
58
|
a graph of records, projections and responses, with their versions, and syncs only what the next
|
|
55
59
|
page needs.
|
|
56
60
|
|
|
57
|
-
Your application keeps every
|
|
58
|
-
|
|
59
|
-
changes no page that was not
|
|
61
|
+
Your application keeps every access decision and every page it cannot reproduce safely on the
|
|
62
|
+
device. It keeps every write too, except the last-value writes you declare, which ship off. A route
|
|
63
|
+
the worker cannot answer goes to the network, so adopting spool changes no page that was not
|
|
64
|
+
declared.
|
|
60
65
|
|
|
61
|
-
|
|
66
|
+
A collection of thousands of rows, such as a directory or a course catalogue, can be held as a
|
|
67
|
+
[dataset](ADVANCED_USAGE.md#datasets). The device seeds it once and then receives only the rows that changed.
|
|
68
|
+
|
|
69
|
+
[docs/impact.md](docs/impact.md) holds the request, cost and energy figures and the measurements
|
|
70
|
+
behind them. See [Energy Reporting](#energy-reporting).
|
|
62
71
|
|
|
63
72
|
---
|
|
64
73
|
|
|
65
|
-
##
|
|
74
|
+
## Install
|
|
66
75
|
|
|
67
76
|
### Package
|
|
68
77
|
|
|
@@ -88,17 +97,17 @@ notes list the hash for each version.
|
|
|
88
97
|
></script>
|
|
89
98
|
```
|
|
90
99
|
|
|
91
|
-
[Page Runtime](
|
|
100
|
+
[Page Runtime](#page-runtime) shows how `createSpool` injects the same tag.
|
|
92
101
|
|
|
93
102
|
### Direct Files
|
|
94
103
|
|
|
95
104
|
`spool-sw.js`, `client.js` and `manifest.json` are attached to every GitHub release and published
|
|
96
105
|
under `https://cdn.drupflare.com/spool/<version>/`. A site that serves them as plain files needs no
|
|
97
|
-
server library. See [Static Sites](
|
|
106
|
+
server library. See [Static Sites](#static-sites).
|
|
98
107
|
|
|
99
108
|
---
|
|
100
109
|
|
|
101
|
-
##
|
|
110
|
+
## Quick Start
|
|
102
111
|
|
|
103
112
|
Every runtime below makes the same three moves:
|
|
104
113
|
|
|
@@ -108,6 +117,9 @@ Every runtime below makes the same three moves:
|
|
|
108
117
|
every HTML page the application returns, and `spool.decorate` does the same where there is no
|
|
109
118
|
Fetch response to wrap.
|
|
110
119
|
|
|
120
|
+
The [Getting Started](docs/getting-started.md) page walks through the same steps with the install
|
|
121
|
+
options, a run and the checks that confirm the setup.
|
|
122
|
+
|
|
111
123
|
### Declare Routes
|
|
112
124
|
|
|
113
125
|
A route has a path pattern, a strategy, the tags it depends on and a `load` function that returns
|
|
@@ -169,7 +181,7 @@ holding a page that depends on them fetch a fresh copy before they show it again
|
|
|
169
181
|
|
|
170
182
|
---
|
|
171
183
|
|
|
172
|
-
##
|
|
184
|
+
## Page Runtime
|
|
173
185
|
|
|
174
186
|
`handle` and `decorate` inject one script tag into each HTML page. By default the script is
|
|
175
187
|
served from the same origin at `/.spool/<version>/client.js`, which suits a strict
|
|
@@ -193,8 +205,27 @@ it against the hash, so a changed file never runs. Release files are never overw
|
|
|
193
205
|
script fails to load, the page works as it did before.
|
|
194
206
|
|
|
195
207
|
The script registers `/spool-sw.js` and exposes `window.spool.ready`, which resolves to the client
|
|
196
|
-
handle or to `null` where service workers do not exist.
|
|
197
|
-
|
|
208
|
+
handle or to `null` where service workers do not exist. It also exposes `window.spool.activity`,
|
|
209
|
+
described below, and the write calls of [Declared Writes](ADVANCED_USAGE.md#declared-writes). To bundle the runtime
|
|
210
|
+
yourself, import `register` from `@drupflare/spool/client`. The `window.spool` types come with that
|
|
211
|
+
path too, so a project that imports it gets them.
|
|
212
|
+
|
|
213
|
+
`createSpool({ workerMode: 'never' })` serves a site with no service worker. The page installs its
|
|
214
|
+
pack and answers held reads itself, and `/spool-sw.js` answers `404`. A declared write goes through
|
|
215
|
+
`window.spool.fetch` in place of `fetch`. The answer updates the held pack from the delta or digest it
|
|
216
|
+
carries, and any other URL goes to `fetch` unchanged. `spool.write` still queues a value for the
|
|
217
|
+
worker, so without a worker it answers `unavailable`. A public HTML page carries the declarations in
|
|
218
|
+
its ring, or in a `spool-writes` element when it has no ring. On a site with the bootstrap gate, every
|
|
219
|
+
public HTML page carries the ring.
|
|
220
|
+
|
|
221
|
+
```ts
|
|
222
|
+
// a page with no worker
|
|
223
|
+
const response = await window.spool?.fetch('/courses/101', {
|
|
224
|
+
method: 'PATCH',
|
|
225
|
+
headers: { 'content-type': 'application/json' },
|
|
226
|
+
body: JSON.stringify({ title: 'Biology 101' })
|
|
227
|
+
});
|
|
228
|
+
```
|
|
198
229
|
|
|
199
230
|
### Activity Hooks
|
|
200
231
|
|
|
@@ -231,462 +262,68 @@ measured, the off-screen pause lowered power for a page whose frame loop kept ru
|
|
|
231
262
|
hidden-page pause showed no saving for a once-a-second poll. Both readings are in
|
|
232
263
|
[docs/impact.md](docs/impact.md#metered-figures-beside-the-ledger).
|
|
233
264
|
|
|
234
|
-
|
|
235
|
-
|
|
236
|
-
## ๐งฉ Runtimes
|
|
237
|
-
|
|
238
|
-
Each example uses the `routes`, `page` and `spool` from [Quick Start](#-quick-start). The
|
|
239
|
-
[compatibility table](#-compatibility) lists what each framework hands a handler.
|
|
240
|
-
|
|
241
|
-
### Cloudflare Workers
|
|
242
|
-
|
|
243
|
-
Bind the Durable Object that holds the version vector, then pass its namespace to
|
|
244
|
-
`durableObjectStore`.
|
|
245
|
-
|
|
246
|
-
```jsonc
|
|
247
|
-
{
|
|
248
|
-
"durable_objects": { "bindings": [{ "name": "SPOOL_VERSIONS", "class_name": "SpoolVersions" }] },
|
|
249
|
-
"migrations": [{ "tag": "v1", "new_sqlite_classes": ["SpoolVersions"] }]
|
|
250
|
-
}
|
|
251
|
-
```
|
|
252
|
-
|
|
253
|
-
```ts
|
|
254
|
-
import { createSpool } from '@drupflare/spool/server';
|
|
255
|
-
import { durableObjectStore } from '@drupflare/spool/stores/durable-object';
|
|
256
|
-
import { page, routes } from './routes';
|
|
257
|
-
|
|
258
|
-
export { SpoolVersions } from '@drupflare/spool/stores/durable-object';
|
|
259
|
-
|
|
260
|
-
export default {
|
|
261
|
-
fetch(request: Request, env: Cloudflare.Env) {
|
|
262
|
-
const spool = createSpool({ routes, store: durableObjectStore(env.SPOOL_VERSIONS) });
|
|
263
|
-
return spool.handle(request, page);
|
|
264
|
-
}
|
|
265
|
-
};
|
|
266
|
-
```
|
|
267
|
-
|
|
268
|
-
### Cloudflare Pages Functions
|
|
269
|
-
|
|
270
|
-
A function takes a context with the request and a `next` that runs the next function or the
|
|
271
|
-
static asset server.
|
|
272
|
-
|
|
273
|
-
```ts
|
|
274
|
-
import { spool } from './spool';
|
|
275
|
-
|
|
276
|
-
export const onRequest = (context: { request: Request; next(): Promise<Response> }) =>
|
|
277
|
-
spool.handle(context.request, () => context.next());
|
|
278
|
-
```
|
|
279
|
-
|
|
280
|
-
### Hono
|
|
281
|
-
|
|
282
|
-
Hono ignores the response a middleware returns once a route has answered, so assign it to
|
|
283
|
-
`c.res`.
|
|
284
|
-
|
|
285
|
-
```ts
|
|
286
|
-
import { Hono } from 'hono';
|
|
287
|
-
import { page } from './routes';
|
|
288
|
-
import { spool } from './spool';
|
|
289
|
-
|
|
290
|
-
const app = new Hono();
|
|
291
|
-
|
|
292
|
-
app.use('*', async (c, next) => {
|
|
293
|
-
c.res = await spool.handle(c.req.raw, async () => {
|
|
294
|
-
await next();
|
|
295
|
-
return c.res;
|
|
296
|
-
});
|
|
297
|
-
});
|
|
298
|
-
app.get('/node/:id', (c) => page(c.req.raw));
|
|
299
|
-
|
|
300
|
-
export default app;
|
|
301
|
-
```
|
|
302
|
-
|
|
303
|
-
### Elysia
|
|
304
|
-
|
|
305
|
-
`onRequest` answers the spool paths and returns nothing for the rest. The hook has to be an
|
|
306
|
-
`async` function, because Elysia does not await a promise that a plain function returns.
|
|
307
|
-
|
|
308
|
-
```ts
|
|
309
|
-
import { Elysia } from 'elysia';
|
|
310
|
-
import { page } from './routes';
|
|
311
|
-
import { spool } from './spool';
|
|
312
|
-
|
|
313
|
-
export const app = new Elysia()
|
|
314
|
-
.onRequest(async ({ request }) => {
|
|
315
|
-
const { pathname } = new URL(request.url);
|
|
316
|
-
return pathname === '/spool-sw.js' ? spool.stub() : await spool.respond(request);
|
|
317
|
-
})
|
|
318
|
-
.get('/node/:id', ({ request }) => spool.decorate(request, page(request)));
|
|
319
|
-
```
|
|
320
|
-
|
|
321
|
-
### itty-router
|
|
322
|
-
|
|
323
|
-
A handler that returns `undefined` passes the request to the next one, which is what
|
|
324
|
-
`spool.respond` returns for a path that is not spool's.
|
|
325
|
-
|
|
326
|
-
```ts
|
|
327
|
-
import { IttyRouter } from 'itty-router';
|
|
328
|
-
import { page } from './routes';
|
|
329
|
-
import { spool } from './spool';
|
|
330
|
-
|
|
331
|
-
const router = IttyRouter();
|
|
332
|
-
|
|
333
|
-
router
|
|
334
|
-
.all('*', (request) => spool.respond(request))
|
|
335
|
-
.get('/spool-sw.js', () => spool.stub())
|
|
336
|
-
.get('/node/:id', (request) => spool.decorate(request, page(request)));
|
|
337
|
-
|
|
338
|
-
export default { fetch: router.fetch };
|
|
339
|
-
```
|
|
340
|
-
|
|
341
|
-
### Bun
|
|
342
|
-
|
|
343
|
-
Bun serves a default export that has a `fetch` function.
|
|
344
|
-
|
|
345
|
-
```ts
|
|
346
|
-
import { page } from './routes';
|
|
347
|
-
import { spool } from './spool';
|
|
348
|
-
|
|
349
|
-
export default {
|
|
350
|
-
port: 3000,
|
|
351
|
-
fetch: (request: Request) => spool.handle(request, page)
|
|
352
|
-
};
|
|
353
|
-
```
|
|
354
|
-
|
|
355
|
-
### Deno
|
|
356
|
-
|
|
357
|
-
Run it with `deno serve`.
|
|
358
|
-
|
|
359
|
-
```ts
|
|
360
|
-
import { page } from './routes';
|
|
361
|
-
import { spool } from './spool';
|
|
362
|
-
|
|
363
|
-
export default {
|
|
364
|
-
fetch: (request: Request) => spool.handle(request, page)
|
|
365
|
-
};
|
|
366
|
-
```
|
|
367
|
-
|
|
368
|
-
### Netlify Functions
|
|
369
|
-
|
|
370
|
-
A function takes a `Request` and returns a `Response`. `config.path` sends the spool paths and the
|
|
371
|
-
pages to it.
|
|
372
|
-
|
|
373
|
-
```ts
|
|
374
|
-
import type { Config, Context } from '@netlify/functions';
|
|
375
|
-
import { page } from './routes';
|
|
376
|
-
import { spool } from './spool';
|
|
377
|
-
|
|
378
|
-
export default (request: Request, context: Context) => spool.handle(request, page);
|
|
379
|
-
|
|
380
|
-
export const config: Config = { path: ['/node/*', '/spool-sw.js', '/.spool/*'] };
|
|
381
|
-
```
|
|
382
|
-
|
|
383
|
-
### Netlify Edge Functions
|
|
384
|
-
|
|
385
|
-
An edge function runs in front of the site. `context.next()` returns the response the site would
|
|
386
|
-
have sent, which `handle` decorates.
|
|
387
|
-
|
|
388
|
-
```ts
|
|
389
|
-
import type { Config, Context } from '@netlify/edge-functions';
|
|
390
|
-
import { spool } from './spool';
|
|
391
|
-
|
|
392
|
-
export default (request: Request, context: Context) => spool.handle(request, () => context.next());
|
|
393
|
-
|
|
394
|
-
export const config: Config = { path: '/*' };
|
|
395
|
-
```
|
|
396
|
-
|
|
397
|
-
### Vercel Functions
|
|
398
|
-
|
|
399
|
-
A function under `api/` takes a default export with `fetch`. Send every path to it from
|
|
400
|
-
`vercel.json`, so the function sees `/.spool/*` and `/spool-sw.js` as well as the pages. Function
|
|
401
|
-
instances do not share memory, so `store` has to be a shared `VersionStore`.
|
|
402
|
-
|
|
403
|
-
```json
|
|
404
|
-
{ "rewrites": [{ "source": "/(.*)", "destination": "/api" }] }
|
|
405
|
-
```
|
|
406
|
-
|
|
407
|
-
```ts
|
|
408
|
-
import { page } from './routes';
|
|
409
|
-
import { spool } from './spool';
|
|
410
|
-
|
|
411
|
-
export default {
|
|
412
|
-
fetch: (request: Request) => spool.handle(request, page)
|
|
413
|
-
};
|
|
414
|
-
```
|
|
415
|
-
|
|
416
|
-
### Next.js
|
|
417
|
-
|
|
418
|
-
Next.js route handlers return `Response` objects, so `handle` wraps one directly. The spool
|
|
419
|
-
endpoints and the worker script go through the proxy, which returns `undefined` for every other
|
|
420
|
-
request. Next.js 16 calls the file `proxy.ts` and the export `proxy`. Earlier versions call them
|
|
421
|
-
`middleware.ts` and `middleware`. The proxy and the route handlers need the same shared store.
|
|
422
|
-
|
|
423
|
-
```ts
|
|
424
|
-
import { spool } from './spool';
|
|
425
|
-
|
|
426
|
-
export function proxy(request: Request) {
|
|
427
|
-
const { pathname } = new URL(request.url);
|
|
428
|
-
return pathname === '/spool-sw.js' ? spool.stub() : spool.respond(request);
|
|
429
|
-
}
|
|
430
|
-
```
|
|
431
|
-
|
|
432
|
-
```ts
|
|
433
|
-
import { page } from './routes';
|
|
434
|
-
import { spool } from './spool';
|
|
435
|
-
|
|
436
|
-
export const GET = (request: Request) => spool.handle(request, page);
|
|
437
|
-
```
|
|
438
|
-
|
|
439
|
-
### Nuxt and Nitro
|
|
440
|
-
|
|
441
|
-
h3 turns an event into a standard request with `toWebRequest`, and it sends a `Response` that a
|
|
442
|
-
handler returns. A server middleware that returns nothing passes the request on. Put the first
|
|
443
|
-
file in `server/middleware/` and the second in `server/routes/node/[id].ts`. Both share one store
|
|
444
|
-
from `./store`.
|
|
445
|
-
|
|
446
|
-
```ts
|
|
447
|
-
import { defineEventHandler, toWebRequest } from 'h3';
|
|
448
|
-
import { spool } from './spool';
|
|
449
|
-
|
|
450
|
-
export default defineEventHandler((event) => {
|
|
451
|
-
const request = toWebRequest(event);
|
|
452
|
-
return new URL(request.url).pathname === '/spool-sw.js' ? spool.stub() : spool.respond(request);
|
|
453
|
-
});
|
|
454
|
-
```
|
|
455
|
-
|
|
456
|
-
```ts
|
|
457
|
-
import { defineEventHandler, toWebRequest } from 'h3';
|
|
458
|
-
import { page } from './routes';
|
|
459
|
-
import { spool } from './spool';
|
|
460
|
-
|
|
461
|
-
export default defineEventHandler((event) => spool.handle(toWebRequest(event), page));
|
|
462
|
-
```
|
|
265
|
+
### In-Page Navigation
|
|
463
266
|
|
|
464
|
-
|
|
267
|
+
The page answers a link or a GET form in place when its content sits in a swap region. The region is the first element marked `data-spool-swap`, or else the first `<main>`:
|
|
465
268
|
|
|
466
|
-
|
|
467
|
-
|
|
468
|
-
|
|
469
|
-
|
|
470
|
-
|
|
471
|
-
|
|
472
|
-
|
|
473
|
-
|
|
474
|
-
|
|
475
|
-
```
|
|
476
|
-
|
|
477
|
-
### Azure Functions
|
|
478
|
-
|
|
479
|
-
An Azure v4 HTTP function takes an `HttpRequest`, which follows the Fetch standard, and returns an
|
|
480
|
-
`HttpResponseInit`. Route every path to it with `{*path}`.
|
|
481
|
-
|
|
482
|
-
```ts
|
|
483
|
-
import { app, type HttpRequest, type HttpResponseInit } from '@azure/functions';
|
|
484
|
-
import { page } from './routes';
|
|
485
|
-
import { spool } from './spool';
|
|
486
|
-
|
|
487
|
-
export async function handler(request: HttpRequest): Promise<HttpResponseInit> {
|
|
488
|
-
const hasBody = request.method !== 'GET' && request.method !== 'HEAD';
|
|
489
|
-
const response = await spool.handle(
|
|
490
|
-
new Request(request.url, {
|
|
491
|
-
method: request.method,
|
|
492
|
-
headers: [...request.headers],
|
|
493
|
-
body: hasBody ? await request.arrayBuffer() : undefined
|
|
494
|
-
}),
|
|
495
|
-
page
|
|
496
|
-
);
|
|
497
|
-
return {
|
|
498
|
-
status: response.status,
|
|
499
|
-
headers: [...response.headers],
|
|
500
|
-
body: await response.arrayBuffer()
|
|
501
|
-
};
|
|
502
|
-
}
|
|
503
|
-
|
|
504
|
-
app.http('spool', {
|
|
505
|
-
route: '{*path}',
|
|
506
|
-
methods: ['GET', 'POST'],
|
|
507
|
-
authLevel: 'anonymous',
|
|
508
|
-
handler
|
|
509
|
-
});
|
|
510
|
-
```
|
|
511
|
-
|
|
512
|
-
### Node
|
|
513
|
-
|
|
514
|
-
`toNodeListener` turns a Fetch-style handler into a `node:http` listener. Its `toRequest` and
|
|
515
|
-
`writeResponse` parts are what the frameworks below use to cross between Node and Fetch.
|
|
516
|
-
|
|
517
|
-
```ts
|
|
518
|
-
import { createServer } from 'node:http';
|
|
519
|
-
import { toNodeListener } from '@drupflare/spool/node';
|
|
520
|
-
import { page } from './routes';
|
|
521
|
-
import { spool } from './spool';
|
|
522
|
-
|
|
523
|
-
export const server = createServer(toNodeListener((request) => spool.handle(request, page)));
|
|
524
|
-
```
|
|
525
|
-
|
|
526
|
-
### Express
|
|
527
|
-
|
|
528
|
-
Express responds through its own `res`, so it has no Fetch response to wrap. `spool.respond`
|
|
529
|
-
answers `/.spool/*` and returns `undefined` for everything else, which sends the request on to the
|
|
530
|
-
next Express handler. A route passes its page through `spool.decorate` to get the envelope and the
|
|
531
|
-
script tag.
|
|
532
|
-
|
|
533
|
-
```ts
|
|
534
|
-
import express from 'express';
|
|
535
|
-
import { toNodeListener, toRequest, writeResponse } from '@drupflare/spool/node';
|
|
536
|
-
import { page } from './routes';
|
|
537
|
-
import { spool } from './spool';
|
|
538
|
-
|
|
539
|
-
export const app = express();
|
|
540
|
-
|
|
541
|
-
app.use(toNodeListener((request) => spool.respond(request)));
|
|
542
|
-
app.get('/spool-sw.js', (req, res) => writeResponse(res, spool.stub()));
|
|
543
|
-
app.get('/node/:id', async (req, res) => {
|
|
544
|
-
const request = toRequest(req, res);
|
|
545
|
-
await writeResponse(res, await spool.decorate(request, page(request)));
|
|
546
|
-
});
|
|
547
|
-
```
|
|
548
|
-
|
|
549
|
-
### Fastify
|
|
550
|
-
|
|
551
|
-
Fastify's `reply.send` takes a Fetch `Response`. An `onRequest` hook answers the spool paths from
|
|
552
|
-
the raw request before Fastify reads the body.
|
|
553
|
-
|
|
554
|
-
```ts
|
|
555
|
-
import Fastify from 'fastify';
|
|
556
|
-
import { toRequest } from '@drupflare/spool/node';
|
|
557
|
-
import { page } from './routes';
|
|
558
|
-
import { spool } from './spool';
|
|
559
|
-
|
|
560
|
-
export const app = Fastify();
|
|
561
|
-
|
|
562
|
-
app.addHook('onRequest', async (req, reply) => {
|
|
563
|
-
const response = await spool.respond(toRequest(req.raw, reply.raw));
|
|
564
|
-
if (response) return reply.send(response);
|
|
565
|
-
});
|
|
566
|
-
app.get('/spool-sw.js', (req, reply) => reply.send(spool.stub()));
|
|
567
|
-
app.get('/node/:id', async (req, reply) => {
|
|
568
|
-
const request = toRequest(req.raw, reply.raw);
|
|
569
|
-
return reply.send(await spool.decorate(request, page(request)));
|
|
570
|
-
});
|
|
571
|
-
```
|
|
572
|
-
|
|
573
|
-
### Koa
|
|
574
|
-
|
|
575
|
-
Koa exposes the Node objects as `ctx.req` and `ctx.res`. Setting `ctx.respond` to `false` stops
|
|
576
|
-
Koa from writing a second response.
|
|
577
|
-
|
|
578
|
-
```ts
|
|
579
|
-
import Koa from 'koa';
|
|
580
|
-
import { toRequest, writeResponse } from '@drupflare/spool/node';
|
|
581
|
-
import { page } from './routes';
|
|
582
|
-
import { spool } from './spool';
|
|
583
|
-
|
|
584
|
-
export const app = new Koa();
|
|
585
|
-
|
|
586
|
-
app.use(async (ctx, next) => {
|
|
587
|
-
const request = toRequest(ctx.req, ctx.res);
|
|
588
|
-
const response = ctx.path === '/spool-sw.js' ? spool.stub() : await spool.respond(request);
|
|
589
|
-
if (!response) return next();
|
|
590
|
-
ctx.respond = false;
|
|
591
|
-
await writeResponse(ctx.res, response);
|
|
592
|
-
});
|
|
593
|
-
app.use(async (ctx) => {
|
|
594
|
-
if (!ctx.path.startsWith('/node/')) return;
|
|
595
|
-
const request = toRequest(ctx.req, ctx.res);
|
|
596
|
-
ctx.respond = false;
|
|
597
|
-
await writeResponse(ctx.res, await spool.decorate(request, page(request)));
|
|
598
|
-
});
|
|
599
|
-
```
|
|
600
|
-
|
|
601
|
-
### hapi
|
|
602
|
-
|
|
603
|
-
hapi keeps the Node objects at `request.raw`. Returning `h.abandon` tells it the response is
|
|
604
|
-
already written.
|
|
605
|
-
|
|
606
|
-
```ts
|
|
607
|
-
import Hapi from '@hapi/hapi';
|
|
608
|
-
import { toRequest, writeResponse } from '@drupflare/spool/node';
|
|
609
|
-
import { page } from './routes';
|
|
610
|
-
import { spool } from './spool';
|
|
611
|
-
|
|
612
|
-
export const server = Hapi.server();
|
|
613
|
-
|
|
614
|
-
server.ext('onRequest', async (request, h) => {
|
|
615
|
-
const { req, res } = request.raw;
|
|
616
|
-
const spoolRequest = toRequest(req, res);
|
|
617
|
-
const response =
|
|
618
|
-
request.path === '/spool-sw.js' ? spool.stub() : await spool.respond(spoolRequest);
|
|
619
|
-
if (!response) return h.continue;
|
|
620
|
-
await writeResponse(res, response);
|
|
621
|
-
return h.abandon;
|
|
622
|
-
});
|
|
623
|
-
server.route({
|
|
624
|
-
method: 'GET',
|
|
625
|
-
path: '/node/{id}',
|
|
626
|
-
handler: async (request, h) => {
|
|
627
|
-
const { req, res } = request.raw;
|
|
628
|
-
const spoolRequest = toRequest(req, res);
|
|
629
|
-
await writeResponse(res, await spool.decorate(spoolRequest, page(spoolRequest)));
|
|
630
|
-
return h.abandon;
|
|
631
|
-
}
|
|
632
|
-
});
|
|
269
|
+
```html
|
|
270
|
+
<body>
|
|
271
|
+
<header>
|
|
272
|
+
<a href="/">Site</a>
|
|
273
|
+
</header>
|
|
274
|
+
<main data-spool-swap>
|
|
275
|
+
<h1>Course 101</h1>
|
|
276
|
+
</main>
|
|
277
|
+
</body>
|
|
633
278
|
```
|
|
634
279
|
|
|
635
|
-
|
|
636
|
-
|
|
637
|
-
On the Express adapter, `app.use` takes spool's listener and a controller method that injects
|
|
638
|
-
`@Req()` and `@Res()` writes the page. The example needs `experimentalDecorators` in
|
|
639
|
-
`tsconfig.json`.
|
|
640
|
-
|
|
641
|
-
```ts
|
|
642
|
-
import { Controller, Get, Module, Req, Res } from '@nestjs/common';
|
|
643
|
-
import { NestFactory } from '@nestjs/core';
|
|
644
|
-
import type { Request, Response } from 'express';
|
|
645
|
-
import 'reflect-metadata';
|
|
646
|
-
import { toNodeListener, toRequest, writeResponse } from '@drupflare/spool/node';
|
|
647
|
-
import { page } from './routes';
|
|
648
|
-
import { spool } from './spool';
|
|
649
|
-
|
|
650
|
-
@Controller()
|
|
651
|
-
class NodeController {
|
|
652
|
-
@Get('node/:id')
|
|
653
|
-
async show(@Req() req: Request, @Res() res: Response) {
|
|
654
|
-
const request = toRequest(req, res);
|
|
655
|
-
await writeResponse(res, await spool.decorate(request, page(request)));
|
|
656
|
-
}
|
|
657
|
-
}
|
|
280
|
+
The page replaces the region, the element marked `data-spool-swap-sidebar` when both pages have one, and the title. It answers only public routes declared with `family`, from the rows the device holds ([Dataset Routes](ADVANCED_USAGE.md#dataset-routes)). Modified clicks, links with `target` or `download`, cross-origin links and non-GET forms go to the browser, and so does any route the device does not hold. The page writes at most 90 history entries in any 10 seconds. Past that count it swaps the content and leaves the address bar on the previous URL until the window clears.
|
|
658
281
|
|
|
659
|
-
|
|
660
|
-
class AppModule {}
|
|
282
|
+
A page with no swap element navigates by document, and no error is shown.
|
|
661
283
|
|
|
662
|
-
|
|
663
|
-
const app = await NestFactory.create(AppModule, { logger: false });
|
|
664
|
-
app.use(toNodeListener((request) => spool.respond(request)));
|
|
665
|
-
app.use('/spool-sw.js', (req: Request, res: Response) => writeResponse(res, spool.stub()));
|
|
666
|
-
return app;
|
|
667
|
-
}
|
|
668
|
-
```
|
|
669
|
-
|
|
670
|
-
### Google Cloud Functions
|
|
284
|
+
---
|
|
671
285
|
|
|
672
|
-
|
|
673
|
-
|
|
674
|
-
|
|
675
|
-
|
|
676
|
-
|
|
677
|
-
|
|
678
|
-
|
|
679
|
-
|
|
680
|
-
|
|
681
|
-
|
|
682
|
-
|
|
683
|
-
|
|
684
|
-
)
|
|
685
|
-
|
|
286
|
+
## Runtimes
|
|
287
|
+
|
|
288
|
+
Each runtime has its own page, with the framework context and a minimal example. The
|
|
289
|
+
[Adapters](docs/adapters/README.md) index lists every page with its import path.
|
|
290
|
+
|
|
291
|
+
| Runtime | Page |
|
|
292
|
+
| -------------------------- | ---------------------------------------------------------------------------- |
|
|
293
|
+
| Cloudflare Workers | [cloudflare-workers.md](docs/adapters/cloudflare-workers.md) |
|
|
294
|
+
| Cloudflare Pages Functions | [cloudflare-pages-functions.md](docs/adapters/cloudflare-pages-functions.md) |
|
|
295
|
+
| Hono | [hono.md](docs/adapters/hono.md) |
|
|
296
|
+
| Elysia | [elysia.md](docs/adapters/elysia.md) |
|
|
297
|
+
| itty-router | [itty-router.md](docs/adapters/itty-router.md) |
|
|
298
|
+
| Bun | [bun.md](docs/adapters/bun.md) |
|
|
299
|
+
| Deno | [deno.md](docs/adapters/deno.md) |
|
|
300
|
+
| Netlify Functions | [netlify-functions.md](docs/adapters/netlify-functions.md) |
|
|
301
|
+
| Netlify Edge Functions | [netlify-edge-functions.md](docs/adapters/netlify-edge-functions.md) |
|
|
302
|
+
| Vercel Functions | [vercel-functions.md](docs/adapters/vercel-functions.md) |
|
|
303
|
+
| Next.js | [nextjs.md](docs/adapters/nextjs.md) |
|
|
304
|
+
| Nuxt and Nitro | [nuxt-and-nitro.md](docs/adapters/nuxt-and-nitro.md) |
|
|
305
|
+
| Nuxt Module | [nuxt-module.md](docs/adapters/nuxt-module.md) |
|
|
306
|
+
| React Router | [react-router.md](docs/adapters/react-router.md) |
|
|
307
|
+
| Next Adapter | [next-adapter.md](docs/adapters/next-adapter.md) |
|
|
308
|
+
| Vite | [vite.md](docs/adapters/vite.md) |
|
|
309
|
+
| Vue | [vue.md](docs/adapters/vue.md) |
|
|
310
|
+
| Angular | [angular.md](docs/adapters/angular.md) |
|
|
311
|
+
| AWS Lambda | [aws-lambda.md](docs/adapters/aws-lambda.md) |
|
|
312
|
+
| Azure Functions | [azure-functions.md](docs/adapters/azure-functions.md) |
|
|
313
|
+
| Node | [node.md](docs/adapters/node.md) |
|
|
314
|
+
| Express | [express.md](docs/adapters/express.md) |
|
|
315
|
+
| Fastify | [fastify.md](docs/adapters/fastify.md) |
|
|
316
|
+
| Koa | [koa.md](docs/adapters/koa.md) |
|
|
317
|
+
| hapi | [hapi.md](docs/adapters/hapi.md) |
|
|
318
|
+
| NestJS | [nestjs.md](docs/adapters/nestjs.md) |
|
|
319
|
+
| Google Cloud Functions | [google-cloud-functions.md](docs/adapters/google-cloud-functions.md) |
|
|
320
|
+
|
|
321
|
+
Pick the page for your framework. A runtime with a Fetch `fetch(request)` handler needs only
|
|
322
|
+
`spool.handle`, as the [Quick Start](#quick-start) shows.
|
|
686
323
|
|
|
687
324
|
---
|
|
688
325
|
|
|
689
|
-
##
|
|
326
|
+
## Compatibility
|
|
690
327
|
|
|
691
328
|
The gate column says how far each row is checked. **runs** means the fixture above drives the
|
|
692
329
|
framework's own package. **stand-in** means a fixture calls the platform's handler with a plain
|
|
@@ -695,14 +332,14 @@ from the framework's documentation and nothing here has run it.
|
|
|
695
332
|
|
|
696
333
|
| Framework or host | What a handler receives | Spool entry | Gate |
|
|
697
334
|
| ------------------------------------ | -------------------------------------------------------------- | ------------------------------------------------------------ | -------------------------------------------- |
|
|
698
|
-
| Express | Node `req`, `res` | `
|
|
335
|
+
| Express | Node `req`, `res` | `spoolExpress` | runs |
|
|
699
336
|
| Fastify | Node `req.raw`, `reply.raw`; `reply.send` takes a `Response` | `toRequest`, `respond`, `decorate` | runs |
|
|
700
337
|
| Koa | Node `ctx.req`, `ctx.res` | `toRequest`, `writeResponse` | runs |
|
|
701
338
|
| hapi | Node `request.raw.req`, `request.raw.res` | `toRequest`, `writeResponse` | runs |
|
|
702
339
|
| NestJS, Express adapter | Node, through Express | `app.use` with `toNodeListener`; `toRequest` in a controller | runs |
|
|
703
340
|
| NestJS, Fastify adapter | Node, through Fastify | the Fastify entry | not tested |
|
|
704
341
|
| AdonisJS | Node at `ctx.request.request` and `ctx.response.response` | `toRequest`, `writeResponse` in a middleware | not tested |
|
|
705
|
-
| Hono | Fetch | `
|
|
342
|
+
| Hono | Fetch | `spoolHono` | runs |
|
|
706
343
|
| Elysia | Fetch | `respond`, `decorate` | runs |
|
|
707
344
|
| itty-router | Fetch | `respond`, `decorate` | runs |
|
|
708
345
|
| Oak | Fetch, through `app.handle`; published on JSR only | `handle` | not tested |
|
|
@@ -722,8 +359,8 @@ from the framework's documentation and nothing here has run it.
|
|
|
722
359
|
| Deno, Deno Deploy | Fetch, through `Deno.serve` or `deno serve` | `handle` | stand-in |
|
|
723
360
|
| Bun | Fetch | `handle` | runs, under `Bun.serve` |
|
|
724
361
|
| Vercel Functions | Fetch | `handle` | stand-in |
|
|
725
|
-
| AWS Lambda, Function URL or HTTP API | JSON event version 2.0 |
|
|
726
|
-
| AWS Lambda, API Gateway REST API | JSON event version 1.0 |
|
|
362
|
+
| AWS Lambda, Function URL or HTTP API | JSON event version 2.0 | `@hono/aws-lambda` around a Hono app | stand-in event |
|
|
363
|
+
| AWS Lambda, API Gateway REST API | JSON event version 1.0 | `@hono/aws-lambda` around a Hono app | not tested |
|
|
727
364
|
| Lambda@Edge | CloudFront event with header arrays | none; a converter to `Request` is needed | not tested |
|
|
728
365
|
| Fastly Compute | Fetch, through `event.request` in a `fetch` listener | `event.respondWith(spool.handle(event.request, next))` | not tested |
|
|
729
366
|
| Azure Functions v4 | `HttpRequest`, which follows Fetch; returns `HttpResponseInit` | convert, then `handle` | runs, with a real `HttpRequest` |
|
|
@@ -731,11 +368,11 @@ from the framework's documentation and nothing here has run it.
|
|
|
731
368
|
| Google Cloud Run | any HTTP server on `PORT` | the Node example | not tested |
|
|
732
369
|
|
|
733
370
|
A host that serves static files and runs no code, such as an object store or a plain web server, uses
|
|
734
|
-
[Static Sites](
|
|
371
|
+
[Static Sites](#static-sites).
|
|
735
372
|
|
|
736
373
|
---
|
|
737
374
|
|
|
738
|
-
##
|
|
375
|
+
## Static Sites
|
|
739
376
|
|
|
740
377
|
A site with no spool server can still answer repeat views from the device. It serves three files
|
|
741
378
|
from one directory:
|
|
@@ -744,7 +381,7 @@ from one directory:
|
|
|
744
381
|
- `client.js`, the page script, from the same place.
|
|
745
382
|
- `spool.json`, the list of files the worker may keep.
|
|
746
383
|
|
|
747
|
-
Each page loads the script with the tag from [CDN Script](
|
|
384
|
+
Each page loads the script with the tag from [CDN Script](#cdn-script), or from the site's own copy.
|
|
748
385
|
The worker has no server to talk to, so it reads `spool.json` from the directory its own script
|
|
749
386
|
sits in.
|
|
750
387
|
|
|
@@ -796,1133 +433,137 @@ importScripts('https://cdn.drupflare.com/spool/<version>/spool-sw.js');
|
|
|
796
433
|
|
|
797
434
|
`importScripts` has no integrity check, so this form trusts the CDN host with the worker.
|
|
798
435
|
|
|
799
|
-
|
|
800
|
-
|
|
801
|
-
## ๐ Strategies
|
|
802
|
-
|
|
803
|
-
A route is answered locally in one of five ways. A `template` route stores JSON and renders it
|
|
804
|
-
through a compiled template. An `assembly` route stores a shell with named holes, fragment markup
|
|
805
|
-
for the holes and values for named slots. A `static` route stores a file as the server sent it. A
|
|
806
|
-
`data` route stores JSON and answers a GET for it. A `projection` route stores JSON and renders a
|
|
807
|
-
page from it with a function your site bundles into its own worker.
|
|
808
|
-
|
|
809
|
-
```ts
|
|
810
|
-
import { placeholder, slot } from '@drupflare/spool/protocol';
|
|
811
|
-
import { defineRoute } from '@drupflare/spool/server';
|
|
812
|
-
|
|
813
|
-
export const about = defineRoute('/about', {
|
|
814
|
-
strategy: 'assembly',
|
|
815
|
-
depends: () => ['menu'],
|
|
816
|
-
load: () => ({
|
|
817
|
-
shell: `<!doctype html><html><head><title>About</title></head><body>${placeholder('menu')}<main>Updated ${slot('updated')}</main></body></html>`,
|
|
818
|
-
fragments: { menu: '<nav><a href="/">Home</a></nav>' },
|
|
819
|
-
slots: { updated: '2026-10-19' }
|
|
820
|
-
})
|
|
821
|
-
});
|
|
822
|
-
```
|
|
823
|
-
|
|
824
|
-
The worker leaves a hole it cannot fill in place and refuses to render a page with a gap, so the
|
|
825
|
-
request goes to the network instead.
|
|
826
|
-
|
|
827
|
-
A page the worker renders keeps the site's security headers. The worker remembers
|
|
828
|
-
`Content-Security-Policy` (and its report-only form), `X-Frame-Options`, `X-Content-Type-Options`,
|
|
829
|
-
`Referrer-Policy`, `Permissions-Policy` and the three `Cross-Origin-*` policies from the network
|
|
830
|
-
answer to each path, and sends them with every local render of it. A path with no answer of its own
|
|
831
|
-
uses the latest set from any navigation. A page with neither goes to the network until the worker
|
|
832
|
-
has seen one, so a local view never goes out without the site's policy.
|
|
833
|
-
|
|
834
|
-
### Records and Projections
|
|
835
|
-
|
|
836
|
-
Pages that read the same data share one stored copy. `ctx.record(id, data)` stores an
|
|
837
|
-
authoritative value once under its id, and `ctx.projection(id, data)` stores a value derived from
|
|
838
|
-
records. Both return a reference to put in the page's data. A record's id is its invalidation tag.
|
|
839
|
-
Invalidating `course:101` refreshes the records and projections that read it, and no others.
|
|
840
|
-
|
|
841
|
-
```ts
|
|
842
|
-
import { defineRoute } from '@drupflare/spool/server';
|
|
843
|
-
|
|
844
|
-
const courses: Record<string, string> = { '101': 'Intro', '102': 'Advanced' };
|
|
845
|
-
|
|
846
|
-
export const course = defineRoute('/courses/:id', {
|
|
847
|
-
strategy: 'template',
|
|
848
|
-
template: {
|
|
849
|
-
source: '<h1>{{title}}</h1><nav>{{#each menu}}<a>{{this}}</a>{{/each}}</nav>'
|
|
850
|
-
},
|
|
851
|
-
load: (ctx) => {
|
|
852
|
-
const id = ctx.params['id'] ?? '';
|
|
853
|
-
return {
|
|
854
|
-
title: ctx.record(`course:${id}`, courses[id]),
|
|
855
|
-
menu: ctx.projection(
|
|
856
|
-
'menu',
|
|
857
|
-
Object.keys(courses).map((key) => ctx.record(`course:${key}`, courses[key]))
|
|
858
|
-
)
|
|
859
|
-
};
|
|
860
|
-
}
|
|
861
|
-
});
|
|
862
|
-
```
|
|
863
|
-
|
|
864
|
-
The server sends only the records and projections the device does not already hold at a current
|
|
865
|
-
version. A sync for the next page transfers the entries that page reads and that changed.
|
|
866
|
-
|
|
867
|
-
### Templates
|
|
868
|
-
|
|
869
|
-
Templates use Mustache-shaped syntax with no expressions, so the worker needs no `eval`.
|
|
870
|
-
|
|
871
|
-
| Syntax | Meaning |
|
|
872
|
-
| ------------------------------------ | -------------------------------------------------------- |
|
|
873
|
-
| `{{field}}` | the field, HTML-escaped; `a.b` reaches into objects |
|
|
874
|
-
| `{{{field}}}` | the field as markup; allowed only for names in `trusted` |
|
|
875
|
-
| `{{#each list}}...{{/each}}` | repeats for each item; `{{this}}` and `{{@index}}` work |
|
|
876
|
-
| `{{#if field}}...{{else}}...{{/if}}` | renders a branch on whether the field is truthy |
|
|
877
|
-
| `{{! note }}` | a comment that renders nothing |
|
|
878
|
-
|
|
879
|
-
```ts
|
|
880
|
-
import { defineRoute } from '@drupflare/spool/server';
|
|
881
|
-
|
|
882
|
-
export const news = defineRoute('/news/:slug', {
|
|
883
|
-
strategy: 'template',
|
|
884
|
-
template: {
|
|
885
|
-
source:
|
|
886
|
-
'<article><h1>{{title}}</h1>{{#if byline}}<p>{{byline}}</p>{{else}}<p>Staff</p>{{/if}}<div>{{{body}}}</div><ul>{{#each tags}}<li>{{@index}} {{this}}</li>{{/each}}</ul></article>',
|
|
887
|
-
trusted: ['body']
|
|
888
|
-
},
|
|
889
|
-
depends: (ctx) => [`news:${ctx.params['slug']}`],
|
|
890
|
-
load: () => ({
|
|
891
|
-
title: 'Hello',
|
|
892
|
-
byline: null,
|
|
893
|
-
body: '<p>Trusted markup</p>',
|
|
894
|
-
tags: ['a', 'b']
|
|
895
|
-
})
|
|
896
|
-
});
|
|
897
|
-
```
|
|
898
|
-
|
|
899
|
-
A template that names a field it does not trust for markup fails when the route is declared.
|
|
900
|
-
|
|
901
|
-
A public `template` route can carry its own data in the page. `inlineMaxBytes` on `createSpool`
|
|
902
|
-
sets the largest artifact, in bytes, that `handle` and `decorate` write into the HTML as an
|
|
903
|
-
escaped `<script type="application/spool+json">` element. The worker reads it from the response
|
|
904
|
-
and skips the `/.spool/artifact` request. It is 0, which is off, because every network view of the
|
|
905
|
-
page carries the element, used or not. The page the visitor sees and the shadow comparison ignore
|
|
906
|
-
it.
|
|
907
|
-
|
|
908
|
-
### Static Files
|
|
909
|
-
|
|
910
|
-
A `static` route has no `load` and no template. The application serves the file as it always did,
|
|
911
|
-
and `handle` or `decorate` marks the response for the worker, which keeps the bytes. A tag in
|
|
912
|
-
`depends` ties the file to a deploy, and `spool.invalidate` after one makes devices fetch the
|
|
913
|
-
file again. A static route is always public and is always answered from the device, so it takes no
|
|
914
|
-
`scope`, `persist` or `mode`. A query string makes a different file: `/app.js?v=2` is not
|
|
915
|
-
answered with the copy of `/app.js`, and a query the worker has not stored goes to the network.
|
|
916
|
-
|
|
917
|
-
```ts
|
|
918
|
-
import { defineRoute } from '@drupflare/spool/server';
|
|
919
|
-
|
|
920
|
-
export const about = defineRoute('/about', {
|
|
921
|
-
strategy: 'static',
|
|
922
|
-
depends: () => ['site'],
|
|
923
|
-
freshness: { maxStale: 3600, mode: 'soft' }
|
|
924
|
-
});
|
|
925
|
-
```
|
|
926
|
-
|
|
927
|
-
### Projection and Data Routes
|
|
928
|
-
|
|
929
|
-
A `data` route answers a GET for JSON, such as an API read, from the stored value, so the page's
|
|
930
|
-
`fetch` makes no request. The worker answers with `JSON.stringify` of what `load` returned, which
|
|
931
|
-
means the application's own handler has to return the same JSON. A navigation to the URL still goes
|
|
932
|
-
to the network.
|
|
933
|
-
|
|
934
|
-
A `projection` route needs a renderer. Spool ships no bundler: write a worker with `serve` from
|
|
935
|
-
`@drupflare/spool/sw`, bundle it with your own toolchain, and pass the bundle to `createSpool` as
|
|
936
|
-
`worker`. The server puts the config in front of your bundle as `self.__SPOOL_CONFIG__`, and
|
|
937
|
-
`spool.config` is the same object as JSON if you would rather inline it at build time. A route with
|
|
938
|
-
no renderer is refused by the worker and the network answers.
|
|
939
|
-
|
|
940
|
-
```ts
|
|
941
|
-
import { defineRoute } from '@drupflare/spool/server';
|
|
942
|
-
|
|
943
|
-
export const course = defineRoute('/api/courses/:id', {
|
|
944
|
-
strategy: 'data',
|
|
945
|
-
depends: (ctx) => [`course:${ctx.params['id']}`],
|
|
946
|
-
load: (ctx) => ({ id: ctx.params['id'], title: 'Intro' }),
|
|
947
|
-
provision: { reuse: 0.8, futureViews: 6 }
|
|
948
|
-
});
|
|
949
|
-
|
|
950
|
-
export const roster = defineRoute('/rosters/:id', {
|
|
951
|
-
strategy: 'projection',
|
|
952
|
-
depends: (ctx) => [`roster:${ctx.params['id']}`],
|
|
953
|
-
load: (ctx) => ({ id: ctx.params['id'], names: ['Ada', 'Bo'] })
|
|
954
|
-
});
|
|
955
|
-
```
|
|
956
|
-
|
|
957
|
-
```ts
|
|
958
|
-
import { escapeHtml } from '@drupflare/spool/protocol';
|
|
959
|
-
import { serve } from '@drupflare/spool/sw';
|
|
960
|
-
|
|
961
|
-
declare const CONFIG: string;
|
|
962
|
-
|
|
963
|
-
serve(JSON.parse(CONFIG), {
|
|
964
|
-
'/rosters/:id': (artifact) => {
|
|
965
|
-
const { names } = (artifact.body as { data: { names: string[] } }).data;
|
|
966
|
-
return `<!doctype html><html><head><title>Roster</title></head><body><ul>${names.map((name) => `<li>${escapeHtml(name)}</li>`).join('')}</ul></body></html>`;
|
|
967
|
-
}
|
|
968
|
-
});
|
|
969
|
-
```
|
|
970
|
-
|
|
971
|
-
The server also decides whether to send an artifact at all. It weighs the bytes it would send,
|
|
972
|
-
records included, against what a local view saves: reuse times future views times the bytes a view
|
|
973
|
-
avoids, less the cost of the request. When that is not positive it answers `204` with
|
|
974
|
-
`Spool-Provision: declined`, and the worker leaves that path to the network until it restarts.
|
|
975
|
-
The priors are scenario assumptions: set them from your own telemetry in `provision`, or per route
|
|
976
|
-
as above. A `data` route weighs against its own size, so its reuse times future views must exceed
|
|
977
|
-
one. The defaults of 0.5 and 2 give exactly one, which breaks even, and the route is left out.
|
|
978
|
-
|
|
979
|
-
### Query Routes
|
|
980
|
-
|
|
981
|
-
A URL with a query string goes to the network. A public `template`, `projection` or `data` route
|
|
982
|
-
can declare the query strings it answers, and the device then answers a new filter, sort or page of
|
|
983
|
-
a collection it already holds with no request.
|
|
984
|
-
|
|
985
|
-
```ts
|
|
986
|
-
import { datasetProof, defineRoute } from '@drupflare/spool/server';
|
|
987
|
-
|
|
988
|
-
const people = [
|
|
989
|
-
{ name: 'Ada', department: 'math', age: 36 },
|
|
990
|
-
{ name: 'Bo', department: 'cs', age: 25 }
|
|
991
|
-
];
|
|
992
|
-
|
|
993
|
-
export const directory = defineRoute('/people', {
|
|
994
|
-
strategy: 'template',
|
|
995
|
-
template: {
|
|
996
|
-
source:
|
|
997
|
-
'<h1>{{#if query}}Page {{query.page}} of {{query.pages}}{{else}}Everyone{{/if}}</h1><ul>{{#each people}}<li>{{name}}</li>{{/each}}</ul>'
|
|
998
|
-
},
|
|
999
|
-
depends: () => ['people'],
|
|
1000
|
-
load: async () => ({ people, proof: await datasetProof(people) }),
|
|
1001
|
-
query: {
|
|
1002
|
-
list: 'people',
|
|
1003
|
-
filter: { department: 'string', age: 'number' },
|
|
1004
|
-
sort: ['name', 'age'],
|
|
1005
|
-
pageSize: 20
|
|
1006
|
-
},
|
|
1007
|
-
provision: { reuse: 0.8, futureViews: 6 }
|
|
1008
|
-
});
|
|
1009
|
-
```
|
|
1010
|
-
|
|
1011
|
-
`list` names the key of the loaded data that holds the whole collection, an array of flat objects.
|
|
1012
|
-
Each key of `filter` is a query parameter that keeps the items whose field of the same name equals
|
|
1013
|
-
it, read as a `string` or a `number`. A repeated parameter matches any of its values, and different
|
|
1014
|
-
parameters combine with and. `sort` lists the fields the `sort` parameter may name, and `-name`
|
|
1015
|
-
sorts descending. Ties keep the stored order. Strings compare by UTF-16 code unit, with no locale
|
|
1016
|
-
and no case folding, so the application's own sort has to agree. A database that compares bytes,
|
|
1017
|
-
such as SQLite by default, agrees except for characters above U+FFFF, and a case-insensitive or
|
|
1018
|
-
locale collation does not. `pageSize` sets the items per page and `page` counts from 1; without it
|
|
1019
|
-
the route is one page of everything. A route with `principal` scope, or any
|
|
1020
|
-
other strategy, cannot declare `query`.
|
|
1021
|
-
|
|
1022
|
-
`load` returns the array at `list` and a `proof` made by `datasetProof`. The server refuses to
|
|
1023
|
-
build the artifact when the proof is missing or does not match the array. The worker computes the
|
|
1024
|
-
proof again from the stored copy before it answers a query. The proof shows that the device holds
|
|
1025
|
-
the array the server sent. That the array is the whole collection is the route's declaration:
|
|
1026
|
-
`list` has to hold every item the application's own answer could draw from, or a filter returns
|
|
1027
|
-
fewer rows than the server would.
|
|
1028
|
-
|
|
1029
|
-
The template, or the renderer of a `projection` route, receives the stored data with the array
|
|
1030
|
-
replaced by the page's items and one added key, `query`, holding `{ total, page, pages }`. `total`
|
|
1031
|
-
counts the items after filtering. A `data` route answers with that same object as JSON. The URL
|
|
1032
|
-
without a query string renders the stored data as it is and has no `query` key, which is what
|
|
1033
|
-
`{{#if query}}` tests above.
|
|
1034
|
-
|
|
1035
|
-
The worker answers locally only when the artifact for the path is held and fresh under the route's
|
|
1036
|
-
freshness contract, the proof matches and the query is one the route declares. Everything else goes
|
|
1037
|
-
to the network as before: a parameter the route does not name, a number filter that is not a
|
|
1038
|
-
finite number, a `sort` given twice or not listed, a `page` that is not a positive integer or
|
|
1039
|
-
lies past the last page, and an item that is not an object. A copy that fails its proof is dropped
|
|
1040
|
-
and fetched whole again. The worker never stores anything under a URL with a query string, and in
|
|
1041
|
-
shadow mode it compares its render with the network answer for the same URL.
|
|
1042
|
-
|
|
1043
|
-
A query route's artifact is the whole collection, so it is large. The provisioning rule weighs it
|
|
1044
|
-
like any other artifact, and `provision` on the route sets the priors. A collection repays its
|
|
1045
|
-
transfer after `entryBreakEven(bytes, documentBytes)` local answers per device: a 40 KB collection
|
|
1046
|
-
that replaces 17 KB documents needs 2.3 (L2). Pages of one visit come from one held copy until a
|
|
1047
|
-
sync replaces it, so paging never mixes two versions on a device.
|
|
1048
|
-
|
|
1049
|
-
---
|
|
1050
|
-
|
|
1051
|
-
## โณ Freshness Contracts
|
|
1052
|
-
|
|
1053
|
-
Each route declares how long a stored page may go without validation. `maxStale` is in seconds.
|
|
1054
|
-
|
|
1055
|
-
- **hard:** once `maxStale` passes, the worker validates the page with the server before it
|
|
1056
|
-
shows it. Offline, the route's `offline` rule decides.
|
|
1057
|
-
- **soft:** the worker shows the stored page at once and syncs afterward. A soft bound is a
|
|
1058
|
-
stale-while-revalidate window and promises nothing about how fresh the shown page is.
|
|
1059
|
-
|
|
1060
|
-
Public routes default to 15 minutes soft, and routes with `scope: 'principal'` to five minutes
|
|
1061
|
-
hard; [Authenticated Data](#-authenticated-data) covers how their pages are stored. `offline`
|
|
1062
|
-
defaults to `serve` for public routes and `refuse` for principal routes. A soft route may set `sync: 'deferred'` to queue its refresh
|
|
1063
|
-
for the next network request that happens anyway.
|
|
1064
|
-
|
|
1065
|
-
```ts
|
|
1066
|
-
import { defineRoute } from '@drupflare/spool/server';
|
|
1067
|
-
|
|
1068
|
-
export const stock = defineRoute('/stock/:sku', {
|
|
1069
|
-
strategy: 'template',
|
|
1070
|
-
template: { source: '<p>{{sku}}: {{count}}</p>' },
|
|
1071
|
-
depends: (ctx) => [`stock:${ctx.params['sku']}`],
|
|
1072
|
-
load: (ctx) => ({ sku: ctx.params['sku'], count: 3 }),
|
|
1073
|
-
freshness: { maxStale: 60, mode: 'hard' },
|
|
1074
|
-
offline: 'refuse'
|
|
1075
|
-
});
|
|
1076
|
-
|
|
1077
|
-
export const notice = defineRoute('/notices/:id', {
|
|
1078
|
-
strategy: 'template',
|
|
1079
|
-
template: { source: '<p>{{text}}</p>' },
|
|
1080
|
-
depends: (ctx) => [`notice:${ctx.params['id']}`],
|
|
1081
|
-
load: () => ({ text: 'Closed on Friday' }),
|
|
1082
|
-
freshness: { maxStale: 3600, mode: 'soft' },
|
|
1083
|
-
sync: 'deferred'
|
|
1084
|
-
});
|
|
1085
|
-
```
|
|
1086
|
-
|
|
1087
|
-
The worker syncs the smallest set of changed dependencies the next page needs. One worker
|
|
1088
|
-
leads the sync for every tab of an origin. A response to a request that happens anyway can carry
|
|
1089
|
-
invalidations, so a sync does not always need a request of its own.
|
|
1090
|
-
|
|
1091
|
-
---
|
|
1092
|
-
|
|
1093
|
-
## ๐ Authenticated Data
|
|
1094
|
-
|
|
1095
|
-
Set `scope: 'principal'` on a route that depends on who is signed in, and give `createSpool` an
|
|
1096
|
-
`identity` option. The server sends `Spool-Partition`, an HMAC of the principal under your secret,
|
|
1097
|
-
so the browser can tell users apart without learning who they are. The worker keeps each partition
|
|
1098
|
-
in its own IndexedDB database with the artifacts, records, projections and sync state sealed with
|
|
1099
|
-
AES-GCM. Each value is bound to its partition, kind and id, so a row copied into another slot does
|
|
1100
|
-
not decrypt.
|
|
1101
|
-
|
|
1102
|
-
`retention` picks the key. `persistent` is a non-extractable key in IndexedDB that survives a
|
|
1103
|
-
restart and is deleted on logout or a change of partition. `session`, the default, is a key held in
|
|
1104
|
-
the worker's memory, which the browser discards when it stops the worker. Deleting a key makes what
|
|
1105
|
-
it sealed unreadable, which is crypto-shredding of data at rest. It does not protect against script
|
|
1106
|
-
running on your origin, which can ask the worker for whatever the page can. The ids and versions
|
|
1107
|
-
that serve as storage keys stay readable, so use opaque ids for anything sensitive.
|
|
1108
|
-
|
|
1109
|
-
A principal route defaults to a five-minute hard bound and `offline: 'refuse'`. Revocation rides
|
|
1110
|
-
the same bound: tag a page with the permission it needs, call `spool.invalidate` when the
|
|
1111
|
-
permission goes, and the first required sync past the bound finds the page invalid.
|
|
1112
|
-
|
|
1113
|
-
A different `Spool-Partition` wipes the partition before it. A login or account-switch handler has
|
|
1114
|
-
to send the new value itself with `spool.partition(principal)`, because the request it answers
|
|
1115
|
-
still carries the old identity. A sync answer carries `Spool-Partition` for the request's identity
|
|
1116
|
-
too, so a sign-in the worker never saw ends at the first required sync, no later than the route's
|
|
1117
|
-
`maxStale` plus the next view. Paths in `logout` are wiped when the request reaches the worker,
|
|
1118
|
-
before it is sent, so logout empties every tab even with the network down. The worker first sends
|
|
1119
|
-
the same-origin posts it holds in `beacons`, while they still carry the session that is ending,
|
|
1120
|
-
and drops any that do not get through. The page runtime
|
|
1121
|
-
dispatches a `spool:cleared` event on `window` in every tab when that happens, so a page can blank
|
|
1122
|
-
what it shows. A response can also send
|
|
1123
|
-
`Spool-Clear`: `*` wipes everything, a partition id wipes one. `decorate(request, response, { clear:
|
|
1124
|
-
['*'], hardReset: true })` adds `Clear-Site-Data: "storage"` as well, the browser's own reset
|
|
1125
|
-
of everything the origin holds, service worker included. Its Safari support is unverified, so rely
|
|
1126
|
-
on the worker's wipe.
|
|
1127
|
-
|
|
1128
|
-
```ts
|
|
1129
|
-
import { createSpool, defineRoute } from '@drupflare/spool/server';
|
|
1130
|
-
import { memoryStore } from '@drupflare/spool/stores';
|
|
1131
|
-
|
|
1132
|
-
export const grades = defineRoute('/grades', {
|
|
1133
|
-
strategy: 'template',
|
|
1134
|
-
scope: 'principal',
|
|
1135
|
-
retention: 'persistent',
|
|
1136
|
-
template: { source: '<h1>Grades for {{owner}}</h1>' },
|
|
1137
|
-
depends: (ctx) => [`perm:${ctx.identity?.principal}`, `grades:${ctx.identity?.principal}`],
|
|
1138
|
-
load: (ctx) => ({ owner: ctx.identity?.principal })
|
|
1139
|
-
});
|
|
1140
|
-
|
|
1141
|
-
export const spool = createSpool({
|
|
1142
|
-
routes: [grades],
|
|
1143
|
-
store: memoryStore(),
|
|
1144
|
-
identity: {
|
|
1145
|
-
secret: 'a long random value kept on the server',
|
|
1146
|
-
resolve: (request) => {
|
|
1147
|
-
const user = /user=(\w+)/.exec(request.headers.get('cookie') ?? '')?.[1];
|
|
1148
|
-
return user ? { principal: user } : null;
|
|
1149
|
-
}
|
|
1150
|
-
},
|
|
1151
|
-
logout: ['/logout']
|
|
1152
|
-
});
|
|
1153
|
-
|
|
1154
|
-
export async function login(user: string): Promise<Response> {
|
|
1155
|
-
return new Response('Signed in', {
|
|
1156
|
-
headers: {
|
|
1157
|
-
'set-cookie': `user=${user}; Path=/`,
|
|
1158
|
-
'spool-partition': await spool.partition(user)
|
|
1159
|
-
}
|
|
1160
|
-
});
|
|
1161
|
-
}
|
|
1162
|
-
```
|
|
1163
|
-
|
|
1164
|
-
A sync that asks for everything changed lists the tags of every change, so tags for principal data
|
|
1165
|
-
should be opaque too. The worker syncs a principal page with `need`, which names only the tags
|
|
1166
|
-
that page reads.
|
|
1167
|
-
|
|
1168
|
-
---
|
|
1169
|
-
|
|
1170
|
-
## ๐ Catalogue
|
|
1171
|
-
|
|
1172
|
-
A catalogue sends the data of pages a visitor may open next, so that opening one makes no
|
|
1173
|
-
request. An entry costs its bytes whether or not the page is opened, and an opened one saves a
|
|
1174
|
-
document request, so an entry pays only when enough visitors open it. The break-even share is
|
|
1175
|
-
`k * E / (k * D + o)`, where `E` is the entry's compressed bytes, `D` the bytes of the document
|
|
1176
|
-
it replaces, `k` the joules to move a byte and `o` the origin request a view avoids. An entry the
|
|
1177
|
-
size of the document needs 96% of visitors to open it, which no page gets. An entry that is only
|
|
1178
|
-
the data for a template the device already holds is a few hundred bytes to a few kilobytes and
|
|
1179
|
-
pays at a few percent. On the `payload` reading, the rendered fields of a node were a median 473
|
|
1180
|
-
bytes compressed on 19 pages from two sites (L1, `measurements/payload`), against a document of
|
|
1181
|
-
about 17,000 bytes. References such as media and terms are left out of that figure, so a real entry
|
|
1182
|
-
is larger. An entry is data for a `template`, `projection` or `data` route and never HTML.
|
|
1183
|
-
|
|
1184
|
-
The site supplies the candidates. A route's `catalogue` returns the pages a visitor on it may open
|
|
1185
|
-
next, each with the share expected to. Take the share from your own `nextShare` over the sink
|
|
1186
|
-
(each `spool.view` names the route the visitor came from), from link position, or from a fixed
|
|
1187
|
-
prior. Pass `{ first: true }` to `nextShare` to count only first views, which is what an entry
|
|
1188
|
-
saves: the share over all views ran 1.5 to 7 times high in the bins that decide an entry in a replay
|
|
1189
|
-
of public visitors ([docs/impact.md](docs/impact.md#next-page-prediction)). `documentBytes` on the entry's route says what a hit saves, and the provisioning prior is
|
|
1190
|
-
used when it is absent.
|
|
1191
|
-
|
|
1192
|
-
A share taken from counts can carry `n`, the observations behind it. `catalogueConfidence` on
|
|
1193
|
-
`createSpool` is a number of standard deviations, and with it above 0 an entry whose guess has an
|
|
1194
|
-
`n` is judged on the lower confidence bound of its `p`, so a share seen on 5 visits stops counting
|
|
1195
|
-
as one seen on 5,000. It defaults to 0, which judges every guess on its `p`. `NavModel` in
|
|
1196
|
-
`@drupflare/spool/protocol` keeps first-order counts with decay and a prior and returns a `p` with
|
|
1197
|
-
the weight to pass as `n`. In a replay of 989,818 public visitors, a model like that earned more
|
|
1198
|
-
per provisioned kilobyte than the most popular page
|
|
1199
|
-
([docs/impact.md](docs/impact.md#next-page-prediction)).
|
|
1200
|
-
|
|
1201
|
-
```ts
|
|
1202
|
-
import { createSpool, defineRoute } from '@drupflare/spool/server';
|
|
1203
|
-
import { memoryStore } from '@drupflare/spool/stores';
|
|
1204
|
-
|
|
1205
|
-
export const home = defineRoute('/', {
|
|
1206
|
-
strategy: 'template',
|
|
1207
|
-
template: { source: '<h1>Courses</h1>' },
|
|
1208
|
-
load: () => ({}),
|
|
1209
|
-
catalogue: () => [
|
|
1210
|
-
{ path: '/courses/101', p: 0.3 },
|
|
1211
|
-
{ path: '/courses/102', p: 0.1 }
|
|
1212
|
-
]
|
|
1213
|
-
});
|
|
1214
|
-
|
|
1215
|
-
export const course = defineRoute('/courses/:id', {
|
|
1216
|
-
strategy: 'template',
|
|
1217
|
-
template: { source: '<h1>{{title}}</h1>' },
|
|
1218
|
-
depends: (ctx) => [`course:${ctx.params['id']}`],
|
|
1219
|
-
load: (ctx) => ({ title: `Course ${ctx.params['id']}` }),
|
|
1220
|
-
documentBytes: 17_000
|
|
1221
|
-
});
|
|
1222
|
-
|
|
1223
|
-
export const spool = createSpool({
|
|
1224
|
-
routes: [home, course],
|
|
1225
|
-
store: memoryStore(),
|
|
1226
|
-
catalogueBytes: 16_384
|
|
1227
|
-
});
|
|
1228
|
-
```
|
|
1229
|
-
|
|
1230
|
-
The server sends an entry only when its share reaches its own break-even, best expected net first,
|
|
1231
|
-
until `catalogueBytes` is spent. The cap counts compressed bytes and defaults to 32 KiB. It builds
|
|
1232
|
-
at most 32 pages for one request, measures each entry with Brotli where the runtime has it and
|
|
1233
|
-
gzip where it does not, and skips a `static` or `assembly` route. A request with `Save-Data: on`
|
|
1234
|
-
gets no entries.
|
|
1235
|
-
|
|
1236
|
-
The ask rides the beacon batch the worker already sends. A due ask also makes the next network
|
|
1237
|
-
moment send that batch, when a view is held to carry it, so a visit of two or three pages gets its
|
|
1238
|
-
entries without waiting for 20 views or an hour. That costs one `POST /.spool/beacon` for each
|
|
1239
|
-
catalogue page, device and day, sent at a request that happens anyway. A local view never sends
|
|
1240
|
-
it, and a moment with nothing held sends nothing. The worker stores the entries like provisioned
|
|
1241
|
-
pages, and the first view of an unopened one is reported with the reason `catalogue`. `catalogueFigures` reads the views and the worker's ledger for the entries sent
|
|
1242
|
-
and used, the hit share, and the bytes sent against the bytes saved. Until shadow views measure
|
|
1243
|
-
the shares, the saving is a scenario.
|
|
1244
|
-
|
|
1245
|
-
A hit saves the document request and nothing else. On the Drupal pages measured (75 first views of
|
|
1246
|
-
internal pages on 10 sites, L1, `measurements/trim`), a view of a page still fetched that page's
|
|
1247
|
-
new images and stylesheet and script aggregates: only 3% of those first views moved nothing but
|
|
1248
|
-
the document. A catalogue view takes the document off the wire, and the rest of the page is
|
|
1249
|
-
the site's own HTTP cache and [asset trimming](#-trimmed-stylesheets).
|
|
1250
|
-
|
|
1251
|
-
A `principal` route can be an entry. The worker then names every catalogue entry no page has
|
|
1252
|
-
opened at each sync of a principal page, so a revoked one is dropped at the first required sync.
|
|
1253
|
-
The request that asks for a catalogue names the path of the page the visitor is on, including one
|
|
1254
|
-
the worker showed locally, so a route that should not tell the server leaves `catalogue` off.
|
|
1255
|
-
[docs/security.md](docs/security.md#catalogue-entries) covers both.
|
|
1256
|
-
|
|
1257
|
-
---
|
|
1258
|
-
|
|
1259
|
-
## ๐ก Analytics Batching
|
|
1260
|
-
|
|
1261
|
-
A page that keeps analytics scripts posting to an endpoint makes a request on every view, which
|
|
1262
|
-
defeats a local view. List the same-origin POST paths in `beacons`. The worker answers each post
|
|
1263
|
-
with a `204` and queues it in IndexedDB. At the next network request that happens anyway it sends
|
|
1264
|
-
the post to the URL the page meant, with its method, content type, query and body, so the vendor
|
|
1265
|
-
receives what the page sent. Spool's own events go to `/.spool/beacon` in one batch, where a
|
|
1266
|
-
`sink` receives them. A held post goes out on its own at most once every 24 hours, when the oldest
|
|
1267
|
-
is a day old. A body over 64 KiB, counted in bytes, or a binary body goes to the network at once.
|
|
1268
|
-
Firefox does not route `fetch` calls made with `keepalive: true` through the worker, so a script
|
|
1269
|
-
that posts that way is not queued there.
|
|
1270
|
-
|
|
1271
|
-
```ts
|
|
1272
|
-
import { createSpool, jsonLinesSink } from '@drupflare/spool/server';
|
|
1273
|
-
import { memoryStore } from '@drupflare/spool/stores';
|
|
1274
|
-
import { routes } from './routes';
|
|
1275
|
-
|
|
1276
|
-
export const spool = createSpool({
|
|
1277
|
-
routes,
|
|
1278
|
-
store: memoryStore(),
|
|
1279
|
-
beacons: ['/cdn-cgi/rum'],
|
|
1280
|
-
sink: jsonLinesSink((line) => console.log(line))
|
|
1281
|
-
});
|
|
1282
|
-
```
|
|
1283
|
-
|
|
1284
|
-
`jsonLinesSink` writes one JSON line per event. `analyticsEngineSink` writes to a Workers
|
|
1285
|
-
Analytics Engine dataset, splitting a long body across data points so nothing is cut off, and
|
|
1286
|
-
`memorySink` keeps events for tests. The worker also queues one
|
|
1287
|
-
`spool.view` event per page view with the tier, the reason, the staleness and the route the
|
|
1288
|
-
visitor came from (a pattern or `external`, never a URL). `viewsOf` reads
|
|
1289
|
-
those events back. Set `views: false` to turn that off. `workSeries` records how long each route's
|
|
1290
|
-
handler took, for peak-work reports.
|
|
1291
|
-
|
|
1292
|
-
### Collectors
|
|
1293
|
-
|
|
1294
|
-
A `collectors` entry holds requests to another origin, such as an analytics host. The page gets an
|
|
1295
|
-
empty answer at once and the request leaves at a network moment, inside `maxAgeMs`. An event past
|
|
1296
|
-
the cap is dropped, never sent late, and each drop is counted and written to the sink as
|
|
1297
|
-
`/.spool/dropped`. With `delivery: 'replay'` the worker sends the held request to the vendor with no
|
|
1298
|
-
cookies, so identity has to be in the payload, and `stamp` writes the event's time into a query
|
|
1299
|
-
parameter for a vendor that takes one. With `delivery: 'sink'` the server half forwards it.
|
|
1300
|
-
|
|
1301
|
-
GA4's browser collector takes no backdating. The Measurement Protocol accepts events with their
|
|
1302
|
-
own timestamps up to 72 hours old and wants them within 48 hours to attribute them, and it needs an
|
|
1303
|
-
API secret the page must not hold. `GA4_COLLECTOR` holds the `g/collect` hit for 48 hours and
|
|
1304
|
-
`ga4Sink` sends it as a Measurement Protocol request carrying the original `timestamp_micros`.
|
|
1305
|
-
`ga4Sink` checks its requests against the rules spool knows (`mpViolations`); set `debug: true` to
|
|
1306
|
-
send them to `/debug/mp/collect` with your own measurement id and secret. Spool holds events
|
|
1307
|
-
whether or not the visitor consented, so gate the analytics tag itself, and ask your counsel
|
|
1308
|
-
whether holding an event in IndexedDB needs consent where you operate. Programmatic ads are never
|
|
1309
|
-
held, replayed or answered locally.
|
|
1310
|
-
|
|
1311
|
-
```ts
|
|
1312
|
-
import { GA4_COLLECTOR } from '@drupflare/spool/protocol';
|
|
1313
|
-
import { combineSinks, createSpool, ga4Sink, jsonLinesSink } from '@drupflare/spool/server';
|
|
1314
|
-
import { memoryStore } from '@drupflare/spool/stores';
|
|
1315
|
-
import { routes } from './routes';
|
|
1316
|
-
|
|
1317
|
-
export const spool = createSpool({
|
|
1318
|
-
routes,
|
|
1319
|
-
store: memoryStore(),
|
|
1320
|
-
collectors: [
|
|
1321
|
-
GA4_COLLECTOR,
|
|
1322
|
-
{
|
|
1323
|
-
vendor: 'matomo',
|
|
1324
|
-
match: 'https://stats.example.edu/matomo.php',
|
|
1325
|
-
maxAgeMs: 23 * 3_600_000,
|
|
1326
|
-
delivery: 'replay',
|
|
1327
|
-
stamp: { param: 'cdt', kind: 'datetime-utc' }
|
|
1328
|
-
}
|
|
1329
|
-
],
|
|
1330
|
-
sink: combineSinks(
|
|
1331
|
-
jsonLinesSink((line) => console.log(line)),
|
|
1332
|
-
ga4Sink({ measurementId: 'G-XXXXXXXXXX', apiSecret: 'kept on the server' })
|
|
1333
|
-
)
|
|
1334
|
-
});
|
|
1335
|
-
```
|
|
1336
|
-
|
|
1337
|
-
---
|
|
1338
|
-
|
|
1339
|
-
## ๐งท Third-Party Files
|
|
1340
|
-
|
|
1341
|
-
A view that loads a file from another origin is not zero-network. The page reports, when it is
|
|
1342
|
-
hidden, how many cross-origin resources it loaded and from which origins (origins only, never
|
|
1343
|
-
paths). `thirdPartyReport` reads those reports from `viewsOf` and ranks the origins by the views
|
|
1344
|
-
each one alone blocks, with a class and a remedy from a short table you can replace. The counts
|
|
1345
|
-
are an upper bound on network requests, since a cross-origin timing entry says nothing about cache
|
|
1346
|
-
hits without `Timing-Allow-Origin`.
|
|
1347
|
-
|
|
1348
|
-
The opt-in `thirdParty` cache removes the request for files that name their own bytes. List the
|
|
1349
|
-
origins in `allow`. The worker learns a file from the first view and keeps it only when the URL
|
|
1350
|
-
carries a hash, a version or a `/v12/` segment, or the origin marks it immutable. A file versioned
|
|
1351
|
-
only by a query parameter is checked again at a network moment once it is a day old. It keeps
|
|
1352
|
-
nothing else, and nothing it cannot read: an opaque response counts about 7 MB against quota in
|
|
1353
|
-
Chromium, so `allowOpaque` is off by default. WebKit does not send font requests from a stylesheet
|
|
1354
|
-
through the worker, so fonts stay on the network there.
|
|
1355
|
-
|
|
1356
|
-
```ts
|
|
1357
|
-
import { thirdPartyReport } from '@drupflare/spool/report';
|
|
1358
|
-
import { createSpool, memorySink, viewsOf } from '@drupflare/spool/server';
|
|
1359
|
-
import { memoryStore } from '@drupflare/spool/stores';
|
|
1360
|
-
import { routes } from './routes';
|
|
1361
|
-
|
|
1362
|
-
const sink = memorySink();
|
|
1363
|
-
|
|
1364
|
-
export const spool = createSpool({
|
|
1365
|
-
routes,
|
|
1366
|
-
store: memoryStore(),
|
|
1367
|
-
sink,
|
|
1368
|
-
thirdParty: {
|
|
1369
|
-
allow: ['https://fonts.gstatic.com', 'https://cdn.example.edu'],
|
|
1370
|
-
maxEntries: 32
|
|
1371
|
-
}
|
|
1372
|
-
});
|
|
1373
|
-
|
|
1374
|
-
export const report = () => thirdPartyReport(viewsOf(sink.events));
|
|
1375
|
-
```
|
|
1376
|
-
|
|
1377
|
-
---
|
|
1378
|
-
|
|
1379
|
-
## โก Subresource Handling
|
|
1380
|
-
|
|
1381
|
-
By default the worker answers an immutable same-origin asset from CacheStorage. `assets` changes
|
|
1382
|
-
that. `passthrough` leaves assets to the browser's cache and never touches them. `static-routing`
|
|
1383
|
-
asks the browser through `InstallEvent.addRoutes` to send only navigations, beacons and routed
|
|
1384
|
-
paths to the worker and the rest of the origin to the network, and acts as `passthrough` where the
|
|
1385
|
-
API does not exist: Chromium has it, Playwright's WebKit has it on macOS and not on Linux, and
|
|
1386
|
-
Firefox does not. Both options cut the service worker's fetch events (Static Routing took them from
|
|
1387
|
-
32 to 3 per view on a desktop Chromium rig); neither changes server-side Worker invocations.
|
|
1388
|
-
Neither is measured to save energy. In the same run, new-tab views drew 2.78 J,
|
|
1389
|
-
2.83 J and 2.84 J of RAPL package energy above idle with CacheStorage, navigation-only handling
|
|
1390
|
-
and Static Routing (8 rounds each), and the ranges overlap. Under Static Routing a same-origin write no
|
|
1391
|
-
longer reaches the worker, so it is not a network moment and its response cannot carry
|
|
1392
|
-
invalidations; navigations still are.
|
|
1393
|
-
|
|
1394
|
-
```ts
|
|
1395
|
-
import { createSpool } from '@drupflare/spool/server';
|
|
1396
|
-
import { memoryStore } from '@drupflare/spool/stores';
|
|
1397
|
-
import { routes } from './routes';
|
|
1398
|
-
|
|
1399
|
-
export const spool = createSpool({
|
|
1400
|
-
routes,
|
|
1401
|
-
store: memoryStore(),
|
|
1402
|
-
assets: 'static-routing'
|
|
1403
|
-
});
|
|
1404
|
-
```
|
|
436
|
+
`spool.publishStatic(sink)` is a different route to the same tier, for a host that runs no spool server. It writes the page runtime, the worker script, the packs and their deltas, one boot script per pack and a `_headers` file to the sink, and the host answers those files with no handler call. It does not write `spool.json`. With no policy known, the stub names none, and a device renders locally after one network view that teaches it the headers. Pass `policy` to `createSpool` (`{}` for a site with no security headers) to name them. The Vite, Vue, React Router and Nuxt plugins call `publishStatic` at the end of a build by default; `static: false` turns that off. See [Static Tier](docs/protocol.md#static-tier) and the [static example](examples/static/README.md).
|
|
1405
437
|
|
|
1406
438
|
---
|
|
1407
439
|
|
|
1408
|
-
##
|
|
1409
|
-
|
|
1410
|
-
|
|
1411
|
-
the
|
|
1412
|
-
|
|
1413
|
-
|
|
1414
|
-
|
|
1415
|
-
|
|
1416
|
-
|
|
1417
|
-
|
|
1418
|
-
|
|
1419
|
-
|
|
1420
|
-
|
|
1421
|
-
|
|
1422
|
-
|
|
1423
|
-
|
|
1424
|
-
|
|
1425
|
-
|
|
1426
|
-
|
|
1427
|
-
|
|
1428
|
-
|
|
440
|
+
## Advanced Usage
|
|
441
|
+
|
|
442
|
+
[ADVANCED_USAGE.md](ADVANCED_USAGE.md) covers each declaration and setting past the Quick Start,
|
|
443
|
+
in the order an operator meets them. Each section gives the use, the smallest declaration and the
|
|
444
|
+
limit.
|
|
445
|
+
|
|
446
|
+
- [Strategies](ADVANCED_USAGE.md#strategies): the five ways a route is answered locally, and the
|
|
447
|
+
security headers a local page keeps.
|
|
448
|
+
- [Records and Projections](ADVANCED_USAGE.md#records-and-projections): shared records, stored once
|
|
449
|
+
per id.
|
|
450
|
+
- [Templates](ADVANCED_USAGE.md#templates): the template syntax and the trusted-field rule.
|
|
451
|
+
- [Static Files](ADVANCED_USAGE.md#static-files): files kept as the server sent them.
|
|
452
|
+
- [Projection and Data Routes](ADVANCED_USAGE.md#projection-and-data-routes): JSON answers, bundled
|
|
453
|
+
renderers and the provisioning rule.
|
|
454
|
+
- [Query Routes](ADVANCED_USAGE.md#query-routes): filter, sort and page a held collection with no
|
|
455
|
+
request.
|
|
456
|
+
- [Freshness Contracts](ADVANCED_USAGE.md#freshness-contracts): hard and soft bounds, and the script
|
|
457
|
+
lease.
|
|
458
|
+
- [Datasets](ADVANCED_USAGE.md#datasets): row patches with a proof, plus
|
|
459
|
+
[Dataset Routes](ADVANCED_USAGE.md#dataset-routes) and
|
|
460
|
+
[Principal Datasets](ADVANCED_USAGE.md#principal-datasets).
|
|
461
|
+
- [Authenticated Data](ADVANCED_USAGE.md#authenticated-data): per-principal partitions, sealed
|
|
462
|
+
storage and logout wipes.
|
|
463
|
+
- [Declared Writes](ADVANCED_USAGE.md#declared-writes): queued last-value edits with a version check.
|
|
464
|
+
- [Page Runtime Settings](ADVANCED_USAGE.md#page-runtime-settings): the bootstrap gate, `workerMode`,
|
|
465
|
+
swap elements and the `window.spool` surface.
|
|
466
|
+
- [Catalogue](ADVANCED_USAGE.md#catalogue): data for the pages a visitor may open next.
|
|
467
|
+
- [Analytics Batching](ADVANCED_USAGE.md#analytics-batching) and
|
|
468
|
+
[Collectors](ADVANCED_USAGE.md#collectors): beacons, sinks and third-party collection.
|
|
469
|
+
- [Third-Party Files](ADVANCED_USAGE.md#third-party-files) and
|
|
470
|
+
[Subresource Handling](ADVANCED_USAGE.md#subresource-handling).
|
|
471
|
+
- [Shadow Mode](ADVANCED_USAGE.md#shadow-mode): compare local renders with the network before
|
|
472
|
+
serving.
|
|
473
|
+
- [Proxy Mode](ADVANCED_USAGE.md#proxy-mode): front an origin that cannot load the library.
|
|
474
|
+
- [Trimmed Stylesheets](ADVANCED_USAGE.md#trimmed-stylesheets) and
|
|
475
|
+
[Images and Video](ADVANCED_USAGE.md#images-and-video): proxy options for CSS, images and video.
|
|
476
|
+
- [Capacity Reports](ADVANCED_USAGE.md#capacity-reports): the capacity plan, the institution report
|
|
477
|
+
and request arithmetic.
|
|
1429
478
|
|
|
1430
479
|
---
|
|
1431
480
|
|
|
1432
|
-
##
|
|
1433
|
-
|
|
1434
|
-
`spoolProxy` fronts an origin that cannot load the library, such as PHP, Drupal with no module,
|
|
1435
|
-
or anything else that speaks HTTP. It returns a Fetch handler. The proxy answers `/.spool/*` and
|
|
1436
|
-
`/spool-sw.js`. Every other request goes to the origin with its method, headers and body, and a
|
|
1437
|
-
redirect is passed on, not followed. A page the proxy may keep is decorated like a `createSpool`
|
|
1438
|
-
response, so the worker stores it and answers later views with no request. A cold visit costs one
|
|
1439
|
-
origin fetch: the proxy names a hash of the page in `Spool-Artifact`, and the worker keeps the
|
|
1440
|
-
page it already received as the artifact when its own hash matches. On any mismatch it asks
|
|
1441
|
-
`/.spool/artifact` as before. In the first-payload reading (L1, a synthetic 11,145-byte page) a
|
|
1442
|
-
cold visit made 1 request and 1 origin fetch where it made 2 and 2 before.
|
|
1443
|
-
|
|
1444
|
-
`spool` takes the options `createSpool` takes, minus `routes`. The proxy adds one public
|
|
1445
|
-
`assembly` route that matches every path (`/**`) and uses the origin's finished HTML as the
|
|
1446
|
-
artifact. `routes` lists overrides, such as a shorter freshness bound for one pattern. They
|
|
1447
|
-
match first and take the same fields as `defineRoute`, without `load`.
|
|
481
|
+
## Security
|
|
1448
482
|
|
|
1449
|
-
|
|
1450
|
-
and has no
|
|
1451
|
-
|
|
1452
|
-
|
|
1453
|
-
|
|
1454
|
-
|
|
1455
|
-
|
|
1456
|
-
import { memoryStore } from '@drupflare/spool/stores';
|
|
1457
|
-
|
|
1458
|
-
const proxy = spoolProxy({
|
|
1459
|
-
origin: process.env['ORIGIN'] as string,
|
|
1460
|
-
spool: { store: memoryStore() },
|
|
1461
|
-
purgeSecret: process.env['PURGE_SECRET']
|
|
1462
|
-
});
|
|
1463
|
-
|
|
1464
|
-
export const server = createServer(toNodeListener(proxy));
|
|
1465
|
-
```
|
|
483
|
+
Authorisation stays on the server. The device holds only what the server already sent that
|
|
484
|
+
principal, and the worker has no access rule. Principal data sits in a per-partition IndexedDB
|
|
485
|
+
database sealed with AES-GCM, and deleting the key at logout is crypto-shredding of data at rest.
|
|
486
|
+
That does not protect against script running on your origin. A revoked permission can still show
|
|
487
|
+
for up to a route's `maxStale`, because nothing pushes a revocation to a device. Tag names are
|
|
488
|
+
readable through `/.spool/sync`, so use opaque ids for anything sensitive. Principal datasets and
|
|
489
|
+
declared writes ship off and have had no external review.
|
|
1466
490
|
|
|
1467
|
-
|
|
1468
|
-
|
|
1469
|
-
|
|
1470
|
-
|
|
1471
|
-
```ts
|
|
1472
|
-
import { spoolProxy, type SpoolProxy } from '@drupflare/spool/proxy';
|
|
1473
|
-
import { durableObjectStore } from '@drupflare/spool/stores/durable-object';
|
|
1474
|
-
|
|
1475
|
-
export { SpoolVersions } from '@drupflare/spool/stores/durable-object';
|
|
1476
|
-
|
|
1477
|
-
let proxy: SpoolProxy | undefined;
|
|
1478
|
-
|
|
1479
|
-
export default {
|
|
1480
|
-
fetch(request: Request, env: Cloudflare.Env) {
|
|
1481
|
-
proxy ??= spoolProxy({
|
|
1482
|
-
origin: 'https://origin.example.edu',
|
|
1483
|
-
spool: { store: durableObjectStore(env.SPOOL_VERSIONS) },
|
|
1484
|
-
purgeSecret: env.PURGE_SECRET
|
|
1485
|
-
});
|
|
1486
|
-
return proxy(request);
|
|
1487
|
-
}
|
|
1488
|
-
};
|
|
1489
|
-
```
|
|
1490
|
-
|
|
1491
|
-
### Tags
|
|
1492
|
-
|
|
1493
|
-
The tags the origin sends become the page's dependencies. The proxy reads `Cache-Tag` (comma
|
|
1494
|
-
separated), `Surrogate-Key` and `X-Drupal-Cache-Tags` (space separated) and `xkey` (spaces or
|
|
1495
|
-
commas), then removes them from the response unless `exposeTags` is true. A page with no tags
|
|
1496
|
-
depends on one tag derived from its path.
|
|
1497
|
-
|
|
1498
|
-
### Eligibility
|
|
1499
|
-
|
|
1500
|
-
The proxy keeps a response only when all of these hold:
|
|
1501
|
-
|
|
1502
|
-
- the request is a GET or HEAD, has no query string and no `Authorization` header, and carries
|
|
1503
|
-
no cookie outside `publicCookies`, which is empty by default
|
|
1504
|
-
- the response is a 200 `text/html`, or the answer of a `data` route
|
|
1505
|
-
- the page is UTF-8, or declares no charset in its `Content-Type` or in a `<meta>` within its
|
|
1506
|
-
first 1024 bytes
|
|
1507
|
-
- the response sets no cookie outside `publicCookies`, has no `private` or `no-store` in
|
|
1508
|
-
`Cache-Control`, a `Vary` that names only `Accept-Encoding` and `Cookie`, no more than
|
|
1509
|
-
`maxTags` tags and no more than `maxBytes` bytes
|
|
1510
|
-
|
|
1511
|
-
Everything else is forwarded untouched. A page the proxy declines tells the worker to drop its
|
|
1512
|
-
copy of that path, and a response that sets a cookie outside `publicCookies` tells the worker to
|
|
1513
|
-
clear everything it holds, so a login leaves no anonymous copy to show. Put cookies that carry no
|
|
1514
|
-
identity on the list, such as a bot-check or load-balancer cookie set on every response; otherwise
|
|
1515
|
-
proxy mode never keeps a page on that site. Writes always go to the origin and are never
|
|
1516
|
-
answered locally.
|
|
1517
|
-
|
|
1518
|
-
Principal routes are off unless `spool.identity` is set. With it, a route declared
|
|
1519
|
-
`scope: 'principal'` follows the partitions in [Authenticated Data](#-authenticated-data). The
|
|
1520
|
-
cookie, `Authorization` and `private` rules do not apply to it. `Set-Cookie` and `no-store`
|
|
1521
|
-
still do.
|
|
1522
|
-
|
|
1523
|
-
### Invalidation
|
|
1524
|
-
|
|
1525
|
-
Two signals reach the proxy. When a page it forwards differs from the last body it saw, it bumps
|
|
1526
|
-
that page's tags. The proxy hashes the body after `normalise`, and `ignore` takes patterns to
|
|
1527
|
-
blank, such as a per-render form token. `POST /.spool/purge` bumps the tags it is given, so a CMS
|
|
1528
|
-
hook or Drupal's purge module can call it. It needs `Authorization: Bearer <purgeSecret>` and a
|
|
1529
|
-
JSON body, and it refuses every call when no secret is set.
|
|
1530
|
-
|
|
1531
|
-
```sh
|
|
1532
|
-
curl -X POST https://example.edu/.spool/purge \
|
|
1533
|
-
-H "Authorization: Bearer $PURGE_SECRET" \
|
|
1534
|
-
-H "Content-Type: application/json" \
|
|
1535
|
-
-d '{"tags": ["node:1", "config:system.menu.main"]}'
|
|
1536
|
-
```
|
|
1537
|
-
|
|
1538
|
-
### Limits
|
|
1539
|
-
|
|
1540
|
-
The proxy sees finished HTML, not the database behind it. An artifact is a whole page, the
|
|
1541
|
-
replica holds no records, and two pages never share a fragment. A page stays on a device until its
|
|
1542
|
-
route's `maxStale` passes and the next sync reports a change, so freshness is bounded by
|
|
1543
|
-
`maxStale` plus the time a purge takes to arrive.
|
|
1544
|
-
|
|
1545
|
-
A tag shared by many pages invalidates all of them when one changes. The remembered bodies live
|
|
1546
|
-
in the instance: a restart forgets them, and each instance of a scaled-out proxy keeps its own.
|
|
1547
|
-
A whole page is a large artifact, and the provisioning rule declines one that costs more than the
|
|
1548
|
-
priors say it saves, so set `provision` on the spool for large pages. A `Content-Security-Policy`
|
|
1549
|
-
that allows only nonce or hash scripts blocks the injected client tag. A URL with a query string
|
|
1550
|
-
always goes to the origin, because the proxy holds finished HTML and has no collection to filter;
|
|
1551
|
-
[Query Routes](#query-routes) apply to `createSpool` only. [Trimmed Stylesheets](#-trimmed-stylesheets) covers the `trim` option
|
|
1552
|
-
and [Images and Video](#-images-and-video) the `images` and `video` options.
|
|
491
|
+
[docs/security.md](docs/security.md) is the threat model. It names the file and the test behind
|
|
492
|
+
each claim and lists the open limitations in one table. Report a vulnerability as
|
|
493
|
+
[SECURITY.md](https://github.com/drupflare/spool/blob/master/SECURITY.md) describes.
|
|
1553
494
|
|
|
1554
495
|
---
|
|
1555
496
|
|
|
1556
|
-
##
|
|
1557
|
-
|
|
1558
|
-
A new page of a Drupal site usually loads a stylesheet aggregate the visitor has not seen, and
|
|
1559
|
-
most of its rules match nothing on the page. `@drupflare/spool/trim` builds a manifest that removes
|
|
1560
|
-
the rules no page of the site can match, and `spoolProxy` serves the trimmed text in place of the
|
|
1561
|
-
origin's file. Each file is trimmed against every page and script of the site, never against one
|
|
1562
|
-
page, so it keeps its URL and its one cached copy. Rules are only removed, so the cascade of what
|
|
1563
|
-
stays is unchanged.
|
|
497
|
+
## Energy Reporting
|
|
1564
498
|
|
|
1565
|
-
|
|
1566
|
-
|
|
1567
|
-
|
|
1568
|
-
import { buildTrimManifest } from '@drupflare/spool/trim';
|
|
499
|
+
Energy is one objective among those spool reports, and the administrator decides how much weight
|
|
500
|
+
it gets. The energy of one deployment is small against an institution's total, and small against
|
|
501
|
+
the idle and hosting energy drupflare removes per site. The figures below are research figures.
|
|
1569
502
|
|
|
1570
|
-
|
|
1571
|
-
|
|
1572
|
-
|
|
1573
|
-
|
|
1574
|
-
|
|
503
|
+
[docs/impact.md](docs/impact.md) is generated from the readings in `measurements/` by
|
|
504
|
+
`bun run impact`, and `bun run impact:check` fails when the page differs from them. Energy is
|
|
505
|
+
reported in kWh per million views, and nothing sets a target intensity. Every figure carries a
|
|
506
|
+
layer: L0 published, L1 measured here, L2 derived, L3 scenario. A per-view figure that spreads a
|
|
507
|
+
published intensity over bytes is an allocation of that figure, and the page prints the metered
|
|
508
|
+
reading beside it. The savings from trimming, images and dictionaries apply to network views and
|
|
509
|
+
never add to the local-view figure.
|
|
1575
510
|
|
|
1576
|
-
|
|
1577
|
-
origin: 'https://origin.example.edu',
|
|
1578
|
-
spool: { store: memoryStore() },
|
|
1579
|
-
trim: manifest
|
|
1580
|
-
});
|
|
1581
|
-
```
|
|
511
|
+
The request counts in [Economics](docs/economics.md#headline-numbers) come from library readings on Chromium and Firefox. WebKit is absent from every library reading. The library's headline covers Chromium and Firefox only. WebKit, which is every browser on iOS, defaults to no worker because of an upstream script check on every navigation; its figures are model rows or single script-check readings, never library readings. "Chromium and Firefox" is well under all of mobile.
|
|
1582
512
|
|
|
1583
|
-
|
|
1584
|
-
|
|
1585
|
-
|
|
1586
|
-
`'fs-carousel' + '-enabled'`, is only in the rendered page. Pass the scripts of other origins that
|
|
1587
|
-
the pages load as `foreignScripts`, since a widget that builds markup adds classes too.
|
|
1588
|
-
|
|
1589
|
-
Two checks protect a visitor. The proxy serves a trimmed file only when the origin's bytes still
|
|
1590
|
-
hash to the manifest's `source`, and otherwise sends the origin's file and counts a mismatch. It
|
|
1591
|
-
also reads every HTML page before it leaves: a page with a token the trim removed a rule for, or a
|
|
1592
|
-
same-origin script the manifest was not built with, links the full file through `spool-trim=0`.
|
|
1593
|
-
`proxy.trim` counts the files served, the mismatches and the guard firings, and the sink gets
|
|
1594
|
-
each mismatch and firing as an event. [docs/security.md](docs/security.md#trimmed-stylesheets) names
|
|
1595
|
-
what the guard cannot see: a class a vendor script adds after the crawl, markup injected after load
|
|
1596
|
-
from a page the crawl never requested, and a class name built at runtime that no crawl rendered.
|
|
1597
|
-
|
|
1598
|
-
The proxy trims CSS and leaves JavaScript alone. Coverage at load cannot see what a click or a
|
|
1599
|
-
scroll will need: the used share of same-origin JavaScript rose from 36% at load to 40% after one
|
|
1600
|
-
scroll on the pages measured (L1), so no load-time figure says which of the rest is dead. The same
|
|
1601
|
-
saving from a build step that runs PurgeCSS with a safelist needs no proxy. The proxy gets it
|
|
1602
|
-
without a change to the origin. The saving applies to a network view of a page the visitor has not
|
|
1603
|
-
seen, never to a repeat or local view, and it is not part of the per-view headline;
|
|
1604
|
-
[docs/impact.md](docs/impact.md#asset-trimming) has the bytes, joules and prices.
|
|
513
|
+
[An Institution's Figures](docs/impact.md#an-institutions-figures) leads with economics, storage
|
|
514
|
+
and trust and treats energy as a research figure. [What This Does Not
|
|
515
|
+
Claim](docs/impact.md#what-this-does-not-claim) lists what the page leaves unsaid.
|
|
1605
516
|
|
|
1606
517
|
---
|
|
1607
518
|
|
|
1608
|
-
##
|
|
519
|
+
## Prior Art
|
|
1609
520
|
|
|
1610
|
-
|
|
1611
|
-
|
|
1612
|
-
|
|
1613
|
-
|
|
1614
|
-
or Vimeo iframe for a link that loads the player on click.
|
|
521
|
+
spool is a server-authoritative replica whose freshness policy lets a provisioned view render with
|
|
522
|
+
no per-view validation request, with shadow mode and a fall back to the server for anything it
|
|
523
|
+
cannot answer. HTTP caching, service-worker precaching and earlier replication systems cover parts
|
|
524
|
+
of it. What is shared and what differs:
|
|
1615
525
|
|
|
1616
|
-
|
|
1617
|
-
|
|
1618
|
-
|
|
1619
|
-
|
|
526
|
+
| System | What it does | Shared with spool | Difference |
|
|
527
|
+
| ------------------ | ------------------------------------------------------------- | ----------------------------------------------- | ----------------------------------------------------------------------------------------- |
|
|
528
|
+
| HTTP caching | reuses a response within `max-age` with no validation | a repeat view with no request | one response at a time; no per-dependency invalidation and no held collections |
|
|
529
|
+
| Workbox precaching | answers a precached navigation from the cache with no request | a worker answering a navigation with no request | the list and its revisions are fixed at build time and refresh when a new worker installs |
|
|
530
|
+
| Fawkes (NSDI 2020) | caches static page templates and sends a patch of DOM updates | a cached template, the server authoritative | the patch is requested on every load, so a cached template never answers a view alone |
|
|
531
|
+
| Speculation Rules | prefetches or prerenders pages the visitor may open next | acting on a predicted next view | the origin answers every speculated request, used or not |
|
|
1620
532
|
|
|
1621
|
-
|
|
1622
|
-
|
|
1623
|
-
origin: 'https://origin.example.edu',
|
|
1624
|
-
spool: { store: memoryStore() },
|
|
1625
|
-
images: { encoder, width: { fallback: 1600 } },
|
|
1626
|
-
video: { preload: 'none', autoplay: 'defer', when: 'slow' }
|
|
1627
|
-
});
|
|
1628
|
-
}
|
|
1629
|
-
```
|
|
1630
|
-
|
|
1631
|
-
An `ImageEncoder` takes the origin's bytes and `{ format: 'webp', quality, width }` and returns
|
|
1632
|
-
`{ bytes }`. It never upscales, so `width` is a ceiling, and it throws on input it cannot decode.
|
|
1633
|
-
spool has no runtime dependencies and ships an adapter for [tinyimg](https://tinyimg.gmitch215.dev),
|
|
1634
|
-
which decodes and encodes inside WebAssembly:
|
|
1635
|
-
|
|
1636
|
-
```js
|
|
1637
|
-
import { readFile } from 'node:fs/promises';
|
|
1638
|
-
import { TinyImgModule, transform } from '@gmitch215/tinyimg';
|
|
1639
|
-
import { tinyimgEncoder } from '@drupflare/spool/media';
|
|
1640
|
-
|
|
1641
|
-
const wasm = new URL(import.meta.resolve('@gmitch215/tinyimg/tinyimg.wasm'));
|
|
1642
|
-
const module = await TinyImgModule.loadBytes(await readFile(wasm));
|
|
1643
|
-
|
|
1644
|
-
export const encoder = tinyimgEncoder({ module, transform });
|
|
1645
|
-
```
|
|
1646
|
-
|
|
1647
|
-
Use tinyimg 1.1.1 or later. Version 1.1.0 and earlier drop the EXIF orientation tag without
|
|
1648
|
-
rotating the pixels, so a phone photo comes out sideways. `tinyimgEncoder` throws when the module
|
|
1649
|
-
reports an older version.
|
|
1650
|
-
|
|
1651
|
-
Quality is 80, 65 for `Save-Data: on` and 70 for an `ECT` of `slow-2g`, `2g` or `3g`. The width
|
|
1652
|
-
comes from `Sec-CH-Width`, then `Sec-CH-Viewport-Width` times `Sec-CH-DPR`, then
|
|
1653
|
-
`width.fallback`. The proxy converts a `GET` that matches `paths` (JPEG and PNG names by default),
|
|
1654
|
-
has no `Range` or `Authorization`, and gets a `200` JPEG or PNG without a cookie between
|
|
1655
|
-
`minBytes` (4096) and `maxBytes` (15,000,000). Anything else, and any file the encoder cannot
|
|
1656
|
-
shrink, goes out as the origin sent it. The proxy fetches the origin for every request, so a
|
|
1657
|
-
visitor only gets bytes the origin sent that visitor.
|
|
1658
|
-
|
|
1659
|
-
A converted response carries `Vary` for `Accept`, `Save-Data`, `Sec-CH-Width`, `Sec-CH-DPR`,
|
|
1660
|
-
`Sec-CH-Viewport-Width` and `ECT`, a weak `ETag` and `X-Spool-Image`. HTML responses gain
|
|
1661
|
-
`Accept-CH` for the three size hints and `ECT` unless `hints` is false, and the browser then
|
|
1662
|
-
sends them with later image requests to the same origin.
|
|
1663
|
-
|
|
1664
|
-
`maxWidth` (2560) is the width for a request that carries no hint, and `false` removes it. `png`
|
|
1665
|
-
is `'lossy'` by default. `'lossless'` encodes every PNG without loss, and `'auto'` encodes a PNG
|
|
1666
|
-
under 200 KB both ways and serves the smaller file, which suits screenshots and flat graphics.
|
|
1667
|
-
Requests for the same file, quality and width share one encoder call while it runs. The proxy
|
|
1668
|
-
waits `timeoutMs` (10,000) for the encoder and then serves the origin's file, and the late result
|
|
1669
|
-
is cached for the next request. `timeoutMs: false` waits without limit.
|
|
1670
|
-
|
|
1671
|
-
Drupal's video formatter has no `preload` setting, and `preload="none"` does not stop `autoplay`.
|
|
1672
|
-
`autoplay: 'defer'` removes the attribute, adds `controls` and marks the tag
|
|
1673
|
-
`data-spool-autoplay`. With `when: 'always'` the page the spool keeps carries the rewrite. With
|
|
1674
|
-
`'save-data'` or `'slow'` the rewritten page is private to that visitor: it gets `private` in
|
|
1675
|
-
`Cache-Control` and `Save-Data, ECT` in `Vary`, and the spool does not keep it. A device's stored
|
|
1676
|
-
copy of that page keeps the origin's tags.
|
|
1677
|
-
|
|
1678
|
-
`proxy.images` counts `converted`, `bytesIn` and `bytesOut` (converted responses only),
|
|
1679
|
-
`cacheHits`, `coalesced`, `lazied`, `responsive`, and `skipped` by reason. `proxy.video` counts
|
|
1680
|
-
`pages` and `rewritten`. Read the saving as `bytesIn - bytesOut` over real traffic on the site. A
|
|
1681
|
-
request for any other path counts under `skipped.path`, so that key follows total traffic.
|
|
1682
|
-
[docs/security.md](docs/security.md#images-and-video) covers the decoder.
|
|
1683
|
-
|
|
1684
|
-
### Page Rewrites
|
|
1685
|
-
|
|
1686
|
-
Three options change the HTML the origin sends. Each edits start tags only and leaves every other
|
|
1687
|
-
byte as it was, and a rewritten page gets a weak `ETag`.
|
|
1688
|
-
|
|
1689
|
-
```ts
|
|
1690
|
-
import { type ImageEncoder } from '@drupflare/spool/media';
|
|
1691
|
-
import { spoolProxy } from '@drupflare/spool/proxy';
|
|
1692
|
-
import { memoryStore } from '@drupflare/spool/stores';
|
|
1693
|
-
|
|
1694
|
-
export function rewriteProxy(encoder: ImageEncoder) {
|
|
1695
|
-
return spoolProxy({
|
|
1696
|
-
origin: 'https://origin.example.edu',
|
|
1697
|
-
spool: { store: memoryStore() },
|
|
1698
|
-
images: {
|
|
1699
|
-
encoder,
|
|
1700
|
-
png: 'auto',
|
|
1701
|
-
maxWidth: 1920,
|
|
1702
|
-
timeoutMs: 5000,
|
|
1703
|
-
lazy: { skip: 3 },
|
|
1704
|
-
responsive: { widths: [480, 960, 1440], sizes: '(min-width: 60em) 50vw, 100vw' }
|
|
1705
|
-
},
|
|
1706
|
-
embeds: { providers: ['youtube'], thumbnail: false }
|
|
1707
|
-
});
|
|
1708
|
-
}
|
|
1709
|
-
```
|
|
1710
|
-
|
|
1711
|
-
`lazy` adds `loading="lazy"` and `decoding="async"` to each `<img>` after the first `skip` (2 by
|
|
1712
|
-
default), and `iframes: true` does the same for `<iframe>`. It leaves a tag that has `loading`,
|
|
1713
|
-
`fetchpriority` or a lazy-loading library's `data-src`, a tag inside a `<header>` or `<noscript>`,
|
|
1714
|
-
and an image that a `<link rel="preload" as="image">` names. A browser that defers the largest
|
|
1715
|
-
image of a page paints it later, so `skip` is the only guard and the option is off until set. In a
|
|
1716
|
-
crawl of 70 pages per profile on 16 sites, the default `skip` of 2 made the largest image lazy on 2
|
|
1717
|
-
of 23 desktop pages and 2 of 20 phone pages that had an image as their largest element, and no
|
|
1718
|
-
`skip` from 2 to 4 reached zero on both profiles.
|
|
1719
|
-
|
|
1720
|
-
`responsive` adds a `srcset` of `?spool-w=<width>` variants to a same-origin `<img>` that has none,
|
|
1721
|
-
one per entry of `widths` (320, 480, 640, 960, 1280 and 1920 by default), and a `sizes` of `100vw`
|
|
1722
|
-
unless the tag has one. The proxy removes `spool-w` before it asks the origin, answers only widths
|
|
1723
|
-
from the list and never sends a file larger than the origin's. Firefox and Safari send no
|
|
1724
|
-
`Sec-CH-Width`, and this is how they get a resized file. Name the layout width in `sizes`, since
|
|
1725
|
-
`100vw` makes the browser pick the widest candidate it can use. A tag with `srcset` or `data-src`,
|
|
1726
|
-
and any tag inside `<picture>`, stays as it is.
|
|
1727
|
-
|
|
1728
|
-
`embeds` replaces a server-rendered YouTube or Vimeo `<iframe>` with one whose `srcdoc` holds a
|
|
1729
|
-
link to the player. YouTube shows its thumbnail (`thumbnail: false` shows a plain frame) and the
|
|
1730
|
-
player loads on click. A page that sends `Content-Security-Policy` is left alone unless
|
|
1731
|
-
`ignoreCsp` is true, and so is an embed with `enablejsapi=1`. An embed that plays on its own by
|
|
1732
|
-
design stays too: a URL with `background=1`, or with `autoplay=1` beside `mute`, `muted` or `loop`,
|
|
1733
|
-
and an iframe whose `class` or `id` names a hero, ambient, background or banner. In a crawl of the
|
|
1734
|
-
server-rendered pages of 52 sites (L1, `measurements/embed-markup`), 2 of 4 Vimeo iframes were
|
|
1735
|
-
background loops. `proxy.embeds` counts `pages`, `rewritten` and `skipped` by reason. [docs/security.md](docs/security.md#embed-facades) covers the rules.
|
|
1736
|
-
|
|
1737
|
-
### Dictionary Transport
|
|
1738
|
-
|
|
1739
|
-
A visitor who holds an earlier stylesheet or script of a directory can fetch the next one as the
|
|
1740
|
-
difference from it. `dictionary` implements this for Chromium 130 and later
|
|
1741
|
-
([RFC 9842](https://www.rfc-editor.org/rfc/rfc9842)). The proxy marks public `.css` and `.js`
|
|
1742
|
-
responses with `Use-As-Dictionary` and answers a request that names one of them with
|
|
1743
|
-
`Content-Encoding: dcz`. spool has no compression library, so the caller supplies the `encoder`,
|
|
1744
|
-
and Node's `zlib` does zstd:
|
|
1745
|
-
|
|
1746
|
-
```ts
|
|
1747
|
-
import { promisify } from 'node:util';
|
|
1748
|
-
import { zstdCompress } from 'node:zlib';
|
|
1749
|
-
import { spoolProxy } from '@drupflare/spool/proxy';
|
|
1750
|
-
import { memoryStore } from '@drupflare/spool/stores';
|
|
1751
|
-
|
|
1752
|
-
const compress = promisify(zstdCompress);
|
|
1753
|
-
|
|
1754
|
-
export const proxy = spoolProxy({
|
|
1755
|
-
origin: 'https://origin.example.edu',
|
|
1756
|
-
spool: { store: memoryStore() },
|
|
1757
|
-
dictionary: {
|
|
1758
|
-
encoder: async (dictionary, input) => new Uint8Array(await compress(input, { dictionary }))
|
|
1759
|
-
}
|
|
1760
|
-
});
|
|
1761
|
-
```
|
|
1762
|
-
|
|
1763
|
-
The `encoder` gets the dictionary, the new file and the coding (`'dcz'`, or `'dcb'` when `formats`
|
|
1764
|
-
lists it) and returns the compressed stream. spool adds the coding's header. An encoder that
|
|
1765
|
-
throws, or whose output is no smaller, leaves the plain file. Only responses the proxy would keep
|
|
1766
|
-
as public ones, with no cookie, ever become dictionaries. `proxy.dictionary` counts `registered`,
|
|
1767
|
-
`encoded`, `bytesIn`, `bytesOut` and `skipped`.
|
|
1768
|
-
[docs/security.md](docs/security.md#dictionary-transport) covers the exposure.
|
|
1769
|
-
|
|
1770
|
-
`base` names the files that register as dictionaries, as a function of the pathname, a regular
|
|
1771
|
-
expression or a list of them. Without it the browser holds the latest file of each directory it
|
|
1772
|
-
fetched, and the sizing in [docs/impact.md](docs/impact.md#savings-today-and-at-maximum) shows that
|
|
1773
|
-
choice recovers little. With it the browser holds only the files you name, one per directory, and
|
|
1774
|
-
every other file of the directory is encoded against the base the request names.
|
|
1775
|
-
|
|
1776
|
-
`chooseBases` picks the bases from a crawl. It takes the site's stylesheets and scripts with their
|
|
1777
|
-
compressed sizes and a `delta` function that returns the size of one file compressed against
|
|
1778
|
-
another. For each directory and file kind it returns the file whose deltas against the others, each
|
|
1779
|
-
capped at the other file's plain size, add up smallest. That `held` policy scores a base for a
|
|
1780
|
-
visitor who already holds it. Pass `{ policy: 'first-visit' }` to add the base's own plain size
|
|
1781
|
-
to the sum, for visitors who load the whole group before they hold anything. Neither policy
|
|
1782
|
-
weighs how often a file is requested or how long browsers keep the base, so neither is a proven
|
|
1783
|
-
minimum of total transfer. A directory with one file has no base, and a tie goes to the URL that
|
|
1784
|
-
sorts first. `baseMatcher` turns the choices into a `base` function.
|
|
1785
|
-
|
|
1786
|
-
```ts
|
|
1787
|
-
import { promisify } from 'node:util';
|
|
1788
|
-
import { brotliCompressSync, zstdCompress } from 'node:zlib';
|
|
1789
|
-
import { baseMatcher, chooseBases, spoolProxy } from '@drupflare/spool/proxy';
|
|
1790
|
-
import { memoryStore } from '@drupflare/spool/stores';
|
|
1791
|
-
|
|
1792
|
-
const compress = promisify(zstdCompress);
|
|
1793
|
-
|
|
1794
|
-
const crawl = [
|
|
1795
|
-
{ url: 'https://origin.example.edu/css/a.css', kind: 'style', text: '.a{color:red}.b{margin:0}' },
|
|
1796
|
-
{
|
|
1797
|
-
url: 'https://origin.example.edu/css/b.css',
|
|
1798
|
-
kind: 'style',
|
|
1799
|
-
text: '.a{color:red}.c{margin:1px}'
|
|
1800
|
-
},
|
|
1801
|
-
{
|
|
1802
|
-
url: 'https://origin.example.edu/css/c.css',
|
|
1803
|
-
kind: 'style',
|
|
1804
|
-
text: '.a{color:red}.d{margin:2px}'
|
|
1805
|
-
}
|
|
1806
|
-
] as const;
|
|
1807
|
-
|
|
1808
|
-
const bytes = (index: number) => Buffer.from(crawl[index]?.text ?? '');
|
|
1809
|
-
|
|
1810
|
-
const choices = await chooseBases(
|
|
1811
|
-
crawl.map(({ url, kind, text }) => ({ url, kind, plain: brotliCompressSync(text).length })),
|
|
1812
|
-
async (file, dictionary) =>
|
|
1813
|
-
(await compress(bytes(file), { dictionary: bytes(dictionary) })).length
|
|
1814
|
-
);
|
|
1815
|
-
|
|
1816
|
-
export const proxy = spoolProxy({
|
|
1817
|
-
origin: 'https://origin.example.edu',
|
|
1818
|
-
spool: { store: memoryStore() },
|
|
1819
|
-
dictionary: {
|
|
1820
|
-
encoder: async (dictionary, input) => new Uint8Array(await compress(input, { dictionary })),
|
|
1821
|
-
base: baseMatcher(choices)
|
|
1822
|
-
}
|
|
1823
|
-
});
|
|
1824
|
-
```
|
|
1825
|
-
|
|
1826
|
-
On the 10 Drupal sites of the reading, a base chosen this way and already held by the browser saves
|
|
1827
|
-
30% of the stylesheet and script bytes at the median site (L1, [docs/impact.md](docs/impact.md#savings-today-and-at-maximum)).
|
|
1828
|
-
`scripts/measure/dictionary-size.ts` records the same choice for a site's crawl.
|
|
1829
|
-
|
|
1830
|
-
### Defaults
|
|
1831
|
-
|
|
1832
|
-
Every option is absent by default. The table shows what a feature does once its parent option is
|
|
1833
|
-
set and how to change it.
|
|
1834
|
-
|
|
1835
|
-
| Feature | Option | Default once the parent is set | Leave or enter |
|
|
1836
|
-
| ------------------------------------------ | ------------------- | ------------------------------------------- | ----------------------------------------------------- |
|
|
1837
|
-
| WebP conversion, `304` and `X-Spool-Image` | `images` | on, needs an `encoder` | leave `images` out |
|
|
1838
|
-
| `Accept-CH` hints and `Vary` | `images.hints` | on | `hints: false` |
|
|
1839
|
-
| Quality by `Save-Data` and `ECT` | `images.quality` | 80, 65 and 70 | give all three one number |
|
|
1840
|
-
| Width ceiling for a request with no width | `images.maxWidth` | 2560 | `maxWidth: false`, or another number |
|
|
1841
|
-
| Shared encodes and the encoder time limit | `images.timeoutMs` | 10,000 ms, identical encodes share one call | `timeoutMs: false` removes the limit |
|
|
1842
|
-
| PNG mode | `images.png` | `'lossy'` | `'auto'` or `'lossless'` |
|
|
1843
|
-
| `loading="lazy"` on images and iframes | `images.lazy` | off | `lazy: {}` or `{ skip, iframes }` |
|
|
1844
|
-
| `srcset` variants | `images.responsive` | off | `responsive: {}` or `{ widths, sizes }` |
|
|
1845
|
-
| `preload="none"` on video | `video.preload` | `'none'` | `preload: false` |
|
|
1846
|
-
| Video autoplay deferral | `video.autoplay` | off | `autoplay: 'defer'` |
|
|
1847
|
-
| YouTube and Vimeo facades | `embeds` | off | `embeds: {}` or `{ providers, thumbnail, ignoreCsp }` |
|
|
1848
|
-
| Dictionary transport | `dictionary` | off | `dictionary: { encoder }` |
|
|
1849
|
-
| Which files become dictionaries | `dictionary.base` | every `.css` and `.js` response | `base: /\/base\.js$/`, a list or a function |
|
|
1850
|
-
|
|
1851
|
-
---
|
|
1852
|
-
|
|
1853
|
-
## ๐ Capacity Reports
|
|
1854
|
-
|
|
1855
|
-
`@drupflare/spool/report` turns peak work and an institution's own limits into three separate
|
|
1856
|
-
results: potential, financial and physical. None is added to another, and any can be empty.
|
|
1857
|
-
|
|
1858
|
-
`capacityPlan` takes intervals of demand per resource with the part spool avoided, and what one
|
|
1859
|
-
node serves per second within the latency target. It returns the nodes needed before and after,
|
|
1860
|
-
as whole numbers, and how many could retire. `perNode.busyMs` is the milliseconds of handler work
|
|
1861
|
-
a node does per second, such as 1000 times its cores times a target utilisation. `spares`, `floor`,
|
|
1862
|
-
`regions` and `minPerRegion` raise the answer for redundancy, and a resource with no `avoided`, such
|
|
1863
|
-
as memory, does not shrink. `intervalsFromWork` builds the intervals from the `WorkSample`s the
|
|
1864
|
-
work series records.
|
|
1865
|
-
|
|
1866
|
-
`institutionReport` adds the plan, an `observed` before and after of the same workload from
|
|
1867
|
-
production telemetry, an invoice total before and after, and a ledger of retired equipment. The
|
|
1868
|
-
plan says what could change, `observed` what changed, the invoices what was paid, and the ledger
|
|
1869
|
-
what was removed. The physical result reads only the ledger, and an entry needs the evidence for
|
|
1870
|
-
its removal. With no ledger the statement reads `No physical capacity saving demonstrated.`
|
|
1871
|
-
|
|
1872
|
-
```ts
|
|
1873
|
-
import { capacityPlan, institutionReport } from '@drupflare/spool/report';
|
|
1874
|
-
|
|
1875
|
-
const plan = capacityPlan(
|
|
1876
|
-
[
|
|
1877
|
-
{
|
|
1878
|
-
start: Date.UTC(2026, 8, 1, 14),
|
|
1879
|
-
seconds: 300,
|
|
1880
|
-
demand: { busyMs: 9000 },
|
|
1881
|
-
avoided: { busyMs: 6300 }
|
|
1882
|
-
},
|
|
1883
|
-
{
|
|
1884
|
-
start: Date.UTC(2026, 8, 1, 14, 5),
|
|
1885
|
-
seconds: 300,
|
|
1886
|
-
demand: { busyMs: 7000 },
|
|
1887
|
-
avoided: { busyMs: 4900 }
|
|
1888
|
-
}
|
|
1889
|
-
],
|
|
1890
|
-
{ perNode: { busyMs: 700 }, spares: 1, floor: 2 }
|
|
1891
|
-
);
|
|
1892
|
-
|
|
1893
|
-
export const report = institutionReport({ plan, planLayer: 'L3' });
|
|
1894
|
-
```
|
|
1895
|
-
|
|
1896
|
-
The example needs 14 nodes without spool and 5 with it, a potential of 9, at layer L3 because the
|
|
1897
|
-
numbers are made up. `channelRows` lists the metered, attributional, capacity and avoided-demand
|
|
1898
|
-
channels as separate rows, and nothing in the module sums them. [docs/impact.md](docs/impact.md#channels)
|
|
1899
|
-
says what each one counts.
|
|
533
|
+
Sync Kit (WWW 2010) also keeps templates and data in the browser to reduce server load. The
|
|
534
|
+
[technical report](TECHNICAL_REPORT.md#executive-summary) states the narrow claim.
|
|
1900
535
|
|
|
1901
536
|
---
|
|
1902
537
|
|
|
1903
|
-
##
|
|
538
|
+
## Out of Scope
|
|
1904
539
|
|
|
1905
540
|
- spool never authorises on the client. The device holds only what the server already sent that
|
|
1906
541
|
principal.
|
|
1907
542
|
- It never shows a hard route past its bound without validating it first.
|
|
1908
|
-
- It never writes. Your application owns every write, and `invalidate` only announces
|
|
543
|
+
- It never writes on its own. Your application owns every write, and `invalidate` only announces
|
|
544
|
+
one. The one exception is a declared write class, which is off until you enable it.
|
|
1909
545
|
- It never infers dependencies from opaque HTML. A route names its tags, or the records it
|
|
1910
546
|
reads, and in proxy mode the origin names them in its cache tag headers.
|
|
1911
547
|
- It never replaces a published release file on the CDN.
|
|
1912
548
|
|
|
1913
549
|
---
|
|
1914
550
|
|
|
1915
|
-
##
|
|
551
|
+
## Browser Support
|
|
1916
552
|
|
|
1917
|
-
| Engine | Controlled navigation
|
|
1918
|
-
| -------- |
|
|
1919
|
-
| Chromium | no request, new tabs included
|
|
1920
|
-
| Firefox | no request, new tabs included
|
|
1921
|
-
| WebKit | one request to re-check `/spool-sw.js`,
|
|
553
|
+
| Engine | Controlled navigation |
|
|
554
|
+
| -------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
555
|
+
| Chromium | no request, new tabs included |
|
|
556
|
+
| Firefox | no request, new tabs included |
|
|
557
|
+
| WebKit | with `workerMode: 'always'`, on macOS, one request to re-check `/spool-sw.js` on each document navigation (none on Linux while the worker runs); with the default `'auto'`, no worker on a page with a ring, so no script request; one refetch of each cacheable subresource on the first controlled load |
|
|
1922
558
|
|
|
1923
559
|
The worker registers with `updateViaCache: 'all'` and the script is served at
|
|
1924
|
-
`max-age=86400`, which is what lets Chromium and Firefox skip the update check.
|
|
1925
|
-
the script
|
|
560
|
+
`max-age=86400`, which is what lets Chromium and Firefox skip the update check. WebKit on macOS
|
|
561
|
+
re-checks the script regardless, so with `workerMode: 'always'` a document navigation there costs one
|
|
562
|
+
request whatever the site does ([docs/impact.md](docs/impact.md#demand-per-client)). With the default
|
|
563
|
+
`workerMode: 'auto'`, WebKit registers no worker on a page that carries a ring, so the script is not
|
|
564
|
+
requested; the page runtime still answers held reads and in-page navigation
|
|
565
|
+
([WebKit Default](docs/protocol.md#webkit-default)). A deploy changes the script bytes and reaches a
|
|
566
|
+
browser within that day. A browser without service workers gets the
|
|
1926
567
|
page it always got.
|
|
1927
568
|
|
|
1928
569
|
A page that loads a resource from another origin is not a zero-request view. spool counts those
|
|
@@ -1930,26 +571,34 @@ loads per view and reports the view as such.
|
|
|
1930
571
|
|
|
1931
572
|
---
|
|
1932
573
|
|
|
1933
|
-
##
|
|
1934
|
-
|
|
1935
|
-
| Import | What is in it
|
|
1936
|
-
| ---------------------------------------- |
|
|
1937
|
-
| `@drupflare/spool` | the protocol and report modules
|
|
1938
|
-
| `@drupflare/spool/protocol` | envelopes, artifacts, patterns, templates, the manifest, view events
|
|
1939
|
-
| `@drupflare/spool/server` | `createSpool`, `defineRoute`, `datasetProof`, sinks, identity and work series |
|
|
1940
|
-
| `@drupflare/spool/node` | `toNodeListener`, `toRequest`, `writeResponse`
|
|
1941
|
-
| `@drupflare/spool/proxy` | `spoolProxy`, `chooseBases`, the tag readers and the eligibility verdict
|
|
1942
|
-
| `@drupflare/spool/stores` | `memoryStore` and the `VersionStore` type
|
|
1943
|
-
| `@drupflare/spool/stores/durable-object` | `durableObjectStore` and the `SpoolVersions` class
|
|
1944
|
-
| `@drupflare/spool/sw` | `serve`, for a site that bundles its own worker
|
|
1945
|
-
| `@drupflare/spool/client` | `register
|
|
1946
|
-
| `@drupflare/spool/report` | the energy model, capacity plans and institution reports
|
|
1947
|
-
| `@drupflare/spool/trim` | `buildTrimManifest`, `trimCss` and `guardPage`
|
|
1948
|
-
| `@drupflare/spool/media` | `tinyimgEncoder`, `imageVariant` and `
|
|
574
|
+
## Subpath Exports
|
|
575
|
+
|
|
576
|
+
| Import | What is in it |
|
|
577
|
+
| ---------------------------------------- | ------------------------------------------------------------------------------------------------------------- |
|
|
578
|
+
| `@drupflare/spool` | the protocol and report modules |
|
|
579
|
+
| `@drupflare/spool/protocol` | envelopes, artifacts, patterns, templates, the manifest, view events |
|
|
580
|
+
| `@drupflare/spool/server` | `createSpool`, `defineRoute`, `defineDataset`, `defineWrite`, `datasetProof`, sinks, identity and work series |
|
|
581
|
+
| `@drupflare/spool/node` | `toNodeListener`, `toRequest`, `writeResponse` |
|
|
582
|
+
| `@drupflare/spool/proxy` | `spoolProxy`, `chooseBases`, the tag readers and the eligibility verdict |
|
|
583
|
+
| `@drupflare/spool/stores` | `memoryStore` and the `VersionStore` type |
|
|
584
|
+
| `@drupflare/spool/stores/durable-object` | `durableObjectStore` and the `SpoolVersions` class |
|
|
585
|
+
| `@drupflare/spool/sw` | `serve`, for a site that bundles its own worker |
|
|
586
|
+
| `@drupflare/spool/client` | `register`, the page handle and `createActivity` |
|
|
587
|
+
| `@drupflare/spool/report` | the energy model, capacity plans and institution reports |
|
|
588
|
+
| `@drupflare/spool/trim` | `buildTrimManifest`, `trimCss` and `guardPage` |
|
|
589
|
+
| `@drupflare/spool/media` | `tinyimgEncoder`, `imageVariant`, `rewriteVideo` and `rewriteEmbeds` |
|
|
590
|
+
| `@drupflare/spool/express` | `spoolExpress` |
|
|
591
|
+
| `@drupflare/spool/hono` | `spoolHono` |
|
|
592
|
+
| `@drupflare/spool/vite` | `spoolVite` |
|
|
593
|
+
| `@drupflare/spool/vue` | `spoolVue` and `vuePages` |
|
|
594
|
+
| `@drupflare/spool/angular` | `spoolAngular` |
|
|
595
|
+
| `@drupflare/spool/nuxt` | the Nuxt module (default export) and `#spool/pages` |
|
|
596
|
+
| `@drupflare/spool/react-router` | `spoolReactRouter`, `reactRouterPages` and `flattenRoutes` |
|
|
597
|
+
| `@drupflare/spool/next` | the Next adapter (default export), `spoolTag`, `spoolRouteHandler`, `nextPages` and `SPOOL_PAGES_FILE` |
|
|
1949
598
|
|
|
1950
599
|
---
|
|
1951
600
|
|
|
1952
|
-
##
|
|
601
|
+
## Testing
|
|
1953
602
|
|
|
1954
603
|
```sh
|
|
1955
604
|
bun run typecheck
|
|
@@ -1957,18 +606,39 @@ bun run test
|
|
|
1957
606
|
bun run test:coverage
|
|
1958
607
|
bun run test:browser
|
|
1959
608
|
bun run prettier:check
|
|
609
|
+
bun run impact:check
|
|
1960
610
|
bun run docs:build
|
|
1961
611
|
bun run build
|
|
1962
612
|
```
|
|
1963
613
|
|
|
1964
|
-
`
|
|
614
|
+
`typecheck` also compiles every `ts` fence in this page and the guides as written, and it fails with
|
|
615
|
+
the page and line of the first fence that does not compile. A fence that cannot stand alone takes
|
|
616
|
+
`ts no-check` after the opening backticks and is skipped. Samples that tests run live under
|
|
617
|
+
`tests/node/readme/`.
|
|
618
|
+
|
|
619
|
+
`impact:check` fails when `docs/impact.md` differs from the readings in `measurements/`. A node
|
|
620
|
+
test fails when `src/server/embedded.ts` differs from the source under `src/protocol`, `src/sw`
|
|
621
|
+
and `src/client`, and `bun run build:sw` regenerates it. `test` runs the unit project in workerd and the node project, which includes a test that packs the
|
|
1965
622
|
package and imports every subpath in plain Node. `test:browser` runs Chromium, Firefox and WebKit
|
|
1966
623
|
against real servers and counts the requests that reach them. `bunx playwright install chromium
|
|
1967
624
|
firefox webkit` fetches the browsers.
|
|
1968
625
|
|
|
1969
626
|
---
|
|
1970
627
|
|
|
1971
|
-
##
|
|
628
|
+
## Documentation
|
|
629
|
+
|
|
630
|
+
- [docs/configuration.md](docs/configuration.md) covers the settings a first deployment decides.
|
|
631
|
+
- [docs/protocol.md](docs/protocol.md) holds the paths, headers, JSON bodies and limits for anyone
|
|
632
|
+
who writes an adapter or a second server. A test pins what `Spool-Protocol: 1` freezes, and the
|
|
633
|
+
page lists what may change without a new number.
|
|
634
|
+
- [docs/security.md](docs/security.md) holds the threat model and the open limitations.
|
|
635
|
+
- [docs/impact.md](docs/impact.md) holds the energy model, the readings and what they leave unsaid.
|
|
636
|
+
- [docs/economics.md](docs/economics.md) holds the bandwidth and Worker invocation projections.
|
|
637
|
+
- [examples/README.md](examples/README.md) lists the example sites and runtimes, and the command that runs each one.
|
|
638
|
+
|
|
639
|
+
---
|
|
640
|
+
|
|
641
|
+
## Related Repositories
|
|
1972
642
|
|
|
1973
643
|
| Repository | What it is |
|
|
1974
644
|
| --------------------------------------------------------------- | -------------------------------------------------------- |
|
|
@@ -1979,6 +649,8 @@ firefox webkit` fetches the browsers.
|
|
|
1979
649
|
|
|
1980
650
|
---
|
|
1981
651
|
|
|
1982
|
-
##
|
|
652
|
+
## License
|
|
1983
653
|
|
|
1984
654
|
MIT (c) Gregory Mitchell 2026. See [LICENSE](LICENSE).
|
|
655
|
+
|
|
656
|
+
The documentation favicons in `docs/` come from Google's [Noto Emoji](https://github.com/googlefonts/noto-emoji) set, licensed under [Apache 2.0](https://www.apache.org/licenses/LICENSE-2.0).
|