coldwire-rails 0.3.0 → 0.5.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +23 -8
- data/README.md +1 -1
- data/VERSION +1 -1
- data/app/assets/javascripts/coldwire/cache_controller.js +78 -46
- data/app/controllers/coldwire/caches_controller.rb +1 -1
- data/app/views/coldwire/caches/show.html.erb +129 -80
- data/app/views/coldwire/service_worker/show.js.erb +11 -0
- data/docs/configuration.md +62 -10
- data/docs/how-it-works.md +21 -0
- data/docs/setup.md +5 -3
- data/lib/coldwire/debug.css +37 -7
- data/lib/coldwire/precache.rb +56 -0
- data/lib/coldwire/worker/collect.js +71 -5
- data/lib/coldwire/worker/events.js +7 -0
- data/lib/coldwire/worker/inspect.js +107 -8
- data/lib/coldwire/worker/rules.js +5 -1
- data/lib/coldwire/worker/serve.js +132 -13
- data/lib/coldwire/worker/sync.js +37 -17
- data/lib/coldwire.rb +1 -0
- data/lib/generators/coldwire/install/templates/coldwire.rb +2 -0
- metadata +2 -1
data/docs/configuration.md
CHANGED
|
@@ -154,6 +154,36 @@ Listing a URL here is an explicit instruction: `cache_as_you_go` does not filter
|
|
|
154
154
|
|
|
155
155
|
A stored page's stylesheets, scripts, and images are fetched with it, whatever the lists say.
|
|
156
156
|
|
|
157
|
+
**Naming a frame or a format.** A URL is one field short of naming a body: the same path
|
|
158
|
+
answers a visit, a Turbo Frame and a `respond_to` format with three different things, and a
|
|
159
|
+
bare listing asks for the page. A Hash says which one you mean:
|
|
160
|
+
|
|
161
|
+
```ruby
|
|
162
|
+
sync.precache_urls = -> {
|
|
163
|
+
Feature.published.map { |f| { url: feature_path(f), frame: "map_feature_popup" } } +
|
|
164
|
+
Report.all.map { |r| { url: report_path(r), format: :json } } +
|
|
165
|
+
[ root_path ]
|
|
166
|
+
}
|
|
167
|
+
```
|
|
168
|
+
|
|
169
|
+
`frame:` is the frame tag's own id, which is what Turbo puts in the `Turbo-Frame` header. The
|
|
170
|
+
worker fetches with that header, so an app branching on `turbo_frame_request?` answers with the
|
|
171
|
+
frame, and the body is stored where a frame request will look for it.
|
|
172
|
+
|
|
173
|
+
`format:` is any registered Rails format, including one your app registered itself. It is
|
|
174
|
+
resolved to that format's media type here and sent as `Accept`, so the worker never has to know
|
|
175
|
+
what `:json` means. `accept:` takes a media type directly, for anything with no registered name.
|
|
176
|
+
|
|
177
|
+
Without this, a frame URL is precached as its whole page. That still works offline, since a
|
|
178
|
+
frame request falls back to the page and Turbo pulls the frame out of it, but only while the
|
|
179
|
+
page actually contains that frame, and it stores the whole document where a fragment would do.
|
|
180
|
+
|
|
181
|
+
List a URL twice to precache both:
|
|
182
|
+
|
|
183
|
+
```ruby
|
|
184
|
+
[ feature_path(f), { url: feature_path(f), frame: "map_feature_popup" } ]
|
|
185
|
+
```
|
|
186
|
+
|
|
157
187
|
### `auto_sync.interval`
|
|
158
188
|
|
|
159
189
|
**Default:** `1.day`
|
|
@@ -203,7 +233,8 @@ collect and the cache grows until the browser evicts the lot.
|
|
|
203
233
|
**Only ever with a connection.** Deleting is the one cache operation with no way back:
|
|
204
234
|
whatever goes is gone until the network can be reached again. So a sweep pings
|
|
205
235
|
[`probe_path`](#probe_path) first and stands down if it cannot be reached, and stands down
|
|
206
|
-
under force offline.
|
|
236
|
+
under force offline. The one exception is somebody choosing a storage limit on the settings
|
|
237
|
+
page, which applies right away: see [`max_size`](#garbage_collectionmax_size). `navigator.onLine` is not consulted — a web view answers it wrongly often
|
|
207
238
|
enough to be worthless for a decision this expensive to get wrong.
|
|
208
239
|
|
|
209
240
|
**Untouched, not old.** Age is measured from when an entry was last *used*, not when it was
|
|
@@ -268,15 +299,27 @@ Measured over what a sweep is allowed to take, which is everything but the offli
|
|
|
268
299
|
assets and downloaded archives. Counting a 300 MB download somebody deliberately kept would
|
|
269
300
|
empty the rest of the cache to make room for a file no sweep may touch.
|
|
270
301
|
|
|
302
|
+
Which leaves two figures on the settings page that do not match: the header counts everything
|
|
303
|
+
on the device, the limit governs only part of it. So Storage and Downloads are separate cards,
|
|
304
|
+
each carrying one line about the other. Storage says its figure is cached pages only and that
|
|
305
|
+
downloads are counted below; Downloads says the limit never counts a download and never clears
|
|
306
|
+
one to make room.
|
|
307
|
+
|
|
271
308
|
The ceiling is applied when a sweep runs, so [`interval`](#garbage_collectioninterval) is also
|
|
272
309
|
how long the cache may sit over it. Lower the interval if a tighter bound matters more than
|
|
273
310
|
the work.
|
|
274
311
|
|
|
275
|
-
**People can change it.** The offline settings page offers a ladder of sizes
|
|
276
|
-
default always among them
|
|
277
|
-
|
|
278
|
-
since a worker cannot read `localStorage`.
|
|
279
|
-
|
|
312
|
+
**People can change it.** The offline settings page offers a ladder of sizes, the configured
|
|
313
|
+
default always among them, and the choice is remembered in `localStorage` for that device the
|
|
314
|
+
way Force offline and the Auto Sync switch are. It travels to the worker with each sweep,
|
|
315
|
+
since a worker cannot read `localStorage`.
|
|
316
|
+
|
|
317
|
+
Picking a size applies it on the spot: if the cache is over the new ceiling, the least
|
|
318
|
+
recently read entries go immediately, until it fits. That one pass does **not** wait for a
|
|
319
|
+
connection, and runs under Force offline, because it is a deliberate instruction about
|
|
320
|
+
somebody's own storage rather than an automatic sweep. Clear cache has always worked the same
|
|
321
|
+
way. Only the ceiling is applied; age collection still waits for a connection it has
|
|
322
|
+
confirmed.
|
|
280
323
|
|
|
281
324
|
### `garbage_collection.interval`
|
|
282
325
|
|
|
@@ -548,10 +591,19 @@ archive is how the rest does.
|
|
|
548
591
|
|
|
549
592
|
**Default:** `[]`
|
|
550
593
|
|
|
551
|
-
Large files somebody can choose to keep
|
|
552
|
-
Nothing downloads on its own
|
|
553
|
-
decision.
|
|
554
|
-
|
|
594
|
+
Large files somebody can choose to keep: a tile archive, an audio guide, a reference PDF.
|
|
595
|
+
Nothing downloads on its own, because hundreds of megabytes over somebody's connection is
|
|
596
|
+
their decision. Each appears in the Downloads card on the offline settings page: the title
|
|
597
|
+
with the button hard right, **Download**, then **Download again** once it is on the device or
|
|
598
|
+
**Resume** where one stopped part way, with a trash button beside it to remove it. The
|
|
599
|
+
description, the size and the progress of a running download read underneath, at the full width
|
|
600
|
+
of the card.
|
|
601
|
+
|
|
602
|
+
They live in that card rather than one of their own because they answer the same question the
|
|
603
|
+
storage limit does: what is on this device, and how much of it do you want to keep? What the
|
|
604
|
+
limit governs is at the top, these sit under it with their own sizes, and the card ends on a
|
|
605
|
+
total that reconciles with the header. A download is never deleted to make room and never
|
|
606
|
+
counted against the limit, so removing one is something only the person who chose it does.
|
|
555
607
|
|
|
556
608
|
```ruby
|
|
557
609
|
config.cache_archives = [
|
data/docs/how-it-works.md
CHANGED
|
@@ -23,6 +23,27 @@ Turbo — rather than a native error screen:
|
|
|
23
23
|
3. **Turbo Frames need a frame.** A frame request discards any response without a matching
|
|
24
24
|
`<turbo-frame>`, leaving the frame loading forever. Coldwire reads the `Turbo-Frame`
|
|
25
25
|
header and answers with one.
|
|
26
|
+
|
|
27
|
+
**And a frame is not the page it came from.** Turbo sends that header on every frame
|
|
28
|
+
navigation, and an app answering it with `turbo_frame_request?` returns just the frame.
|
|
29
|
+
Keyed on the URL alone, that body lands in the slot the page occupies: a later cold visit
|
|
30
|
+
is then served a fragment as a whole document, which is a blank screen, and in Hotwire
|
|
31
|
+
Native a page where `window.Turbo` never appears. Whichever was cached last wins, so it
|
|
32
|
+
also happens in reverse. `Vary` cannot fix this, because matching is URL-only by design
|
|
33
|
+
(see 1), so Coldwire puts the frame in the key instead. A frame takes its own entry first
|
|
34
|
+
and the page second, since Turbo pulls a frame out of a document exactly as it does
|
|
35
|
+
online. An ordinary visit never takes the reverse trade.
|
|
36
|
+
|
|
37
|
+
**And neither is any other format.** The same URL answers a `respond_to` block in as many
|
|
38
|
+
formats as the app defines: `/report` is a page to Turbo, JSON to a `fetch`, a CSV to an
|
|
39
|
+
export link and an RSS feed to a reader. Coldwire names the format in the key too, with the
|
|
40
|
+
page left unnamed so anything cached before this keeps the key it had. The name is worked
|
|
41
|
+
out from the request's `Accept`, not the response's type, because the same name has to be
|
|
42
|
+
produced again when the entry is looked for, and there is no response to read then. A
|
|
43
|
+
request asking for data gets data or nothing, since handing it a page is the mistake in 6
|
|
44
|
+
wearing a different hat. A path that names its own format is left alone: `/report.json` is
|
|
45
|
+
JSON and nothing else, so only `/report` needs telling apart. A Turbo Stream is never stored at all: it is a list of changes to
|
|
46
|
+
make to a page, and replaying a stale one applies yesterday's mutations to today's DOM.
|
|
26
47
|
4. **A followed redirect poisons the cache.** A signed-out request to `/` gets a `302` that
|
|
27
48
|
`fetch` follows; the result looks fine and `cache.put()` stores it without complaint.
|
|
28
49
|
Now `/` holds the sign-in page and keeps `redirected: true` — and serving a redirected
|
data/docs/setup.md
CHANGED
|
@@ -135,9 +135,11 @@ redirects to login, so nothing about the online flow changes.
|
|
|
135
135
|
Mounted at the engine root — `/offline` with the mount above. It inherits your
|
|
136
136
|
`ApplicationController`, so it picks up your layout, authentication, and helpers.
|
|
137
137
|
|
|
138
|
-
This is the page people use to turn offline support on or off, see connection status,
|
|
139
|
-
|
|
140
|
-
|
|
138
|
+
This is the page people use to turn offline support on or off, see connection status, turn
|
|
139
|
+
auto-sync off for this device, force offline, and manage what is cached. The three switches
|
|
140
|
+
sit together in the status card at the top, since each is a choice this device makes about how
|
|
141
|
+
Coldwire behaves. Storage and Downloads are separate cards, since the storage limit governs
|
|
142
|
+
cached pages and never touches a download.
|
|
141
143
|
Turning offline support off asks first, then deletes what is stored and hides the rest of the
|
|
142
144
|
page. It sets `content_for :title` to `"Offline settings"` — yield that in your layout's
|
|
143
145
|
`<title>` (and any native title bar that reads it) rather than expecting an on-page heading.
|
data/lib/coldwire/debug.css
CHANGED
|
@@ -114,14 +114,43 @@
|
|
|
114
114
|
.coldwire-input:focus-visible, .coldwire-select:focus-visible { outline: 2px solid #6b7280;
|
|
115
115
|
outline-offset: 1px; }
|
|
116
116
|
|
|
117
|
+
/* The sync clock, its button and its bar, behind a disclosure: worth having, worth going to,
|
|
118
|
+
and three lines of furniture under a switch you are not currently using. Quiet like the
|
|
119
|
+
Inspect cache summary, because it is the same kind of thing. */
|
|
120
|
+
.coldwire-sync { margin-top: 0.9rem; }
|
|
121
|
+
.coldwire-sync > summary { display: flex; align-items: center; gap: 0.35rem; cursor: pointer;
|
|
122
|
+
list-style: none; font-size: 0.75rem; font-weight: 600; letter-spacing: 0.04em;
|
|
123
|
+
text-transform: uppercase; color: #9ca3af; }
|
|
124
|
+
.coldwire-sync > summary::-webkit-details-marker { display: none; }
|
|
125
|
+
.coldwire-sync > summary::after { content: ""; width: 0.8rem; height: 0.8rem; flex-shrink: 0;
|
|
126
|
+
background: var(--coldwire-caret) no-repeat center / contain; opacity: 0.7;
|
|
127
|
+
transition: transform 160ms ease; }
|
|
128
|
+
.coldwire-sync[open] > summary::after { transform: rotate(180deg); }
|
|
129
|
+
.coldwire-sync > summary:focus-visible { outline: 2px solid #6b7280; outline-offset: 2px;
|
|
130
|
+
border-radius: 0.25rem; }
|
|
131
|
+
/* The first line inside sets its own spacing from the summary. */
|
|
132
|
+
.coldwire-sync-body > .coldwire-facts:first-child { margin-top: 0.5rem; }
|
|
133
|
+
|
|
134
|
+
/* The line each card carries about the other: what the limit does not count, and what the
|
|
135
|
+
limit will never take. Quieter than the figures they qualify, and always under them. */
|
|
136
|
+
.coldwire-downloads-note { margin: 0.9rem 0 0; padding-top: 0.75rem;
|
|
137
|
+
border-top: 1px solid #e5e7eb; font-size: 0.8125rem; line-height: 1.5; color: #9ca3af; }
|
|
138
|
+
.coldwire-downloads-note strong { color: #6b7280; font-weight: 600; }
|
|
139
|
+
|
|
117
140
|
/* The rule separates one download from the next; the first has nothing above it. */
|
|
118
|
-
.coldwire-archive { margin-top:
|
|
119
|
-
.coldwire-archive:first-
|
|
120
|
-
|
|
121
|
-
|
|
122
|
-
|
|
123
|
-
|
|
124
|
-
.coldwire-archive-
|
|
141
|
+
.coldwire-archive { margin-top: 0.9rem; padding-top: 0.9rem; border-top: 1px solid #e5e7eb; }
|
|
142
|
+
.coldwire-archive:first-of-type { margin-top: 0; padding-top: 0; border-top: 0; }
|
|
143
|
+
/* The name and the action on one line, the words underneath at full width. The buttons sit
|
|
144
|
+
hard right, where the eye goes for the action, and keep their width whatever the title does:
|
|
145
|
+
a long title gives way instead, because a button that wraps to two lines is worse than a
|
|
146
|
+
name that ellipsises. */
|
|
147
|
+
.coldwire-archive-head { display: flex; align-items: center; justify-content: space-between;
|
|
148
|
+
gap: 0.75rem; }
|
|
149
|
+
.coldwire-archive-actions { flex: 0 0 auto; flex-wrap: nowrap; }
|
|
150
|
+
.coldwire-archive-title { flex: 1 1 auto; min-width: 0; overflow: hidden;
|
|
151
|
+
text-overflow: ellipsis; white-space: nowrap; font-size: 0.9375rem; font-weight: 600; }
|
|
152
|
+
/* What pressing them got you, under the words rather than under the buttons. */
|
|
153
|
+
.coldwire-archive-meta { margin-top: 0.4rem; font-size: 0.8125rem; }
|
|
125
154
|
/* Under the filter, because it answers what the filter asks. */
|
|
126
155
|
.coldwire-count { margin: 0 0 0.6rem; }
|
|
127
156
|
|
|
@@ -200,5 +229,6 @@
|
|
|
200
229
|
.coldwire-pulse { animation: none; }
|
|
201
230
|
.coldwire-switch span,
|
|
202
231
|
.coldwire-switch span::after,
|
|
232
|
+
.coldwire-sync > summary::after,
|
|
203
233
|
.coldwire-inspect-label::after { transition: none; }
|
|
204
234
|
}
|
|
@@ -0,0 +1,56 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require "active_support/core_ext/object/blank"
|
|
4
|
+
require "active_support/core_ext/array/wrap"
|
|
5
|
+
|
|
6
|
+
module Coldwire
|
|
7
|
+
class << self
|
|
8
|
+
# The precache manifest, as the worker needs it. A bare URL is a page; a Hash can also name
|
|
9
|
+
# the Turbo frame it is loaded into, or the format it is fetched as, which are the two
|
|
10
|
+
# things a URL alone cannot say:
|
|
11
|
+
#
|
|
12
|
+
# sync.precache_urls = -> {
|
|
13
|
+
# Feature.published.map { |f| { url: feature_path(f), frame: "map_feature_popup" } } +
|
|
14
|
+
# Report.all.map { |r| { url: report_path(r), format: :json } }
|
|
15
|
+
# }
|
|
16
|
+
#
|
|
17
|
+
# Listing one URL twice is how you ask for both the page and the frame, so nothing here
|
|
18
|
+
# collapses them.
|
|
19
|
+
def precache_entries(list)
|
|
20
|
+
Array.wrap(list).map { |entry| precache_entry(entry) }
|
|
21
|
+
end
|
|
22
|
+
|
|
23
|
+
private
|
|
24
|
+
|
|
25
|
+
def precache_entry(entry)
|
|
26
|
+
return { url: entry.to_s } unless entry.is_a?(Hash)
|
|
27
|
+
|
|
28
|
+
entry = entry.transform_keys(&:to_sym)
|
|
29
|
+
url = entry[:url].to_s
|
|
30
|
+
raise ArgumentError, "Coldwire precache_urls entries need a url: #{entry.inspect}" if url.empty?
|
|
31
|
+
|
|
32
|
+
{ url: url, frame: entry[:frame].presence&.to_s, accept: precache_accept(entry) }.compact
|
|
33
|
+
end
|
|
34
|
+
|
|
35
|
+
# A format becomes the Accept header the worker asks with, resolved here rather than in the
|
|
36
|
+
# worker because Rails already knows what every registered format means — including any the
|
|
37
|
+
# app registered itself.
|
|
38
|
+
#
|
|
39
|
+
# `format: :json` is the friendly way to say it and `accept:` is the way out when a media
|
|
40
|
+
# type has no registered name. Naming a format that is not registered is a typo worth
|
|
41
|
+
# refusing: it would otherwise be fetched as HTML and cached as the page.
|
|
42
|
+
def precache_accept(entry)
|
|
43
|
+
return entry[:accept].to_s if entry[:accept].present?
|
|
44
|
+
return nil if entry[:format].blank?
|
|
45
|
+
|
|
46
|
+
mime = Mime[entry[:format]]
|
|
47
|
+
unless mime
|
|
48
|
+
raise ArgumentError,
|
|
49
|
+
"Coldwire precache_urls format #{entry[:format].inspect} is not a registered Mime " \
|
|
50
|
+
"type. Register it with `Mime::Type.register`, or give `accept:` instead."
|
|
51
|
+
end
|
|
52
|
+
|
|
53
|
+
mime.to_s
|
|
54
|
+
end
|
|
55
|
+
end
|
|
56
|
+
end
|
|
@@ -3,18 +3,79 @@
|
|
|
3
3
|
// fetched again until one returns. So a sweep proves the network first, and only then takes
|
|
4
4
|
// what nothing has asked for in a long time — and, if the cache is still over its ceiling,
|
|
5
5
|
// whatever has gone longest unread until it fits.
|
|
6
|
+
//
|
|
7
|
+
// Somebody choosing a ceiling on the settings page is the exception, and applyCeiling() below
|
|
8
|
+
// is where it is made.
|
|
6
9
|
let collecting = null
|
|
7
10
|
|
|
11
|
+
// One pass over the cache at a time. Two of them deleting at once would each be measuring a
|
|
12
|
+
// total the other is still taking from, and both would stop early.
|
|
13
|
+
function track(run) {
|
|
14
|
+
collecting = run
|
|
15
|
+
run.catch(() => {}).then(() => { if (collecting === run) collecting = null })
|
|
16
|
+
|
|
17
|
+
return run
|
|
18
|
+
}
|
|
19
|
+
|
|
8
20
|
// The ceiling is a per-device choice, so it arrives with the request rather than being baked
|
|
9
21
|
// into the worker. Undefined is a page that has nothing to say about it — an older client, or
|
|
10
22
|
// one that never loaded the collector — and falls back to what the app configured.
|
|
11
23
|
function collectGarbage({ maxSize } = {}) {
|
|
12
24
|
const limit = maxSize === undefined ? COLLECT_MAX_SIZE : maxSize
|
|
13
|
-
// A sweep already running is the answer to this one too
|
|
14
|
-
//
|
|
15
|
-
|
|
25
|
+
// A sweep already running is the answer to this one too: it is doing the same automatic
|
|
26
|
+
// work, and a sweep is cheap to be late.
|
|
27
|
+
return collecting || track(runCollection(limit))
|
|
28
|
+
}
|
|
29
|
+
|
|
30
|
+
// Somebody has just set the ceiling and is watching the number under it. Two things separate
|
|
31
|
+
// this from a sweep:
|
|
32
|
+
//
|
|
33
|
+
// It does not prove the connection first. The probe is there because nothing asked for an
|
|
34
|
+
// automatic sweep, so the cost of getting it wrong falls on somebody who never requested it.
|
|
35
|
+
// This was requested: it is the same deliberate instruction Clear cache is, and that has
|
|
36
|
+
// never waited for a network to agree. Refusing until the connection returns would answer a
|
|
37
|
+
// question nobody asked.
|
|
38
|
+
//
|
|
39
|
+
// It never joins a run already in flight. That run is working to the ceiling this call
|
|
40
|
+
// replaces, so its answer is the answer to the old question — the very thing that made the
|
|
41
|
+
// setting look like it did nothing. It queues behind instead.
|
|
42
|
+
function applyCeiling(maxSize) {
|
|
43
|
+
const queued = collecting ? collecting.catch(() => {}) : Promise.resolve()
|
|
44
|
+
|
|
45
|
+
return track(queued.then(() => runTrim(maxSize)))
|
|
46
|
+
}
|
|
47
|
+
|
|
48
|
+
// Only the ceiling. Age collection stays behind the probe, because nothing has asked for it.
|
|
49
|
+
async function runTrim(maxSize) {
|
|
50
|
+
if (maxSize === undefined || maxSize === null) return { ok: true, evicted: 0, trimmed: false }
|
|
16
51
|
|
|
17
|
-
|
|
52
|
+
const cache = await caches.open(CACHE_NAME)
|
|
53
|
+
const { evicted, bytes } = await trimToSize(cache, await collectable(cache), maxSize)
|
|
54
|
+
|
|
55
|
+
return { ok: true, evicted, bytes, trimmed: true, finishedAt: Date.now() }
|
|
56
|
+
}
|
|
57
|
+
|
|
58
|
+
// Everything a sweep is allowed to take, with the stamp it sorts by.
|
|
59
|
+
async function collectable(cache) {
|
|
60
|
+
const spared = offlinePageAssets()
|
|
61
|
+
|
|
62
|
+
return (await cache.keys())
|
|
63
|
+
.filter((key) => !isSpared(key, spared))
|
|
64
|
+
.map((key) => ({ key, at: unixTimestamp(key.headers.get(TIMESTAMP_HEADER)) }))
|
|
65
|
+
}
|
|
66
|
+
|
|
67
|
+
// The URLs a sweep may never take, for the settings page.
|
|
68
|
+
//
|
|
69
|
+
// The page has to add up the same bytes the worker does or its bar lies: counting files the
|
|
70
|
+
// worker is not allowed to touch shows an overage that no amount of trimming will ever bring
|
|
71
|
+
// down, and the setting reads as broken when it is working exactly as told. The page knows
|
|
72
|
+
// about downloads by their query, but the offline page's own assets it cannot know, because
|
|
73
|
+
// working them out means parsing the offline page — which only the worker holds.
|
|
74
|
+
//
|
|
75
|
+
// The list, rather than the total: the page has already read every entry and its size to draw
|
|
76
|
+
// the list, so this is all it is missing, and one message beats a second pass over the cache.
|
|
77
|
+
async function sparedUrls() {
|
|
78
|
+
return { ok: true, urls: [ ...offlinePageAssets() ] }
|
|
18
79
|
}
|
|
19
80
|
|
|
20
81
|
async function runCollection(maxSize) {
|
|
@@ -149,5 +210,10 @@ async function renew(cache, key) {
|
|
|
149
210
|
if (!response) return
|
|
150
211
|
|
|
151
212
|
const managed = key.headers.get(MANAGED_HEADER) === "1"
|
|
152
|
-
|
|
213
|
+
// Put it back where it was. Rebuilt from the URL alone, a frame entry would lose the frame
|
|
214
|
+
// it names and land on its page's key, which is the collision this all exists to avoid.
|
|
215
|
+
const params = new URL(key.url).searchParams
|
|
216
|
+
const frame = params.get(FRAME_PARAM)
|
|
217
|
+
const format = params.get(FORMAT_PARAM)
|
|
218
|
+
await putFresh(cache, cacheKey(new Request(key.url, { method: "GET" }), { managed, frame, format }), response)
|
|
153
219
|
}
|
|
@@ -75,6 +75,13 @@ self.addEventListener("message", (event) => {
|
|
|
75
75
|
// rides along, because it is the device's choice and the worker cannot read localStorage.
|
|
76
76
|
if (type === "collect") return reply(collectGarbage(event.data))
|
|
77
77
|
|
|
78
|
+
// The ceiling, applied the moment somebody picks one. Deliberate, so unlike a sweep it
|
|
79
|
+
// does not stand down without a connection.
|
|
80
|
+
if (type === "trim") return reply(applyCeiling(event.data.maxSize))
|
|
81
|
+
|
|
82
|
+
// What the page must leave out of its own tally to agree with what a sweep can take.
|
|
83
|
+
if (type === "spared") return reply(sparedUrls())
|
|
84
|
+
|
|
78
85
|
// What a page missed by not listening yet, and whether it is still going on.
|
|
79
86
|
if (type === "syncState") return reply({ ok: true, running: Boolean(syncing), last: lastSyncMessage })
|
|
80
87
|
|
|
@@ -46,8 +46,14 @@ async function describeCached(request, response) {
|
|
|
46
46
|
async function entrySize(response) {
|
|
47
47
|
if (!response) return 0
|
|
48
48
|
|
|
49
|
-
|
|
50
|
-
|
|
49
|
+
// `get` answers null for a missing header, and Number(null) is 0 — which is finite, and not
|
|
50
|
+
// negative, so it sails through the guard and reports the entry as weighing nothing. Rails
|
|
51
|
+
// sends a great deal of HTML chunked, with no Content-Length at all, so this is not an edge
|
|
52
|
+
// case: it is most pages. A collector measuring them at zero never reaches its ceiling and
|
|
53
|
+
// quietly deletes nothing, reporting success the whole time.
|
|
54
|
+
const declared = response.headers.get("Content-Length")
|
|
55
|
+
const bytes = declared === null ? NaN : Number(declared)
|
|
56
|
+
if (Number.isFinite(bytes) && bytes >= 0) return bytes
|
|
51
57
|
|
|
52
58
|
return (await response.clone().blob()).size
|
|
53
59
|
}
|
|
@@ -61,18 +67,111 @@ async function entrySize(response) {
|
|
|
61
67
|
// would find the entry either way, but every distinct query string would still write its own
|
|
62
68
|
// copy — a map that rewrites lat/lng/zoom on each pan would bury the cache in near-duplicates
|
|
63
69
|
// of one page.
|
|
64
|
-
|
|
70
|
+
// `frame` names the Turbo frame this response is a body for, and is what keeps a frame out of
|
|
71
|
+
// the slot its page occupies. Defaults to whatever the request asked for; passed explicitly
|
|
72
|
+
// where the caller knows better — a frame request answered with a whole document is a page,
|
|
73
|
+
// and renewing an entry has to put it back where it already was.
|
|
74
|
+
function cacheKey(request, { managed = false, frame = request.headers.get("Turbo-Frame"), format = formatOf(request) } = {}) {
|
|
65
75
|
const headers = new Headers(request.headers)
|
|
66
76
|
headers.set(TIMESTAMP_HEADER, String(Math.floor(Date.now() / 1000)))
|
|
67
77
|
if (managed) headers.set(MANAGED_HEADER, "1")
|
|
68
78
|
|
|
69
79
|
// `new Request(request, init)` downgrades a navigation request's mode for us; rebuilding
|
|
70
|
-
// from a URL string needs the method stated explicitly.
|
|
71
|
-
|
|
80
|
+
// from a URL string needs the method stated explicitly. Only available while the URL is
|
|
81
|
+
// being kept as it is, which naming a variant is not: taking this path with a format to
|
|
82
|
+
// write stored a JSON body under the page's own key, which is the whole bug in miniature.
|
|
83
|
+
const named = Boolean(frame) || (format && format !== "page")
|
|
84
|
+
if (!IGNORE_SEARCH && !named) return new Request(request, { headers })
|
|
72
85
|
|
|
73
|
-
|
|
74
|
-
|
|
75
|
-
|
|
86
|
+
return new Request(variantUrl(request.url, { frame, format }), { method: "GET", headers })
|
|
87
|
+
}
|
|
88
|
+
|
|
89
|
+
// Where a body of this kind lives. Built in one place because it is written by cacheKey and
|
|
90
|
+
// looked for by matchStored, and a frame stored under a URL nothing asks for is a frame that
|
|
91
|
+
// never answers.
|
|
92
|
+
function variantUrl(url, { frame = null, format = null } = {}) {
|
|
93
|
+
const target = new URL(url)
|
|
94
|
+
if (IGNORE_SEARCH) target.search = ""
|
|
95
|
+
if (frame) target.searchParams.set(FRAME_PARAM, frame)
|
|
96
|
+
// A page is the default and says nothing, which is what leaves every entry stored before
|
|
97
|
+
// formats existed exactly where it was.
|
|
98
|
+
if (format && format !== "page") target.searchParams.set(FORMAT_PARAM, format)
|
|
99
|
+
|
|
100
|
+
return target.href
|
|
101
|
+
}
|
|
102
|
+
|
|
103
|
+
// The format a request negotiated for, as a short token.
|
|
104
|
+
//
|
|
105
|
+
// Read from the request rather than from the response, which is the opposite of what it should
|
|
106
|
+
// be and is forced: the same derivation has to run when the entry is looked for again, and at
|
|
107
|
+
// that moment there is no response to read. With ignore_query_params off a named key can only
|
|
108
|
+
// be found by building its name, so a name the request cannot produce is a name nothing ever
|
|
109
|
+
// finds.
|
|
110
|
+
//
|
|
111
|
+
// A page is the default and is left unnamed, which is what leaves every entry cached before
|
|
112
|
+
// formats existed exactly where it was.
|
|
113
|
+
function formatOf(request) {
|
|
114
|
+
const token = negotiatedFormat(request)
|
|
115
|
+
if (token === "page") return "page"
|
|
116
|
+
|
|
117
|
+
// A URL that already names its format has nothing to disambiguate. "/report.json" is JSON and
|
|
118
|
+
// nothing else, where "/report" is a page, a JSON body and a CSV depending on who asks, so the
|
|
119
|
+
// param there would be a second way of saying what the path already said.
|
|
120
|
+
//
|
|
121
|
+
// Only where the extension agrees with what was asked for. "/sites/acme.com" is a page whose
|
|
122
|
+
// last segment happens to contain a dot, and ".com" says nothing about a format — so that one
|
|
123
|
+
// keeps its param and stays apart from the JSON at the same URL.
|
|
124
|
+
return extensionFormat(request.url) === token ? "page" : token
|
|
125
|
+
}
|
|
126
|
+
|
|
127
|
+
// What the path itself declares, if anything. Unknown extensions say nothing and keep their
|
|
128
|
+
// param, which is noisier than it needs to be and never wrong.
|
|
129
|
+
const EXTENSION_FORMATS = {
|
|
130
|
+
json: "json", geojson: "json", css: "css", xml: "xml", rss: "xml", atom: "xml",
|
|
131
|
+
csv: "csv", ics: "calendar", pdf: "pdf", txt: "plain", md: "markdown",
|
|
132
|
+
png: "image", jpg: "image", jpeg: "image", gif: "image", webp: "image", avif: "image",
|
|
133
|
+
svg: "image", ico: "image"
|
|
134
|
+
}
|
|
135
|
+
|
|
136
|
+
function extensionFormat(url) {
|
|
137
|
+
const name = new URL(url).pathname.split("/").pop() || ""
|
|
138
|
+
const dot = name.lastIndexOf(".")
|
|
139
|
+
if (dot < 1) return "page"
|
|
140
|
+
|
|
141
|
+
return EXTENSION_FORMATS[name.slice(dot + 1).toLowerCase()] || "page"
|
|
142
|
+
}
|
|
143
|
+
|
|
144
|
+
function negotiatedFormat(request) {
|
|
145
|
+
const accept = (request.headers.get("Accept") || "").split(",")[0].split(";")[0].trim().toLowerCase()
|
|
146
|
+
|
|
147
|
+
// Nothing definite asked for. A precache, a fetch that set no Accept, a browser asking for a
|
|
148
|
+
// script or a font: all of them get the unnamed key, as they always have.
|
|
149
|
+
if (!accept || accept === "*/*" || accept.includes("html")) return "page"
|
|
150
|
+
|
|
151
|
+
const [ top, sub = "" ] = accept.split("/")
|
|
152
|
+
// image/avif and image/webp are one question asked two ways, and which one a browser puts
|
|
153
|
+
// first is not a distinction worth a second copy. The family is the answer.
|
|
154
|
+
if (top && top !== "text" && top !== "application") return top
|
|
155
|
+
|
|
156
|
+
// "application/vnd.api+json" is JSON. "text/csv" is csv.
|
|
157
|
+
const parts = sub.split("+")
|
|
158
|
+
const token = (parts.length > 1 ? parts[parts.length - 1] : parts[0]) || ""
|
|
159
|
+
|
|
160
|
+
return token.replace(/[^a-z0-9.-]/g, "") || "page"
|
|
161
|
+
}
|
|
162
|
+
|
|
163
|
+
// What kind of body an entry holds, read back off its key. Documents have none of these
|
|
164
|
+
// params and answer to null, which is what an ordinary visit asks for.
|
|
165
|
+
function variantOf(key) {
|
|
166
|
+
const params = new URL(key.url).searchParams
|
|
167
|
+
// Downloads are their own thing entirely and must never answer a page request, which
|
|
168
|
+
// ignoreSearch would otherwise let them do.
|
|
169
|
+
if (params.has(CHUNK_PARAM) || params.has(RANGE_PARAM)) return "download"
|
|
170
|
+
|
|
171
|
+
const frame = params.get(FRAME_PARAM)
|
|
172
|
+
if (frame) return `frame:${frame}`
|
|
173
|
+
|
|
174
|
+
return params.get(FORMAT_PARAM) || "page"
|
|
76
175
|
}
|
|
77
176
|
|
|
78
177
|
function unixTimestamp(value) {
|
|
@@ -96,5 +96,9 @@ function isAutoCacheable(request) {
|
|
|
96
96
|
// network error by spec, so the app fails to launch offline rather than showing the
|
|
97
97
|
// cached page. Never store one.
|
|
98
98
|
function isCacheable(request, response) {
|
|
99
|
-
|
|
99
|
+
if (request.method !== "GET" || !response.ok || response.redirected) return false
|
|
100
|
+
|
|
101
|
+
// A stream is a list of changes to make to a page, not a page. Stored, it would sit in the
|
|
102
|
+
// slot the page occupies and be replayed later against a DOM it was never written for.
|
|
103
|
+
return !(response.headers.get("Content-Type") || "").includes(STREAM_TYPE)
|
|
100
104
|
}
|