coldwire-rails 0.1.0 → 0.3.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: f01aeddab9f97e903edaa61b6e17a674359cf508633835ed5df8d3fd77145fd8
-  data.tar.gz: e4b438c9a706473e652121ea5fd474057d5d93d42213a85846c28449e14e2f5e
+  metadata.gz: e48a15a424535143cc620dbbe65eec4a367c9e1feaa60addc104587ee835c836
+  data.tar.gz: b85fc7cfc8b80db94b14e994afbb258cc997d0b161986738e80c33a8a49704d5
 SHA512:
-  metadata.gz: bd80ebb768d19c935f3426c7b48649c562ae8518f56c3539866ab579cc5eeee2caa04097c4c96144c8591c64a8c34857f966cef4dea4c8538b94a33c8a0d8a0f
-  data.tar.gz: 67b95dae87ed9c8bce9301911c85e6fdd26f7520836fcdcd73d47b9470424ae568c0ad1c1e4f4a9a8855e99c412d1f1321c4d71cab5ed7d0755a243e1242a581
+  metadata.gz: 543c39e2d9999a45ce8041d4c87cb2853908056611bdc189d49e0fde87309373260131f433f92715d29943fcd44eb1d429aa6f270ef639287c0d45ce86787f11
+  data.tar.gz: 31bc9a01f008bd987c79700058b49682cae9d1020e873bf9aab03949fe9d15d9619c420c7b5c9efe7032a6b07eddaa090b54d43adecd9cf2f503983d4e702632

data/CHANGELOG.md

@@ -2,11 +2,42 @@
 
 ## [Unreleased]
 
+## [0.3.0]
+
+- **`cache_origins` is `cacheable_hosts`**, and takes bare hosts: `"tiles.example.com"` rather
+  than `"https://tiles.example.com"`. The scheme was never carrying information — a worker
+  runs only on a secure page, and a secure page cannot fetch `http` — so it was a required
+  prefix with exactly one possible value. Requests are matched on a URL's `host`, so a port
+  belongs where it is not the default and `localhost:3001` matches that port and no other.
+  A scheme raises at boot, naming what to write instead.
+
+## [0.2.0]
+
+- **Garbage collection** is on by default. `config.garbage_collection` periodically sweeps the cache, deleting entries that haven't been accessed for `max_age` (default: 60 days), and if the cache grows over `max_size` (default: 250 MB), it continues pruning the least recently accessed until the cache fits. Sweeps run only when confirmed online by pinging `probe_path`, since deletions are irreversible. Cached archives and offline page assets are never collected, and the age of an entry is renewed anytime it or its referenced subresources are accessed or stored. The size ceiling can be adjusted in the offline settings page and is remembered per device.
+- The garbage collector runs safely and automatically; you generally do not need to configure it. But you can tune `max_age`, `max_size`, and `interval` to fit your app's needs.
+
+
+## [0.1.0]
+
 First release. The API may still change before 1.0.
 
 - **`bin/rails coldwire:install`.** Mounts the engine at `/offline`, writes an initializer
   with every option and its default, registers the Stimulus controller, and tags the
   layout. Safe to run twice.
+- **Garbage collection.** `config.garbage_collection` sweeps entries nothing has used in
+  `max_age` (60 days by default) and, once the cache is over `max_size` (250 MB by default),
+  the least recently read of what is left until it fits — so a cache that fills as people
+  browse does not fill forever, on a device that revisits nothing as much as on one that
+  revisits everything. The ceiling is offered as a ladder of sizes on the offline settings
+  page and remembered per device, since how much of a phone to spend is not something an app
+  can know. It measures only what a sweep may take: downloaded archives are an opt-in spend
+  of somebody's data plan, so they are neither counted nor evicted.
+  On by default, unlike syncing: it spends no data. A sweep runs only with a
+  connection it has confirmed by pinging `probe_path`, because deleting is the one cache
+  operation with no way back. Age is measured from last use, not from when an entry was
+  fetched — storing a page renews everything it names, so the stylesheet every page loads
+  keeps a fresh date even though nothing ever refetches it. The offline page's own assets and
+  downloaded archives are never collected.
 - **Offline support switch** in the status header on the offline settings page. Off deletes
   what is stored, unregisters the worker, and hides the rest of the page.
   `config.caching_enabled_by_default` is the starting position of the switch (on). A device

data/README.md

@@ -1,44 +1,45 @@
-# Coldwire
+<picture>
+  <source media="(prefers-color-scheme: dark)" srcset="docs/images/coldwire-logo-dark.svg">
+  <img src="docs/images/coldwire-logo.svg" alt="Coldwire" width="380">
+</picture>
 
 **When your Hotwire wires go cold.**
 
-Offline caching for Rails. Add a gem, mount it, drop a tag in your layout — pages start
-working without a network. A service worker precaches what you nominate, keeps pages as
-people browse, and falls back to a view [Turbo](https://turbo.hotwired.dev) will actually
-render.
+[![Gem](https://img.shields.io/gem/v/coldwire-rails.svg)](https://rubygems.org/gems/coldwire-rails)
+[![Gem](https://img.shields.io/gem/dt/coldwire-rails.svg)](https://rubygems.org/gems/coldwire-rails)
 
-Works the same in a plain Hotwire app, an installed PWA, or Hotwire Native.
 
-> **Status: early.** Extracted from a production app but young as a library. The API may
-> change before 1.0.
+Offline caching for Rails, extracted from a production app. Add a gem, mount it, drop a
+tag in your layout — pages start working without a network. A service worker precaches
+what you nominate, keeps pages as people browse, and falls back to a view
+[Turbo](https://turbo.hotwired.dev) will actually render.
+
+Works the same in a plain  web app, an installed PWA, or Hotwire Native.
 
 > [!TIP]
+> <a href="https://noreastergroup.com"><img src="docs/images/noreaster-group.png" alt="Noreaster Group" width="220"></a>
+>
 > **Need help going offline?** Coldwire is built by [Noreaster Group](https://noreastergroup.com).
 > If you want a hand adding offline to your Hotwire or Hotwire Native app,
 > [talk to us](https://noreastergroup.com).
 
 ## What you get
 
-- **Minutes to wire.** `bin/rails coldwire:install`. The rest has working defaults.
-- **Precaching in Ruby.** Nominate URLs with your own route helpers; assets come with them.
-- **Cache as you go.** Pages someone visits are kept, with the styles, scripts, and images
-  they need to render.
-- **An offline fallback** Turbo — and Hotwire Native — will actually show.
-- **Offline settings** at `/offline`: turn offline support off, force offline, sync, downloads, what's cached.
-- **Safe when people sign in.** The cache drops when the user changes.
-
-<p align="center">
-  <img src="docs/images/offline-settings.png" alt="Offline settings: status, force offline, auto sync, and downloads" width="280">
-  <img src="docs/images/offline-settings-cached.png" alt="Offline settings: every cached entry, with search, sort, and delete" width="280">
-</p>
-
-<p align="center"><em>The offline settings page that ships with it. The URL list is under Inspect cache.</em></p>
+- **[Setup in 30 seconds](docs/setup.md):** Instal gem, then `bin/rails coldwire:install`. The rest has working defaults.
+- **[Cache as you go](docs/configuration.md#cache_as_you_go):** Defaults to all pages get cached as your navigate, customize to only cache certain parts of the app.
+- **[Auto Sycn precaching](docs/configuration.md#auto_sync):** Define urls that can be pre-cached automatically in the background.
+- **[Garbage collection](docs/configuration.md#garbage_collection):** Entries nothing has used in two months are swept, and anything over the storage ceiling goes least recently read first — so the cache doesn't grow forever. Only with a connection; anything a page still loads is renewed on the way past. People can set the ceiling for their own device on `/offline`.
+- **[Offline fallback page](docs/setup.md#the-offline-page):** A customizable page when a user is offline and no page is cached for that given url.
+- **[Offline settings page](docs/setup.md#the-offline-settings-page):** at `/offline`: turn offline support off, force offline, sync, downloads, inspect what's cached.
+- **[Easy Tailwind Variants](docs/configuration.md#tailwind-variants):** Easily show or hide content with an `online:` or `offline:` Tailwind variant.
 
 <p align="center">
-  <img src="docs/images/offline-fallback.png" alt="The offline fallback: You're offline. This page isn't available offline. Reconnect and try again." width="280">
+  <img src="docs/images/offline-settings.png" alt="Offline settings: status, force offline, the storage limit, auto sync, and downloads" width="240">
+  <img src="docs/images/offline-settings-cached.png" alt="Offline settings: every cached entry, with search, sort, and delete" width="240">
+  <img src="docs/images/offline-fallback.png" alt="The offline fallback: You're offline. This page isn't available offline. Reconnect and try again." width="240">
 </p>
 
-<p align="center"><em>When a page isn't cached and there's no network, this is what people see — not a blank screen or a native error. Override it to match your app.</em></p>
+<p align="center"><em>The offline settings page (Inspect cache is the URL list), and the fallback when a page isn't cached.</em></p>
 
 ## Quick start
 
@@ -55,8 +56,9 @@ That mounts the engine at `/offline`, writes `config/initializers/coldwire.rb`,
 the Stimulus controller, and adds the tag to your layout. Visit `/offline` to see what's
 cached.
 
-For signed-in apps, set `cache_identity`. To have pages ready before anyone visits them,
-turn on `auto_sync`. Both are in the [configuration reference](docs/configuration.md).
+For signed-in apps that do not use `current_user` or `Current.user`, set `cache_identity`.
+To have pages ready before anyone visits them, turn on `auto_sync`. Both are in the
+[configuration reference](docs/configuration.md).
 
 ## Docs
 

data/VERSION

@@ -1 +1 @@
-0.1.0
+0.3.0

data/app/assets/javascripts/coldwire/cache_controller.js

@@ -1,8 +1,8 @@
 import { Controller } from "@hotwired/stimulus"
-import { formatBytes, formatCachedAt, formatDuration, formatInterval, displayUrl, plural } from "coldwire/format"
+import { formatBytes, formatLimit, formatCachedAt, formatDuration, formatInterval, displayUrl, plural } from "coldwire/format"
 import { sendToWorker } from "coldwire/worker"
 import { renderArchiveStatus, renderArchiveProgress, toggleArchiveBusy } from "coldwire/archives"
-import { describeEntry, describeCached, describeFinishedSync } from "coldwire/entries"
+import { describeEntry, describeCached, describeFinishedSync, isDownload } from "coldwire/entries"
 
 // Storage goes through the one wrapper the head snippet defines, so the page and the snippet
 // cannot disagree about where anything is kept. Without that snippet there is no service
@@ -15,7 +15,8 @@ const INERT_STORE = {
   number: () => 0,
   on: () => false,
   toggle: () => {},
-  cachingOn: () => true
+  cachingOn: () => true,
+  maxSize: () => null
 }
 const SYNC_MESSAGE = "coldwire:sync"
 
@@ -47,6 +48,10 @@ export default class extends Controller {
     "search",
     "sort",
     "inspect",
+    "maxSize",
+    "usage",
+    "usageBar",
+    "usageLabel",
     "forgetTemplate",
     "detail",
     "detailUrl",
@@ -79,6 +84,7 @@ export default class extends Controller {
     this.restoreInspect()
     this.restoreForced()
     this.restoreAutoSync()
+    this.restoreMaxSize()
     if (this.cachingOn()) {
       this.renderArchives()
       this.refresh()
@@ -746,11 +752,12 @@ export default class extends Controller {
       (cache.entries || []).map((entry) => {
         const path = displayUrl(entry.url)
 
-        return { ...entry, cache: cache.name, path, search: path.toLowerCase() }
+        return { ...entry, cache: cache.name, path, search: path.toLowerCase(), download: isDownload(entry.url) }
       }))
     this.cacheCount = cachesInfo.length
 
     this.renderEntries()
+    this.renderStorage()
   }
 
   filterEntries() {
@@ -792,6 +799,111 @@ export default class extends Controller {
     return entries.reduce((sum, entry) => sum + (entry.size || 0), 0)
   }
 
+  // MARK: the storage ceiling
+
+  // How much the cache may grow to before a sweep starts taking the oldest back, chosen here
+  // and remembered on the device. The app sets the starting position; this is the person
+  // holding the phone deciding how much of it to spend.
+  restoreMaxSize() {
+    if (!this.hasMaxSizeTarget) return
+
+    const limit = this.store.maxSize()
+    const value = limit === null ? "none" : String(limit)
+    const options = [ ...this.maxSizeTarget.options ]
+
+    // A ceiling this device chose before the app changed its own is no longer on the menu.
+    // Showing a blank select instead would misreport the setting that is actually in force,
+    // so it joins the list — in order, and labelled from the same figures as the rest.
+    if (!options.some((option) => option.value === value)) {
+      const option = document.createElement("option")
+      option.value = value
+      option.textContent = formatLimit(limit)
+      const after = options.find((candidate) => Number(candidate.value) > limit)
+      this.maxSizeTarget.insertBefore(option, after || null)
+    }
+
+    this.maxSizeTarget.value = value
+  }
+
+  async changeMaxSize(event) {
+    const select = event.currentTarget
+    this.store.set(this.store.keys.maxSize, select.value)
+    this.renderStorage()
+
+    // Held while the sweep runs. It is usually a moment, but picking a second size over a
+    // sweep still measuring the cache would leave two runs racing to a different ceiling.
+    select.disabled = true
+
+    try {
+      // A lowered ceiling that waits for the next scheduled sweep reads as a setting that did
+      // nothing. This is that same sweep, run now — and bound by the same rule, so with no
+      // connection it stands down and the line under the bar says why.
+      await this.collectNow()
+    } finally {
+      select.disabled = false
+    }
+  }
+
+  async collectNow() {
+    let result = null
+
+    try {
+      result = await sendToWorker("collect", { maxSize: this.store.maxSize() }, 60000)
+    } catch {
+      // No worker controlling this page yet. The cache is unchanged and the line already
+      // says where it stands.
+      return
+    }
+
+    // Nothing was swept, so the clock stays where it is and the next page load finds the
+    // sweep still due — exactly as the head snippet treats a refused run.
+    this.sweepWaiting = Boolean(result?.offline)
+    if (!this.sweepWaiting) this.store.set(this.store.keys.collectedAt, Date.now())
+
+    await this.renderCache()
+  }
+
+  // What the ceiling is measured against: everything but the downloads, which are an opt-in
+  // spend of somebody's data plan and are never collected however full the cache gets — a
+  // 300 MB archive counted here would show a bar pinned full of files no sweep can touch.
+  //
+  // The worker also spares the offline page's own assets, which this cannot pick out of a
+  // list of URLs. That is a handful of files, and erring towards the larger figure is the
+  // right way round for a number somebody is deciding a limit from.
+  managedBytes(entries) {
+    return this.totalBytes((entries || []).filter((entry) => !entry.download))
+  }
+
+  renderStorage() {
+    if (!this.hasUsageLabelTarget) return
+
+    const limit = this.store.maxSize()
+    const used = this.managedBytes(this.entries)
+
+    if (this.hasUsageTarget) this.usageTarget.hidden = limit === null
+
+    if (limit === null) {
+      this.usageLabelTarget.textContent = `${formatBytes(used)} of cached pages, with no limit set.`
+      return
+    }
+
+    const percent = Math.round(Math.min(used / limit, 1) * 100)
+    if (this.hasUsageBarTarget) this.usageBarTarget.style.width = `${percent}%`
+    if (this.hasUsageTarget) this.usageTarget.setAttribute("aria-valuenow", String(percent))
+
+    const parts = [ `${formatBytes(used)} of ${formatLimit(limit)}` ]
+    if (used > limit) {
+      // Over the ceiling is not a fault and not a promise of instant deletion: a sweep needs
+      // a connection, and says so rather than leaving somebody watching a bar that will not
+      // move.
+      parts.push(this.sweepWaiting || this.online === false
+        ? "over, waiting for a connection to sweep"
+        : "over, the oldest go on the next sweep")
+    }
+
+    this.usageLabelTarget.textContent = `${parts.join(" · ")}.`
+  }
+
   renderEntries() {
     if (!this.hasEntriesTarget) return
 

data/app/assets/javascripts/coldwire/entries.js

@@ -6,6 +6,21 @@ import { formatBytes, formatCachedAt } from "coldwire/format"
 // of its own.
 export const TIMESTAMP_HEADER = "timestamp"
 
+// The query params the worker keys a downloaded archive and its byte ranges under. A download
+// is a deliberate spend of somebody's data plan and is never collected, so anything measuring
+// what a sweep can take has to leave it out.
+const DOWNLOAD_PARAMS = [ "__coldwire_chunk", "__coldwire_range" ]
+
+export function isDownload(href) {
+  try {
+    const url = new URL(href)
+
+    return DOWNLOAD_PARAMS.some((param) => url.searchParams.has(param))
+  } catch {
+    return false
+  }
+}
+
 export function describeEntry(entry) {
   const parts = [ formatBytes(entry.size) ]
   if (entry.timestamp) parts.push(`cached ${formatCachedAt(entry.timestamp)}`)

data/app/assets/javascripts/coldwire/format.js

@@ -37,7 +37,17 @@ export function formatBytes(bytes) {
     const kb = bytes / 1024
     return `${kb < 10 ? kb.toFixed(1) : Math.round(kb)} KB`
   }
-  return `${(bytes / (1024 * 1024)).toFixed(1)} MB`
+  if (bytes < 1024 * 1024 * 1024) return `${(bytes / (1024 * 1024)).toFixed(1)} MB`
+
+  // A storage ceiling is the one figure here that reaches gigabytes, and "2048.0 MB" is a
+  // number somebody has to convert in their head before it means anything.
+  return `${(bytes / (1024 * 1024 * 1024)).toFixed(bytes % (1024 * 1024 * 1024) === 0 ? 0 : 1)} GB`
+}
+
+// A ceiling somebody picked off a menu should read back the way the menu wrote it. Beside a
+// select saying "250 MB", a line saying "250.0 MB" reads as a different number.
+export function formatLimit(bytes) {
+  return formatBytes(bytes).replace(".0 ", " ")
 }
 
 // Built once. Constructing an Intl formatter is expensive, and this is called for every row

data/app/helpers/coldwire/service_worker_helper.rb

@@ -34,6 +34,10 @@ module Coldwire
         workerPath: coldwire.service_worker_path,
         workerScope: Coldwire.config.worker_scope,
         syncInterval: Coldwire.config.auto_sync.interval.to_i * 1000,
+        collectInterval: Coldwire.config.garbage_collection.interval.to_i * 1000,
+        # The app's ceiling, in bytes. A device that has chosen its own overrides it; without
+        # this the page would have nothing to fall back to when it has not.
+        maxSize: Coldwire.config.garbage_collection.max_size,
         cachingEnabledByDefault: Coldwire.config.caching_enabled_by_default,
         userAgentCookie: USER_AGENT_COOKIE
       }
@@ -45,6 +49,7 @@ module Coldwire
       parts << "marker" if Coldwire.config.mark_cached_pages
       parts += %w[cookie forced]
       parts << "sync" if Coldwire.config.auto_sync.enabled
+      parts << "collect" if Coldwire.config.garbage_collection.enabled
       parts << "register"
     end
 

data/app/views/coldwire/caches/show.html.erb

@@ -106,6 +106,53 @@
   </div>
 
   <div data-coldwire-cache-target="whenOn">
+  <% gc = Coldwire.config.garbage_collection %>
+  <% if gc.enabled %>
+    <%# What the cache is allowed to grow to on this device. The choice is remembered here the
+        way Force offline and Sync on its own are — the app sets the starting point, the person
+        holding the phone decides how much of it to spend. %>
+    <% size_label = ->(bytes) do
+         gigabytes = bytes.to_f / (1024 * 1024 * 1024)
+         next "#{bytes / (1024 * 1024)} MB" if gigabytes < 1
+         "#{gigabytes == gigabytes.round ? gigabytes.round : gigabytes.round(1)} GB"
+       end %>
+
+    <div class="coldwire-card">
+      <h2>Storage</h2>
+
+      <label class="coldwire-choice">
+        <span>
+          <span class="coldwire-setting-title">Keep at most</span>
+          <span class="coldwire-setting-note">
+            Over this, the pages you have not opened in longest are deleted first.
+          </span>
+        </span>
+        <select class="coldwire-select"
+                aria-label="How much storage cached pages may use"
+                data-coldwire-cache-target="maxSize"
+                data-action="change->coldwire-cache#changeMaxSize">
+          <% gc.size_choices.each do |bytes| %>
+            <option value="<%= bytes %>"><%= size_label.call(bytes) %></option>
+          <% end %>
+          <option value="none">No limit</option>
+        </select>
+      </label>
+
+      <div style="margin-top: 0.9rem;"
+           role="progressbar"
+           aria-valuemin="0"
+           aria-valuemax="100"
+           data-coldwire-cache-target="usage"
+           hidden>
+        <div class="coldwire-track">
+          <div class="coldwire-bar" data-coldwire-cache-target="usageBar"></div>
+        </div>
+      </div>
+
+      <p class="coldwire-facts" data-coldwire-cache-target="usageLabel">Reading the cache…</p>
+    </div>
+  <% end %>
+
   <div class="coldwire-card">
     <h2>Auto Sync</h2>
 

data/app/views/coldwire/service_worker/show.js.erb

@@ -21,7 +21,7 @@ const CACHE_AS_YOU_GO = compileRules(<%= raw Coldwire.cache_rules(Coldwire.confi
 const NEVER_CACHE = compileRules(<%= raw Coldwire.cache_rules(Coldwire.config.never_cache).to_json %>)
 // Origins besides ours that may be cached at all, and the URLs whose byte ranges are cached
 // piece by piece.
-const CACHE_ORIGINS = <%= raw Coldwire.config.cache_origins.to_json %>
+const CACHEABLE_HOSTS = <%= raw Coldwire.config.cacheable_hosts.to_json %>
 const CACHE_RANGES = compileRules(<%= raw Coldwire.cache_rules(Coldwire.config.cache_ranges).to_json %>)
 // A range entry is stored under its own key, with the range in the query and the archive's
 // total size in a header — the two things needed to rebuild a 206 that was never storable.
@@ -44,6 +44,15 @@ const REFETCH_AFTER = <%= raw Coldwire.config.auto_sync.max_age ? Coldwire.confi
 const SYNC_CONCURRENCY = <%= raw Coldwire.config.auto_sync.concurrency.to_i.to_json %>
 // Marks entries the manifest owns, so a sync can retire the ones it dropped without
 // touching assets or pages picked up by ordinary browsing.
+// Collection. A null max age never collects and a null max size never evicts; the probe is
+// how the worker tells a real connection from a hopeful one before it deletes anything. The
+// ceiling here is the app's default — a device that has chosen its own sends that instead.
+<%- gc = Coldwire.config.garbage_collection -%>
+const COLLECT_MAX_AGE = <%= raw gc.enabled && gc.max_age ? gc.max_age.to_i.to_json : "null" %>
+const COLLECT_MAX_SIZE = <%= raw gc.enabled && gc.max_size ? gc.max_size.to_i.to_json : "null" %>
+const RENEW_AFTER = <%= raw gc.renew_after ? gc.renew_after.to_json : "null" %>
+const PROBE_PATH = <%= raw Coldwire.config.probe_path.to_json %>
+
 const MANAGED_HEADER = "coldwire-managed"
 // One message type for the whole sync, carrying `state`. Two types would mean the page
 // stamping the clock and the page drawing progress could drift apart.

data/docs/README.md

@@ -4,7 +4,7 @@ Setup and configuration for [Coldwire](../README.md): the mountable Rails engine
 caches pages for Hotwire, PWAs, and Hotwire Native.
 
 <p align="center">
-  <img src="images/offline-settings.png" alt="Offline settings: status, force offline, auto sync, and downloads" width="280">
+  <img src="images/offline-settings.png" alt="Offline settings: status, force offline, the storage limit, auto sync, and downloads" width="280">
   <img src="images/offline-settings-cached.png" alt="Offline settings: every cached entry, with search, sort, and delete" width="280">
 </p>
 
@@ -16,4 +16,5 @@ caches pages for Hotwire, PWAs, and Hotwire Native.
 
 Everything is set in `config/initializers/coldwire.rb` through `Coldwire.configure`.
 `bin/rails coldwire:install` writes that file with every default. Only `auto_sync` really
-needs your attention on a first install; `cache_identity` needs it if anyone signs in.
+needs your attention on a first install; `cache_identity` already uses `current_user` or
+`Current.user` when either is in scope.