coldwire-rails 0.1.0 → 0.2.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: f01aeddab9f97e903edaa61b6e17a674359cf508633835ed5df8d3fd77145fd8
-  data.tar.gz: e4b438c9a706473e652121ea5fd474057d5d93d42213a85846c28449e14e2f5e
+  metadata.gz: 268be717f39cc2dcb3927c27c22256afcc81981facc029f9be277ef9c22c1ee0
+  data.tar.gz: 03cc56cf120523536b582180dcb8366fc9194ca3ada5dca136abd44761cf7afb
 SHA512:
-  metadata.gz: bd80ebb768d19c935f3426c7b48649c562ae8518f56c3539866ab579cc5eeee2caa04097c4c96144c8591c64a8c34857f966cef4dea4c8538b94a33c8a0d8a0f
-  data.tar.gz: 67b95dae87ed9c8bce9301911c85e6fdd26f7520836fcdcd73d47b9470424ae568c0ad1c1e4f4a9a8855e99c412d1f1321c4d71cab5ed7d0755a243e1242a581
+  metadata.gz: d2bf7e4ef38d0b7f3e25483995663263468f092904810a9aa48186c0dff1abfd5429890ff0aadc628fd14730d287da327e05daac39d07d92f0c66507f10fc4a2
+  data.tar.gz: edea984f52957b16f2fd2f0664c9bcf262cf58735679b646b6c345730a4df65524f36e3a4fa3a8ff32ed17a697f0b030330ee670ac860a3fc0f3d06d6aefa120

data/CHANGELOG.md

@@ -1,12 +1,26 @@
 # Changelog
 
-## [Unreleased]
+## [0.1.0]
 
 First release. The API may still change before 1.0.
 
 - **`bin/rails coldwire:install`.** Mounts the engine at `/offline`, writes an initializer
   with every option and its default, registers the Stimulus controller, and tags the
   layout. Safe to run twice.
+- **Garbage collection.** `config.garbage_collection` sweeps entries nothing has used in
+  `max_age` (60 days by default) and, once the cache is over `max_size` (250 MB by default),
+  the least recently read of what is left until it fits — so a cache that fills as people
+  browse does not fill forever, on a device that revisits nothing as much as on one that
+  revisits everything. The ceiling is offered as a ladder of sizes on the offline settings
+  page and remembered per device, since how much of a phone to spend is not something an app
+  can know. It measures only what a sweep may take: downloaded archives are an opt-in spend
+  of somebody's data plan, so they are neither counted nor evicted.
+  On by default, unlike syncing: it spends no data. A sweep runs only with a
+  connection it has confirmed by pinging `probe_path`, because deleting is the one cache
+  operation with no way back. Age is measured from last use, not from when an entry was
+  fetched — storing a page renews everything it names, so the stylesheet every page loads
+  keeps a fresh date even though nothing ever refetches it. The offline page's own assets and
+  downloaded archives are never collected.
 - **Offline support switch** in the status header on the offline settings page. Off deletes
   what is stored, unregisters the worker, and hides the rest of the page.
   `config.caching_enabled_by_default` is the starting position of the switch (on). A device

data/README.md

@@ -1,44 +1,45 @@
-# Coldwire
+<picture>
+  <source media="(prefers-color-scheme: dark)" srcset="docs/images/coldwire-logo-dark.svg">
+  <img src="docs/images/coldwire-logo.svg" alt="Coldwire" width="380">
+</picture>
 
 **When your Hotwire wires go cold.**
 
-Offline caching for Rails. Add a gem, mount it, drop a tag in your layout — pages start
-working without a network. A service worker precaches what you nominate, keeps pages as
-people browse, and falls back to a view [Turbo](https://turbo.hotwired.dev) will actually
-render.
+[![Gem](https://img.shields.io/gem/v/coldwire-rails.svg)](https://rubygems.org/gems/coldwire-rails)
+[![Gem](https://img.shields.io/gem/dt/coldwire-rails.svg)](https://rubygems.org/gems/coldwire-rails)
 
-Works the same in a plain Hotwire app, an installed PWA, or Hotwire Native.
 
-> **Status: early.** Extracted from a production app but young as a library. The API may
-> change before 1.0.
+Offline caching for Rails, extracted from a production app. Add a gem, mount it, drop a
+tag in your layout — pages start working without a network. A service worker precaches
+what you nominate, keeps pages as people browse, and falls back to a view
+[Turbo](https://turbo.hotwired.dev) will actually render.
+
+Works the same in a plain  web app, an installed PWA, or Hotwire Native.
 
 > [!TIP]
+> <a href="https://noreastergroup.com"><img src="docs/images/noreaster-group.png" alt="Noreaster Group" width="220"></a>
+>
 > **Need help going offline?** Coldwire is built by [Noreaster Group](https://noreastergroup.com).
 > If you want a hand adding offline to your Hotwire or Hotwire Native app,
 > [talk to us](https://noreastergroup.com).
 
 ## What you get
 
-- **Minutes to wire.** `bin/rails coldwire:install`. The rest has working defaults.
-- **Precaching in Ruby.** Nominate URLs with your own route helpers; assets come with them.
-- **Cache as you go.** Pages someone visits are kept, with the styles, scripts, and images
-  they need to render.
-- **An offline fallback** Turbo — and Hotwire Native — will actually show.
-- **Offline settings** at `/offline`: turn offline support off, force offline, sync, downloads, what's cached.
-- **Safe when people sign in.** The cache drops when the user changes.
-
-<p align="center">
-  <img src="docs/images/offline-settings.png" alt="Offline settings: status, force offline, auto sync, and downloads" width="280">
-  <img src="docs/images/offline-settings-cached.png" alt="Offline settings: every cached entry, with search, sort, and delete" width="280">
-</p>
-
-<p align="center"><em>The offline settings page that ships with it. The URL list is under Inspect cache.</em></p>
+- **[Setup in 30 seconds](docs/setup.md):** Instal gem, then `bin/rails coldwire:install`. The rest has working defaults.
+- **[Cache as you go](docs/configuration.md#cache_as_you_go):** Defaults to all pages get cached as your navigate, customize to only cache certain parts of the app.
+- **[Auto Sycn precaching](docs/configuration.md#auto_sync):** Define urls that can be pre-cached automatically in the background.
+- **[Garbage collection](docs/configuration.md#garbage_collection):** Entries nothing has used in two months are swept, and anything over the storage ceiling goes least recently read first — so the cache doesn't grow forever. Only with a connection; anything a page still loads is renewed on the way past. People can set the ceiling for their own device on `/offline`.
+- **[Offline fallback page](docs/setup.md#the-offline-page):** A customizable page when a user is offline and no page is cached for that given url.
+- **[Offline settings page](docs/setup.md#the-offline-settings-page):** at `/offline`: turn offline support off, force offline, sync, downloads, inspect what's cached.
+- **[Easy Tailwind Variants](docs/configuration.md#tailwind-variants):** Easily show or hide content with an `online:` or `offline:` Tailwind variant.
 
 <p align="center">
-  <img src="docs/images/offline-fallback.png" alt="The offline fallback: You're offline. This page isn't available offline. Reconnect and try again." width="280">
+  <img src="docs/images/offline-settings.png" alt="Offline settings: status, force offline, the storage limit, auto sync, and downloads" width="240">
+  <img src="docs/images/offline-settings-cached.png" alt="Offline settings: every cached entry, with search, sort, and delete" width="240">
+  <img src="docs/images/offline-fallback.png" alt="The offline fallback: You're offline. This page isn't available offline. Reconnect and try again." width="240">
 </p>
 
-<p align="center"><em>When a page isn't cached and there's no network, this is what people see — not a blank screen or a native error. Override it to match your app.</em></p>
+<p align="center"><em>The offline settings page (Inspect cache is the URL list), and the fallback when a page isn't cached.</em></p>
 
 ## Quick start
 
@@ -55,8 +56,9 @@ That mounts the engine at `/offline`, writes `config/initializers/coldwire.rb`,
 the Stimulus controller, and adds the tag to your layout. Visit `/offline` to see what's
 cached.
 
-For signed-in apps, set `cache_identity`. To have pages ready before anyone visits them,
-turn on `auto_sync`. Both are in the [configuration reference](docs/configuration.md).
+For signed-in apps that do not use `current_user` or `Current.user`, set `cache_identity`.
+To have pages ready before anyone visits them, turn on `auto_sync`. Both are in the
+[configuration reference](docs/configuration.md).
 
 ## Docs
 

data/VERSION

@@ -1 +1 @@
-0.1.0
+0.2.0

data/app/assets/javascripts/coldwire/cache_controller.js

@@ -1,8 +1,8 @@
 import { Controller } from "@hotwired/stimulus"
-import { formatBytes, formatCachedAt, formatDuration, formatInterval, displayUrl, plural } from "coldwire/format"
+import { formatBytes, formatLimit, formatCachedAt, formatDuration, formatInterval, displayUrl, plural } from "coldwire/format"
 import { sendToWorker } from "coldwire/worker"
 import { renderArchiveStatus, renderArchiveProgress, toggleArchiveBusy } from "coldwire/archives"
-import { describeEntry, describeCached, describeFinishedSync } from "coldwire/entries"
+import { describeEntry, describeCached, describeFinishedSync, isDownload } from "coldwire/entries"
 
 // Storage goes through the one wrapper the head snippet defines, so the page and the snippet
 // cannot disagree about where anything is kept. Without that snippet there is no service
@@ -15,7 +15,8 @@ const INERT_STORE = {
   number: () => 0,
   on: () => false,
   toggle: () => {},
-  cachingOn: () => true
+  cachingOn: () => true,
+  maxSize: () => null
 }
 const SYNC_MESSAGE = "coldwire:sync"
 
@@ -47,6 +48,10 @@ export default class extends Controller {
     "search",
     "sort",
     "inspect",
+    "maxSize",
+    "usage",
+    "usageBar",
+    "usageLabel",
     "forgetTemplate",
     "detail",
     "detailUrl",
@@ -79,6 +84,7 @@ export default class extends Controller {
     this.restoreInspect()
     this.restoreForced()
     this.restoreAutoSync()
+    this.restoreMaxSize()
     if (this.cachingOn()) {
       this.renderArchives()
       this.refresh()
@@ -746,11 +752,12 @@ export default class extends Controller {
       (cache.entries || []).map((entry) => {
         const path = displayUrl(entry.url)
 
-        return { ...entry, cache: cache.name, path, search: path.toLowerCase() }
+        return { ...entry, cache: cache.name, path, search: path.toLowerCase(), download: isDownload(entry.url) }
       }))
     this.cacheCount = cachesInfo.length
 
     this.renderEntries()
+    this.renderStorage()
   }
 
   filterEntries() {
@@ -792,6 +799,111 @@ export default class extends Controller {
     return entries.reduce((sum, entry) => sum + (entry.size || 0), 0)
   }
 
+  // MARK: the storage ceiling
+
+  // How much the cache may grow to before a sweep starts taking the oldest back, chosen here
+  // and remembered on the device. The app sets the starting position; this is the person
+  // holding the phone deciding how much of it to spend.
+  restoreMaxSize() {
+    if (!this.hasMaxSizeTarget) return
+
+    const limit = this.store.maxSize()
+    const value = limit === null ? "none" : String(limit)
+    const options = [ ...this.maxSizeTarget.options ]
+
+    // A ceiling this device chose before the app changed its own is no longer on the menu.
+    // Showing a blank select instead would misreport the setting that is actually in force,
+    // so it joins the list — in order, and labelled from the same figures as the rest.
+    if (!options.some((option) => option.value === value)) {
+      const option = document.createElement("option")
+      option.value = value
+      option.textContent = formatLimit(limit)
+      const after = options.find((candidate) => Number(candidate.value) > limit)
+      this.maxSizeTarget.insertBefore(option, after || null)
+    }
+
+    this.maxSizeTarget.value = value
+  }
+
+  async changeMaxSize(event) {
+    const select = event.currentTarget
+    this.store.set(this.store.keys.maxSize, select.value)
+    this.renderStorage()
+
+    // Held while the sweep runs. It is usually a moment, but picking a second size over a
+    // sweep still measuring the cache would leave two runs racing to a different ceiling.
+    select.disabled = true
+
+    try {
+      // A lowered ceiling that waits for the next scheduled sweep reads as a setting that did
+      // nothing. This is that same sweep, run now — and bound by the same rule, so with no
+      // connection it stands down and the line under the bar says why.
+      await this.collectNow()
+    } finally {
+      select.disabled = false
+    }
+  }
+
+  async collectNow() {
+    let result = null
+
+    try {
+      result = await sendToWorker("collect", { maxSize: this.store.maxSize() }, 60000)
+    } catch {
+      // No worker controlling this page yet. The cache is unchanged and the line already
+      // says where it stands.
+      return
+    }
+
+    // Nothing was swept, so the clock stays where it is and the next page load finds the
+    // sweep still due — exactly as the head snippet treats a refused run.
+    this.sweepWaiting = Boolean(result?.offline)
+    if (!this.sweepWaiting) this.store.set(this.store.keys.collectedAt, Date.now())
+
+    await this.renderCache()
+  }
+
+  // What the ceiling is measured against: everything but the downloads, which are an opt-in
+  // spend of somebody's data plan and are never collected however full the cache gets — a
+  // 300 MB archive counted here would show a bar pinned full of files no sweep can touch.
+  //
+  // The worker also spares the offline page's own assets, which this cannot pick out of a
+  // list of URLs. That is a handful of files, and erring towards the larger figure is the
+  // right way round for a number somebody is deciding a limit from.
+  managedBytes(entries) {
+    return this.totalBytes((entries || []).filter((entry) => !entry.download))
+  }
+
+  renderStorage() {
+    if (!this.hasUsageLabelTarget) return
+
+    const limit = this.store.maxSize()
+    const used = this.managedBytes(this.entries)
+
+    if (this.hasUsageTarget) this.usageTarget.hidden = limit === null
+
+    if (limit === null) {
+      this.usageLabelTarget.textContent = `${formatBytes(used)} of cached pages, with no limit set.`
+      return
+    }
+
+    const percent = Math.round(Math.min(used / limit, 1) * 100)
+    if (this.hasUsageBarTarget) this.usageBarTarget.style.width = `${percent}%`
+    if (this.hasUsageTarget) this.usageTarget.setAttribute("aria-valuenow", String(percent))
+
+    const parts = [ `${formatBytes(used)} of ${formatLimit(limit)}` ]
+    if (used > limit) {
+      // Over the ceiling is not a fault and not a promise of instant deletion: a sweep needs
+      // a connection, and says so rather than leaving somebody watching a bar that will not
+      // move.
+      parts.push(this.sweepWaiting || this.online === false
+        ? "over, waiting for a connection to sweep"
+        : "over, the oldest go on the next sweep")
+    }
+
+    this.usageLabelTarget.textContent = `${parts.join(" · ")}.`
+  }
+
   renderEntries() {
     if (!this.hasEntriesTarget) return
 

data/app/assets/javascripts/coldwire/entries.js

@@ -6,6 +6,21 @@ import { formatBytes, formatCachedAt } from "coldwire/format"
 // of its own.
 export const TIMESTAMP_HEADER = "timestamp"
 
+// The query params the worker keys a downloaded archive and its byte ranges under. A download
+// is a deliberate spend of somebody's data plan and is never collected, so anything measuring
+// what a sweep can take has to leave it out.
+const DOWNLOAD_PARAMS = [ "__coldwire_chunk", "__coldwire_range" ]
+
+export function isDownload(href) {
+  try {
+    const url = new URL(href)
+
+    return DOWNLOAD_PARAMS.some((param) => url.searchParams.has(param))
+  } catch {
+    return false
+  }
+}
+
 export function describeEntry(entry) {
   const parts = [ formatBytes(entry.size) ]
   if (entry.timestamp) parts.push(`cached ${formatCachedAt(entry.timestamp)}`)

data/app/assets/javascripts/coldwire/format.js

@@ -37,7 +37,17 @@ export function formatBytes(bytes) {
     const kb = bytes / 1024
     return `${kb < 10 ? kb.toFixed(1) : Math.round(kb)} KB`
   }
-  return `${(bytes / (1024 * 1024)).toFixed(1)} MB`
+  if (bytes < 1024 * 1024 * 1024) return `${(bytes / (1024 * 1024)).toFixed(1)} MB`
+
+  // A storage ceiling is the one figure here that reaches gigabytes, and "2048.0 MB" is a
+  // number somebody has to convert in their head before it means anything.
+  return `${(bytes / (1024 * 1024 * 1024)).toFixed(bytes % (1024 * 1024 * 1024) === 0 ? 0 : 1)} GB`
+}
+
+// A ceiling somebody picked off a menu should read back the way the menu wrote it. Beside a
+// select saying "250 MB", a line saying "250.0 MB" reads as a different number.
+export function formatLimit(bytes) {
+  return formatBytes(bytes).replace(".0 ", " ")
 }
 
 // Built once. Constructing an Intl formatter is expensive, and this is called for every row

data/app/helpers/coldwire/service_worker_helper.rb

@@ -34,6 +34,10 @@ module Coldwire
         workerPath: coldwire.service_worker_path,
         workerScope: Coldwire.config.worker_scope,
         syncInterval: Coldwire.config.auto_sync.interval.to_i * 1000,
+        collectInterval: Coldwire.config.garbage_collection.interval.to_i * 1000,
+        # The app's ceiling, in bytes. A device that has chosen its own overrides it; without
+        # this the page would have nothing to fall back to when it has not.
+        maxSize: Coldwire.config.garbage_collection.max_size,
         cachingEnabledByDefault: Coldwire.config.caching_enabled_by_default,
         userAgentCookie: USER_AGENT_COOKIE
       }
@@ -45,6 +49,7 @@ module Coldwire
       parts << "marker" if Coldwire.config.mark_cached_pages
       parts += %w[cookie forced]
       parts << "sync" if Coldwire.config.auto_sync.enabled
+      parts << "collect" if Coldwire.config.garbage_collection.enabled
       parts << "register"
     end
 

data/app/views/coldwire/caches/show.html.erb

@@ -106,6 +106,53 @@
   </div>
 
   <div data-coldwire-cache-target="whenOn">
+  <% gc = Coldwire.config.garbage_collection %>
+  <% if gc.enabled %>
+    <%# What the cache is allowed to grow to on this device. The choice is remembered here the
+        way Force offline and Sync on its own are — the app sets the starting point, the person
+        holding the phone decides how much of it to spend. %>
+    <% size_label = ->(bytes) do
+         gigabytes = bytes.to_f / (1024 * 1024 * 1024)
+         next "#{bytes / (1024 * 1024)} MB" if gigabytes < 1
+         "#{gigabytes == gigabytes.round ? gigabytes.round : gigabytes.round(1)} GB"
+       end %>
+
+    <div class="coldwire-card">
+      <h2>Storage</h2>
+
+      <label class="coldwire-choice">
+        <span>
+          <span class="coldwire-setting-title">Keep at most</span>
+          <span class="coldwire-setting-note">
+            Over this, the pages you have not opened in longest are deleted first.
+          </span>
+        </span>
+        <select class="coldwire-select"
+                aria-label="How much storage cached pages may use"
+                data-coldwire-cache-target="maxSize"
+                data-action="change->coldwire-cache#changeMaxSize">
+          <% gc.size_choices.each do |bytes| %>
+            <option value="<%= bytes %>"><%= size_label.call(bytes) %></option>
+          <% end %>
+          <option value="none">No limit</option>
+        </select>
+      </label>
+
+      <div style="margin-top: 0.9rem;"
+           role="progressbar"
+           aria-valuemin="0"
+           aria-valuemax="100"
+           data-coldwire-cache-target="usage"
+           hidden>
+        <div class="coldwire-track">
+          <div class="coldwire-bar" data-coldwire-cache-target="usageBar"></div>
+        </div>
+      </div>
+
+      <p class="coldwire-facts" data-coldwire-cache-target="usageLabel">Reading the cache…</p>
+    </div>
+  <% end %>
+
   <div class="coldwire-card">
     <h2>Auto Sync</h2>
 

data/app/views/coldwire/service_worker/show.js.erb

@@ -44,6 +44,15 @@ const REFETCH_AFTER = <%= raw Coldwire.config.auto_sync.max_age ? Coldwire.confi
 const SYNC_CONCURRENCY = <%= raw Coldwire.config.auto_sync.concurrency.to_i.to_json %>
 // Marks entries the manifest owns, so a sync can retire the ones it dropped without
 // touching assets or pages picked up by ordinary browsing.
+// Collection. A null max age never collects and a null max size never evicts; the probe is
+// how the worker tells a real connection from a hopeful one before it deletes anything. The
+// ceiling here is the app's default — a device that has chosen its own sends that instead.
+<%- gc = Coldwire.config.garbage_collection -%>
+const COLLECT_MAX_AGE = <%= raw gc.enabled && gc.max_age ? gc.max_age.to_i.to_json : "null" %>
+const COLLECT_MAX_SIZE = <%= raw gc.enabled && gc.max_size ? gc.max_size.to_i.to_json : "null" %>
+const RENEW_AFTER = <%= raw gc.renew_after ? gc.renew_after.to_json : "null" %>
+const PROBE_PATH = <%= raw Coldwire.config.probe_path.to_json %>
+
 const MANAGED_HEADER = "coldwire-managed"
 // One message type for the whole sync, carrying `state`. Two types would mean the page
 // stamping the clock and the page drawing progress could drift apart.

data/docs/README.md

@@ -4,7 +4,7 @@ Setup and configuration for [Coldwire](../README.md): the mountable Rails engine
 caches pages for Hotwire, PWAs, and Hotwire Native.
 
 <p align="center">
-  <img src="images/offline-settings.png" alt="Offline settings: status, force offline, auto sync, and downloads" width="280">
+  <img src="images/offline-settings.png" alt="Offline settings: status, force offline, the storage limit, auto sync, and downloads" width="280">
   <img src="images/offline-settings-cached.png" alt="Offline settings: every cached entry, with search, sort, and delete" width="280">
 </p>
 
@@ -16,4 +16,5 @@ caches pages for Hotwire, PWAs, and Hotwire Native.
 
 Everything is set in `config/initializers/coldwire.rb` through `Coldwire.configure`.
 `bin/rails coldwire:install` writes that file with every default. Only `auto_sync` really
-needs your attention on a first install; `cache_identity` needs it if anyone signs in.
+needs your attention on a first install; `cache_identity` already uses `current_user` or
+`Current.user` when either is in scope.