pmtiles-swarm 0.59.0 → 0.61.0

package/CHANGELOG.md

@@ -7,6 +7,99 @@
 ### 🐞 Bug fixes
 - _...Add new stuff here..._
 
+## 0.61.0
+### ✨ Features and improvements
+- **"Style URL" is now "source URL", because that is what it is.** It goes in a style's
+  `sources` block and is not itself a style, so the old name told a reader to put it in the
+  wrong place. The field on `/api/categories` and `/latest/` is `sourceUrl`; `styleUrl` is
+  still sent alongside it and is deprecated, so nothing reading the old name breaks on the
+  correction.
+
+- **A copy button copies, rather than opening a box to copy from.** `navigator.clipboard`
+  needs a secure context and a console reached by IP over plain HTTP on a LAN is not one —
+  which is how most of them are reached — so the console fell back to `window.prompt` every
+  time. It now falls back to the selection API, which predates the clipboard API and carries
+  no such requirement, and confirms on the button itself the way the public page does. The
+  public page gained the same fallback: it was failing outright wherever the console was
+  prompting, which is the same nodes.
+
+- **The XYZ template is offered on the public catalogue page too**, beside the TileJSON it
+  already had. 0.60.0 added it to the console only, which is the wrong way round: the console
+  is for the operator, and the person who needs a tile URL to paste into a Leaflet layer or a
+  GIS client is usually looking at the public page. Both draw from the same builder, so the
+  field was already in the public payload and only the button was missing.
+
+### 🐞 Bug fixes
+- _...Add new stuff here..._
+
+## 0.60.0
+### ✨ Features and improvements
+- **A tile URL that survives a rebuild.** Every archive is addressed by infohash, which is
+  what makes a tile cacheable for a year and what makes it useless in an application: the URL
+  changes with every build, so anything that wrote one down is pinned to a build that
+  eventually stops existing. A category is the only stable handle this system has, and now it
+  has a tile endpoint of its own — `/latest/<category>/{z}/{x}/{y}.<ext>` — which resolves to
+  whichever build is current on every request.
+
+  The category TileJSON advertises it as `tiles`, so a style written once keeps working across
+  a rebuild without being re-fetched for the URLs alone. The immutable template is still
+  published beside it as `latest.tiles`, because it is still the better URL for anything that
+  can re-read the document: it is content-addressed, so it caches for a year and never
+  revalidates. Offering only one of the two would be choosing for the consumer, and the right
+  answer differs by consumer.
+
+  The console offers the template as a copyable **XYZ** button next to the TileJSON one, which
+  is the form a Leaflet layer, an OpenLayers source or a GIS client actually wants.
+
+  Deliberately not a redirect to the immutable URL, though every other `/latest/` route is one.
+  A redirect costs a round trip and a map asks for hundreds of tiles: what is a negligible
+  indirection for a `.torrent` is the difference between a map that feels immediate and one
+  that does not. It is cached as the moving target it is — `max-age=300, must-revalidate`,
+  tagged with the build it resolved to, so a revalidation is a 304 while that build stands and
+  a miss the moment it moves.
+
+- **A benchmark for "why does this feel slower than the other one", in `tools/tile-bench.mjs`.**
+  It reads two TileJSON documents, picks tiles inside the zoom range and bounds both can serve,
+  and requests the same set from each — one server at a time, because run together they compete
+  for the same link and each measures the other's load as its own latency.
+
+  It reports percentiles rather than an average, since what makes a map feel slow is the tail
+  and a mean built from nineteen fast requests does not move for the twentieth. Time to first
+  byte is separated from the total, which is the difference between a slow lookup and a slow
+  link — they want opposite fixes.
+
+  It also detects a pool of unequal nodes, which is a common cause and an invisible one: half
+  the tiles arrive quickly and half do not, and balanced evenly the mean looks tolerable
+  throughout. Two distinct groups are reported as two, with a histogram, and `--header` tallies
+  a response header naming which backend answered. `--a-origin`/`--b-origin` send the tile
+  requests somewhere other than the document was read from, which is the only way to measure
+  one node directly: a node with `publicUrl` set answers with that name however it was asked.
+
+- **`docs/haproxy.md` covers a pool whose nodes are not the same speed.** Round robin assumes
+  the pool is interchangeable, and a tile server on an NVMe disk and one on a spinning disk are
+  not — an archive read is a seek into a large file, which is what a spinning disk is worst at.
+  `backup`, `weight` and least-connections are compared, along with what each does and does not
+  fix.
+
+### 🐞 Bug fixes
+- **An archive at 100% and seeding could serve no tiles until the node was restarted.** Which
+  source an archive is read through is decided once, when a reader opens it, and every other
+  thing that can change that answer already invalidates the reader: a pause, a resume, a mode
+  change, a move, a finished download. A finished *check* did not — and it is the easiest of
+  them to reach, because during `checking_files` libtorrent reports `progress` as the fraction
+  hashed so far, which is indistinguishable from a download sitting at the same figure.
+
+  So a tile read arriving mid-check opened against the swarm, correctly, and kept that handle
+  afterwards. The archive then read from a swarm whose only member is this node, while the
+  whole file sat on the disk beside it, and nothing evicted the handle short of a restart. The
+  completion sweep looked straight past it: an entry already recorded complete never reaches
+  the code that would have noticed.
+
+  The sweep now drops a reader that is going to the swarm for an archive the disk says is
+  whole. The disk is checked before the handle is dropped rather than after, or a reader that
+  would only re-open against the swarm anyway would be invalidated on every sweep, for ever.
+  Cache mode is left alone: it reads from the swarm because that is what it is for.
+
 ## 0.59.0
 ### ✨ Features and improvements
 - **Requires pmtiles-torrent 0.10.2**, which is what actually ends the re-checking: a

package/docs/configuration.md

@@ -407,7 +407,7 @@ on. This is that address.
 Narrower than `publicUrl` on purpose: `publicUrl` overrides every URL the node
 emits and so gives up the multi-domain behaviour, while this overrides only the
 ones that have to be permanent. Everything else — TileJSON, tile templates,
-`.torrent` links, style URLs, the feeds — goes on naming whichever host the
+`.torrent` links, source URLs, the feeds — goes on naming whichever host the
 request arrived as.
 
 ```json

package/docs/haproxy.md

@@ -114,6 +114,70 @@ gives archive affinity, at the price of concentrating one archive on one node.
 For nodes holding complete copies there is no cold read to avoid, so this is a
 cost with no benefit.
 
+### When the nodes are not the same speed
+
+Round robin assumes the pool is interchangeable, and a tile server on an NVMe
+disk and one on a spinning disk are not. Every archive read is a seek into a
+large file — the PMTiles directory, then the leaf, then the tile — and that is
+precisely the access pattern a spinning disk is worst at. The two can easily
+differ by an order of magnitude.
+
+What that does to a pool is worse than the average suggests. Balanced evenly,
+half the tiles arrive quickly and half do not, and a map is judged by the slow
+half: panning stutters wherever the slow node answered. The mean looks
+tolerable throughout, which is why this is easy to miss and easy to blame on
+the software.
+
+Three ways out, and they answer different questions.
+
+**`backup`** keeps the slow node out of service entirely until the fast one
+fails:
+
+```
+backend pmtiles-swarm
+    option httpchk GET /health HTTP/1.1
+    http-check expect status 200
+    server node1 172.16.1.49:8090 check inter 5s
+    server node2 172.16.1.41:8090 check inter 5s backup
+```
+
+Every request goes to `node1` while it is healthy, and the pool falls to
+`node2` when it is not. Latency is then the fast node's latency, and the slow
+one is redundancy rather than capacity. This is the right answer when the fast
+node can carry the load alone, which for tile serving it usually can.
+
+**`weight`** keeps both in service and sends proportionally less to the slow
+one:
+
+```
+    server node1 172.16.1.49:8090 check inter 5s weight 200
+    server node2 172.16.1.41:8090 check inter 5s weight 20
+```
+
+Worth it only when you actually need the second node's throughput. It does not
+remove the slow answers, it makes them rarer — a tenth of requests at ten times
+the latency is still a tenth of requests a person notices.
+
+**Least Connections** is the better of the two ways to keep both in service,
+and it needs no numbers chosen by hand. A slow node holds each connection
+longer, so it accumulates open ones and stops being picked until it catches up:
+the pool tunes itself to what the disks are doing today rather than to a weight
+somebody guessed last year. It shares `weight`'s limitation — the slow answers
+become proportionally rarer, not absent — and it pairs with `backup` without
+conflict, since a backup server is held out of the pool whatever the algorithm.
+
+`tools/tile-bench.mjs` shows which situation you are in. A pool of unequal
+nodes answers in two distinct groups rather than one spread, and the tool says
+so explicitly; `--header` tallies a response header naming the backend, if
+HAProxy is set to add one:
+
+```
+http-response set-header X-Served-By %s
+```
+
+Run it before and after the change. What should move is the tail: `total p90`
+and `total p99` collapse towards `p50` once one node is answering everything.
+
 ### HTTP/2
 
 Enable it on the frontend and leave _HTTP/2 without TLS_ unchecked: the client
@@ -219,9 +283,16 @@ says so, which is worse than having no check at all.
 
 **Cache tiles by infohash aggressively.** `/archives/<infohash>/…` is immutable
 by construction — an infohash names those bytes and no others — and is served
-with `max-age=31536000, immutable`. `/latest/<category>/…` is the opposite: it
-moves on every build, and is served with `max-age=60, must-revalidate` and an
-ETag naming the build it resolved to.
+with `max-age=31536000, immutable`.
+
+Everything under `/latest/<category>/` is the opposite, because it moves on
+every build. The documents — `tiles.json`, the feeds — are served with
+`max-age=60, must-revalidate`; the tiles with `max-age=300`. Both carry an ETag
+naming the build they resolved to, so a revalidation is a 304 while the build
+stands and a miss the moment it moves. Let Cloudflare revalidate rather than
+overriding either with a page rule: the whole point of the category tile URL is
+that it is safe to write into an application, and it is only safe while the
+edge notices a rebuild.
 
 **Do not strip or rewrite the ETag on either.** It is the infohash, which is
 how a PMTiles reader notices that the archive moved underneath a read already

package/docs/internals.md

@@ -725,7 +725,7 @@ own and asks for nothing guarded.
 
 Three things joined the public list to make it work, and each is a read of
 something already published: `/api/categories`, which groups the same archives
-and carries the style URL for each; the per-archive `/preview`; and `/vendor/`,
+and carries the source URL for each; the per-archive `/preview`; and `/vendor/`,
 which is the MapLibre bundle the preview renders with. The preview used to be
 excluded on the grounds that it is console furniture and would not render
 without `/vendor` anyway — both true, and both answered by publishing the pair

package/docs/serving-tiles.md

@@ -45,7 +45,7 @@ See [internals.md](internals.md#serving-an-mbtiles-archive).
 
 `GET /latest/` lists every category this node publishes, with the endpoints
 that resolve to each one's newest build — the TileJSON, the `.torrent`, the
-magnet, the per-category feed, and the style URL with the magnet in its
+magnet, the per-category feed, and the source URL with the magnet in its
 fragment.
 
 Public, and deliberately so. Everything else under `/latest/` is — the
@@ -274,7 +274,7 @@ everywhere:
 | torrent-aware                     | joins **before any network call**, and still can if the fetch fails |
 
 The console's **Copy TileJSON URL + swarm** button produces exactly this, and so
-does the `styleUrl` on every row of `/api/categories` and `/latest/`.
+does the `sourceUrl` on every row of `/api/categories` and `/latest/`.
 
 ### Two handles, and why both
 

package/docs/tilejson.md

@@ -110,7 +110,7 @@ minor one. **A browser has no DHT** — WebTorrent's `browser` field maps
 UDP or TCP sockets at all. Given only a public key, a browser would have to fetch
 this TileJSON before it could join anything. Since this magnet is routinely
 carried in the _fragment of the TileJSON URL itself_ — see
-[`styleUrl`](#where-this-magnet-shows-up) — that would make the fragment useless
+[`sourceUrl`](#where-this-magnet-shows-up) — that would make the fragment useless
 to the one client most likely to be reading it.
 
 The infohash does go stale on the next rebuild. That is acceptable and expected:
@@ -122,7 +122,7 @@ going to follow the series anyway. It is a starting point, not a subscription.
 Three places, all built from the same function, so they agree:
 
 - `torrent.mutable.magnet` in this document.
-- The fragment on `styleUrl`, from `GET /api/categories` — a
+- The fragment on `sourceUrl`, from `GET /api/categories` — a
   `…/latest/<category>/tiles.json#magnet:?…`. One string that a plain client
   fetches over HTTP and a swarm client joins directly, with no extra round trip
   for either.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "pmtiles-swarm",
-  "version": "0.59.0",
+  "version": "0.61.0",
   "description": "BitTorrent distribution for PMTiles map archives: create torrents, watch folders, publish and subscribe to RSS feeds, and seed through qBittorrent or an embedded client",
   "type": "module",
   "main": "src/index.js",

package/src/api.js

@@ -32,7 +32,7 @@ import {
   expandTemplate,
 } from './sources.js';
 import { limitFor, remaining } from './seeding.js';
-import { buildTileJson, extensionMatches } from './tilejson.js';
+import { buildTileJson, extensionMatches, tileExtension } from './tilejson.js';
 import { SUMMARY_VERSION } from './pmtiles-probe.js';
 import { TileReadError } from './tiles.js';
 
@@ -158,12 +158,17 @@ function withSwarmHandles(url, { torrent, magnet }) {
 
 /**
  * A TileJSON URL carrying the ways into the swarm in its fragment.
+ *
+ * A *source* URL, not a style one — it goes in a style's `sources` block, and
+ * a style is the document that would contain it. It was called the other thing
+ * for a while, which is a name that tells a reader to put it in the wrong
+ * place.
  * @param {string} category - Which category.
  * @param {object} newest - Its newest entry.
  * @param {string} base - Public base URL.
- * @returns {string} - The URL a style should use.
+ * @returns {string} - The URL a source should point at.
  */
-function styleUrlFor(category, newest, base) {
+function sourceUrlFor(category, newest, base) {
   const url = `${base}/latest/${category}/tiles.json`;
   const magnet = newest?.mutable?.publicKey
     ? mutableMagnet(newest.mutable.publicKey, {
@@ -2158,6 +2163,14 @@ export function createApp({
           servable,
           endpoints: {
             tileJson: servable ? `${base}/latest/${category}/tiles.json` : null,
+            // The XYZ template itself, which is what most things outside this
+            // system actually want: a leaflet layer, an OpenLayers source, a
+            // GIS client, anything that takes a URL with braces in it rather
+            // than a TileJSON document. It follows the category, so it can be
+            // written into an application and survive a rebuild.
+            xyz: servable
+              ? `${base}/latest/${category}/{z}/{x}/{y}.${tileExtension(newest.pmtiles?.format)}`
+              : null,
             // The same URL with the ways into the swarm in the fragment,
             // which is what a style should carry. A fragment is never sent in
             // a request, so an ordinary client fetches the TileJSON and
@@ -2170,7 +2183,13 @@ export function createApp({
             // and resolves the current build over the DHT. Otherwise the
             // newest build's own magnet, which pins that build but still
             // beats a blank map when the fallback is needed at all.
-            styleUrl: servable ? styleUrlFor(category, newest, base) : null,
+            // What goes in a style's `sources` block. It is a TileJSON URL
+            // with the ways into the swarm in its fragment, so it is a source
+            // by every reading: what it addresses, and where it is written.
+            sourceUrl: servable ? sourceUrlFor(category, newest, base) : null,
+            // The name this had until 0.61.0, kept so a consumer that reads it
+            // is not broken by the correction. Deprecated: read `sourceUrl`.
+            styleUrl: servable ? sourceUrlFor(category, newest, base) : null,
             // Points at the category, not at a build. The page reads the
             // TileJSON beside it, so it renders whatever is current — which
             // makes it the same URL a style holds, demonstrating itself rather
@@ -2253,8 +2272,23 @@ export function createApp({
       // makes that re-read cheap — a client already holding the current build
       // gets a 304 and no body, which Express does on its own once an ETag is
       // set before the response goes out.
+      // Both templates are built, because both are wanted and they are wanted
+      // for opposite reasons.
+      //
+      // `tiles` points at the category, so a style written once keeps working
+      // across a rebuild. That is what the endpoint is for, and an infohash
+      // template cannot do it: it changes every build, so anything holding it
+      // is pinned to a build that eventually stops existing.
+      //
+      // The immutable template is still published, under `latest.tiles`, and
+      // is still the better URL for anything that can re-read this document —
+      // it is content-addressed, so it caches for a year and never
+      // revalidates. Offering only one of the two would be choosing for the
+      // consumer, and the right answer differs by consumer.
+      const pinned = buildTileJson(entry, baseUrl(req));
+      const stableRoot = `${baseUrl(req)}/latest/${encodeURIComponent(req.params.category)}`;
       const doc = {
-        ...buildTileJson(entry, baseUrl(req)),
+        ...buildTileJson(entry, baseUrl(req), { tilesRoot: stableRoot }),
         // Names what it resolved to, so a consumer can tell one build from the
         // next without diffing the tile URLs.
         latest: {
@@ -2262,6 +2296,7 @@ export function createApp({
           infohash: entry.infoHash,
           name: entry.name,
           createdAt: entry.createdAt,
+          tiles: pinned.tiles,
         },
       };
       tagAsLatest(res, doc);
@@ -2833,119 +2868,191 @@ export function createApp({
     }),
   );
 
-  app.get(
-    '/archives/:infoHash/:z/:x/:y.:ext',
-    route(async (req, res) => {
-      const { infoHash, ext } = req.params;
-      let entry = catalog.get(infoHash);
-      if (!entry) return res.status(404).json({ error: 'unknown archive' });
-      // MBTiles passes through here and is turned away by the store instead,
-      // which is the only layer that knows whether the download has finished.
-      if (entry.kind && entry.kind !== 'pmtiles' && entry.kind !== 'mbtiles') {
-        return res.status(415).json({
-          error: `this is a ${entry.kind} archive, and only PMTiles and MBTiles can be served as tiles`,
+  /**
+   * Serves one tile, from an archive the caller has already resolved.
+   *
+   * Shared by the two ways of naming an archive, which differ in one respect
+   * and want the same behaviour in every other. `/archives/<infohash>/…` pins
+   * content, so the answer can be cached for a year and never revalidated.
+   * `/latest/<category>/…` resolves to whichever build is current, so it
+   * cannot: the same URL returns different bytes after a rebuild, and a client
+   * holding it for a year would hold a build that no longer exists.
+   *
+   * The difference is expressed entirely in the caching headers rather than in
+   * two implementations, because everything else — the coordinate checks, the
+   * summary backfill, the abort on a cancelled request, the 204-against-404
+   * decision for a sparse archive — has to be identical or one of the two is
+   * quietly a different endpoint.
+   * @param {object} found - The catalog entry to read from.
+   * @param {import('express').Request} req - The request.
+   * @param {import('express').Response} res - The response.
+   * @param {object} [options] - `immutable` false for a resolved category.
+   * @returns {Promise<void>} - Resolves once answered.
+   */
+  const serveTile = async (found, req, res, options = {}) => {
+    const { ext } = req.params;
+    let entry = found;
+    const infoHash = entry.infoHash;
+    // MBTiles passes through here and is turned away by the store instead,
+    // which is the only layer that knows whether the download has finished.
+    if (entry.kind && entry.kind !== 'pmtiles' && entry.kind !== 'mbtiles') {
+      return res.status(415).json({
+        error: `this is a ${entry.kind} archive, and only PMTiles and MBTiles can be served as tiles`,
+      });
+    }
+
+    const z = Number(req.params.z);
+    const x = Number(req.params.x);
+    const y = Number(req.params.y);
+    if (![z, x, y].every(Number.isInteger)) {
+      return res.status(400).json({ error: 'z, x and y must be integers' });
+    }
+    const limit = 2 ** z;
+    if (z < 0 || z > 26 || x < 0 || y < 0 || x >= limit || y >= limit) {
+      return res.status(400).json({ error: 'tile coordinates out of range' });
+    }
+    // An MBTiles archive never went through the PMTiles prober, so it
+    // reaches here with no summary and nothing to check the extension
+    // against. Read one now: it is a handful of rows out of a local SQLite
+    // file, kept in the catalog, so this is paid once per archive rather
+    // than per tile. A failure is left to the read below to report, which
+    // already knows how to say "not complete yet".
+    if (!entry.pmtiles?.format) {
+      const summary = await tiles.summarize(entry.infoHash).catch(() => null);
+      if (summary) {
+        entry = await catalog.put({
+          infoHash: entry.infoHash,
+          pmtiles: summary,
+          summarySource: 'header',
         });
       }
+    }
 
-      const z = Number(req.params.z);
-      const x = Number(req.params.x);
-      const y = Number(req.params.y);
-      if (![z, x, y].every(Number.isInteger)) {
-        return res.status(400).json({ error: 'z, x and y must be integers' });
-      }
-      const limit = 2 ** z;
-      if (z < 0 || z > 26 || x < 0 || y < 0 || x >= limit || y >= limit) {
-        return res.status(400).json({ error: 'tile coordinates out of range' });
-      }
-      // An MBTiles archive never went through the PMTiles prober, so it
-      // reaches here with no summary and nothing to check the extension
-      // against. Read one now: it is a handful of rows out of a local SQLite
-      // file, kept in the catalog, so this is paid once per archive rather
-      // than per tile. A failure is left to the read below to report, which
-      // already knows how to say "not complete yet".
-      if (!entry.pmtiles?.format) {
-        const summary = await tiles.summarize(entry.infoHash).catch(() => null);
-        if (summary) {
-          entry = await catalog.put({
-            infoHash: entry.infoHash,
-            pmtiles: summary,
-            summarySource: 'header',
-          });
-        }
-      }
+    if (!extensionMatches(entry, ext)) {
+      return res.status(400).json({
+        error: `this archive holds ${entry.pmtiles?.format ?? 'unknown'} tiles`,
+      });
+    }
 
-      if (!extensionMatches(entry, ext)) {
-        return res.status(400).json({
-          error: `this archive holds ${entry.pmtiles?.format ?? 'unknown'} tiles`,
+    const controller = new AbortController();
+    // A panning map abandons requests constantly. Without this the swarm
+    // keeps fetching pieces for tiles nobody is waiting for any more.
+    res.on('close', () => {
+      if (!res.writableEnded) controller.abort();
+    });
+
+    // Counted on the way out rather than at each return: this handler ends
+    // in six different places (200, 204, 404, 415, 400, a read error), and
+    // one hook catches all of them without any of them having to remember.
+    // Abandoned requests are not counted -- a panning map cancels constantly
+    // and those were never served.
+    if (stats) {
+      const startedAt = process.hrtime.bigint();
+      res.on('finish', () => {
+        stats.record({
+          infoHash,
+          name: entry.name,
+          z,
+          x,
+          y,
+          status: res.statusCode,
+          bytes: Number(res.getHeader('content-length')) || 0,
+          ms: Number(process.hrtime.bigint() - startedAt) / 1e6,
+          // Whatever this process can see. Behind a proxy that sends no
+          // X-Forwarded-For this is the proxy's address, which is itself
+          // the answer to "did this arrive directly or through HAProxy".
+          ip: req.ip,
         });
-      }
+      });
+    }
 
-      const controller = new AbortController();
-      // A panning map abandons requests constantly. Without this the swarm
-      // keeps fetching pieces for tiles nobody is waiting for any more.
-      res.on('close', () => {
-        if (!res.writableEnded) controller.abort();
+    let tile;
+    try {
+      tile = await tiles.getTile(infoHash, z, x, y, {
+        signal: controller.signal,
       });
+    } catch (error) {
+      if (error.name === 'AbortError') return;
+      if (error instanceof TileReadError) {
+        return res.status(error.status).json({ error: error.message });
+      }
+      throw error;
+    }
 
-      // Counted on the way out rather than at each return: this handler ends
-      // in six different places (200, 204, 404, 415, 400, a read error), and
-      // one hook catches all of them without any of them having to remember.
-      // Abandoned requests are not counted -- a panning map cancels constantly
-      // and those were never served.
-      if (stats) {
-        const startedAt = process.hrtime.bigint();
-        res.on('finish', () => {
-          stats.record({
-            infoHash,
-            name: entry.name,
-            z,
-            x,
-            y,
-            status: res.statusCode,
-            bytes: Number(res.getHeader('content-length')) || 0,
-            ms: Number(process.hrtime.bigint() - startedAt) / 1e6,
-            // Whatever this process can see. Behind a proxy that sends no
-            // X-Forwarded-For this is the proxy's address, which is itself
-            // the answer to "did this arrive directly or through HAProxy".
-            ip: req.ip,
-          });
+    res.setHeader('access-control-allow-origin', '*');
+    // The tag is the resolved infohash either way, which is what makes the
+    // category URL cheap to hold: when the build has not moved a
+    // revalidation is a 304 with no body, and when it has, the tag changes
+    // on its own without anything having to remember to invalidate.
+    res.setHeader('etag', `"${infoHash}-${z}-${x}-${y}"`);
+    res.setHeader(
+      'cache-control',
+      options.immutable === false
+        ? // A category resolves to whatever build is current, so the same
+          // URL returns different bytes after a rebuild. Five minutes bounds
+          // how long a client can be looking at the previous build, and
+          // must-revalidate is what stops a cache serving it beyond that.
+          'public, max-age=300, must-revalidate'
+        : // An infohash pins content, so a tile under one can never change.
+          // When a mutable archive is updated the infohash changes and so
+          // does this URL, which makes cache invalidation automatic.
+          'public, max-age=31536000, immutable',
+    );
+
+    // A missing tile is normal, and which status says so matters.
+    //
+    //   404 tells MapLibre the tile is absent, so it overzooms the parent —
+    //       which is the only way a sparse raster-dem renders terrain at all.
+    //   204 tells it the tile is empty but present, so it draws nothing and
+    //       does not fall back.
+    //
+    // Vector wants 204 (an empty tile means no features here); raster wants
+    // 404. Same rule and same name as tileserver-gl's `sparse`.
+    if (!tile) return res.status(isSparse(entry) ? 404 : 204).end();
+
+    res.type(entry.pmtiles?.contentType ?? 'application/octet-stream');
+    if (tile.encoding) res.setHeader('content-encoding', tile.encoding);
+    res.send(tile.data);
+  };
+
+  app.get(
+    '/archives/:infoHash/:z/:x/:y.:ext',
+    route(async (req, res) => {
+      const entry = catalog.get(req.params.infoHash);
+      if (!entry) return res.status(404).json({ error: 'unknown archive' });
+      if (entry.kind && entry.kind !== 'pmtiles' && entry.kind !== 'mbtiles') {
+        return res.status(415).json({
+          error: `this is a ${entry.kind} archive, and only PMTiles and MBTiles can be served as tiles`,
         });
       }
+      return serveTile(entry, req, res);
+    }),
+  );
 
-      let tile;
-      try {
-        tile = await tiles.getTile(infoHash, z, x, y, {
-          signal: controller.signal,
+  // The one tile URL that survives a rebuild.
+  //
+  // Every archive is addressed by infohash, which is what makes a tile
+  // cacheable for a year and what makes it useless in an application: the URL
+  // changes with every build. A category is the stable handle, so it needs a
+  // tile endpoint of its own — and this is the URL the category TileJSON
+  // advertises, so a style pointed at `/latest/<category>/tiles.json` keeps
+  // working across rebuilds without being re-fetched for the URLs alone.
+  //
+  // Not a redirect to the immutable URL, though every other `/latest/` route
+  // is one. A redirect costs a round trip, and a map asks for hundreds of
+  // tiles: what is a negligible indirection for a `.torrent` is the difference
+  // between a map that feels immediate and one that does not.
+  app.get(
+    '/latest/:category/:z/:x/:y.:ext',
+    route(async (req, res) => {
+      const entry = newestIn(req.params.category, req);
+      if (!entry) return res.status(404).json({ error: 'no such category' });
+      if (entry.kind && entry.kind !== 'pmtiles' && entry.kind !== 'mbtiles') {
+        return res.status(415).json({
+          error: `the newest archive in this category is a ${entry.kind} archive, and only PMTiles and MBTiles can be served as tiles`,
         });
-      } catch (error) {
-        if (error.name === 'AbortError') return;
-        if (error instanceof TileReadError) {
-          return res.status(error.status).json({ error: error.message });
-        }
-        throw error;
       }
-
-      res.setHeader('access-control-allow-origin', '*');
-      // An infohash pins content, so a tile under one can never change. When a
-      // mutable archive is updated the infohash changes and so does this URL,
-      // which makes cache invalidation automatic.
-      res.setHeader('cache-control', 'public, max-age=31536000, immutable');
-      res.setHeader('etag', `"${infoHash}-${z}-${x}-${y}"`);
-
-      // A missing tile is normal, and which status says so matters.
-      //
-      //   404 tells MapLibre the tile is absent, so it overzooms the parent —
-      //       which is the only way a sparse raster-dem renders terrain at all.
-      //   204 tells it the tile is empty but present, so it draws nothing and
-      //       does not fall back.
-      //
-      // Vector wants 204 (an empty tile means no features here); raster wants
-      // 404. Same rule and same name as tileserver-gl's `sparse`.
-      if (!tile) return res.status(isSparse(entry) ? 404 : 204).end();
-
-      res.type(entry.pmtiles?.contentType ?? 'application/octet-stream');
-      if (tile.encoding) res.setHeader('content-encoding', tile.encoding);
-      res.send(tile.data);
+      return serveTile(entry, req, res, { immutable: false });
     }),
   );
 

package/src/incomplete.js

@@ -191,6 +191,15 @@ export class CompletionWatcher {
         await this.#library.captureMetadata?.(entry.infoHash).catch(() => null);
       }
 
+      // Before the `complete` gate below, deliberately. An archive already
+      // recorded complete is the case this covers: it was re-checking when a
+      // tile reader opened it, so the reader went to the swarm — and when the
+      // check finished nothing noticed, because finalize only runs for a
+      // download. See Library.refreshReader.
+      if (entry.status?.progress >= 1) {
+        await this.#library.refreshReader?.(entry.infoHash).catch(() => null);
+      }
+
       if (entry.complete) continue;
 
       // The engine's account wins whenever it has one: a client allocates the

package/src/library.js

@@ -2778,6 +2778,60 @@ export class Library {
     return { cleared: before };
   }
 
+  /**
+   * Drops a reader still going to the swarm for an archive that is now whole.
+   *
+   * Which source an archive is read through is decided once, when it is
+   * opened, and every other thing that can change the answer — a pause, a
+   * resume, a mode change, a move, a finished download — invalidates the
+   * reader itself. A finished *check* is the one that does not, and it is the
+   * easiest of them to hit: during `checking_files` libtorrent reports
+   * `progress` as the fraction hashed so far, which is indistinguishable from
+   * a download at the same figure. So a tile read arriving mid-check opens
+   * against the swarm, correctly, and then keeps that handle after the check
+   * finishes and the whole file is sitting on disk.
+   *
+   * What that looks like is an archive at 100% and seeding whose tiles will
+   * not load, because the swarm it is being read from has one member: this
+   * node. It survives until something evicts the handle, which in practice
+   * means a restart.
+   *
+   * The disk is checked before the handle is dropped, and that is not
+   * belt-and-braces: without it an archive that re-opens against the swarm
+   * anyway would be invalidated again on every sweep, for ever.
+   * @param {string} infoHash - Which archive.
+   * @returns {Promise<boolean>} - Whether a stale reader was dropped.
+   */
+  async refreshReader(infoHash) {
+    // Cheap and synchronous, and false for all but a handful of archives —
+    // this runs for every entry on the completion timer.
+    if (this.#tiles?.status(infoHash)?.mode !== 'swarm') return false;
+
+    const entry = this.#catalog.get(infoHash);
+    if (!entry?.savePath || !entry?.name) return false;
+
+    // Cache mode reads from the swarm because that is what it is for, not
+    // because anything went stale.
+    if (entry.mode === 'cache') return false;
+
+    const whole = await alreadyComplete({
+      savePath: entry.savePath,
+      name: entry.name,
+      size: entry.size,
+    });
+    if (!whole) return false;
+
+    const dropped = await this.#tiles.invalidate(infoHash).catch(() => false);
+    if (dropped) {
+      console.log(
+        `[tiles] ${entry.name}: was being read from the swarm while a whole ` +
+          'copy sat on disk, which is what a finished re-check leaves behind. ' +
+          'Reopening it locally.',
+      );
+    }
+    return dropped;
+  }
+
   /**
    * Rewrites an archive's web seed list, in the .torrent and everywhere else.
    *

package/src/tilejson.js

@@ -23,21 +23,45 @@ const URL_EXTENSION = {
   mlt: 'mlt',
 };
 
+/**
+ * The extension a tile URL uses for one archive format.
+ *
+ * Exported because the console publishes the XYZ template beside the TileJSON,
+ * and a second copy of this map is a second thing to forget when a format is
+ * added — which would show up as a working document advertising a URL that
+ * answers 400.
+ * @param {string} [format] - The archive's tile format.
+ * @returns {string} - The extension, without a dot.
+ */
+export function tileExtension(format) {
+  return URL_EXTENSION[format] ?? 'bin';
+}
+
 /**
  * Builds the TileJSON document for one catalog entry.
+ *
+ * `tilesRoot` is how a category document points its tiles at itself rather
+ * than at the build it happens to have resolved to. The infohash URL is the
+ * right default — it pins content, which is what lets a tile be cached for a
+ * year — but it is the wrong thing to write into an application, because it
+ * changes with every rebuild. A category has to be able to hand out a URL that
+ * does not.
  * @param {object} entry - Catalog entry.
  * @param {string} baseUrl - Public base URL, without a trailing slash.
+ * @param {object} [options] - Overrides.
+ * @param {string} [options.tilesRoot] - Root for the tile template.
  * @returns {object} - A TileJSON 3.0.0 document.
  */
-export function buildTileJson(entry, baseUrl) {
+export function buildTileJson(entry, baseUrl, options = {}) {
   const summary = entry.pmtiles ?? {};
-  const extension = URL_EXTENSION[summary.format] ?? 'bin';
+  const extension = tileExtension(summary.format);
   const root = `${baseUrl}/archives/${entry.infoHash}`;
+  const tilesRoot = options.tilesRoot ?? root;
 
   const doc = {
     tilejson: '3.0.0',
     scheme: 'xyz',
-    tiles: [`${root}/{z}/{x}/{y}.${extension}`],
+    tiles: [`${tilesRoot}/{z}/{x}/{y}.${extension}`],
     name: summary.name ?? entry.name,
     minzoom: summary.minZoom ?? 0,
     maxzoom: summary.maxZoom ?? 14,

package/src/web/index.html

@@ -241,9 +241,11 @@
         padding: 0 0.45rem;
       }
       button.copy:hover { color: var(--fg); border-color: var(--fg); }
+      /* The same green the public catalogue page confirms a copy with. */
+      button.copy.done { border-color: currentColor; color: #3fb950; }
       /* Category endpoints. `th, td` is nowrap for the archive table, where a
          wrapped number is worse than a wide one — but here the cell holds a
-         style URL and a paragraph explaining it, so nowrap pushed the table
+         source URL and a paragraph explaining it, so nowrap pushed the table
          wider than the window and took the Copy and Open buttons off the far
          edge with it. Fixed layout keeps the two end columns where they are
          and gives the middle whatever is left. */
@@ -999,17 +1001,60 @@
         return data;
       }
 
-      const copy = async (text, what) => {
+      /**
+       * Puts text on the clipboard, in a console that is usually not a secure
+       * context.
+       *
+       * `navigator.clipboard` requires one, and a console reached by IP over
+       * plain HTTP on a LAN is not — which is how most of them are reached. So
+       * the interesting path here is the fallback, not the modern call: the
+       * selection API predates the clipboard API and carries no such
+       * requirement, so it works exactly where the other one does not.
+       *
+       * What this replaces was `window.prompt`, which does put the text
+       * somewhere it can be copied from and makes the reader do the copying.
+       * A button that says "copy" should copy.
+       *
+       * Deliberately duplicated in public.html rather than shared. Both pages
+       * are single files with their script inlined, and one browser shim is a
+       * poor reason to give that up.
+       * @param {string} text - What to put on the clipboard.
+       * @returns {Promise<boolean>} - Whether it landed.
+       */
+      const toClipboard = async (text) => {
         try {
           await navigator.clipboard.writeText(text);
-          toast(`${what} copied`);
+          return true;
         } catch {
-          // Clipboard access needs a secure context, which a plain-http LAN
-          // deployment is not. Fall back to showing it to select by hand.
-          window.prompt(`Copy ${what}:`, text);
+          // Off-screen rather than hidden: an element with `display:none` or
+          // `visibility:hidden` cannot hold a selection, so the copy would
+          // silently do nothing.
+          const field = document.createElement('textarea');
+          field.value = text;
+          field.setAttribute('readonly', '');
+          field.style.position = 'fixed';
+          field.style.top = '-1000px';
+          document.body.append(field);
+          field.select();
+          // select() alone is not enough on iOS.
+          field.setSelectionRange(0, field.value.length);
+          try {
+            return document.execCommand('copy');
+          } catch {
+            return false;
+          } finally {
+            field.remove();
+          }
         }
       };
 
+      const copy = async (text, what) => {
+        if (await toClipboard(text)) return toast(`${what} copied`);
+        // Both ways of reaching the clipboard refused. Showing the text is the
+        // only thing left that is better than nothing.
+        window.prompt(`Copy ${what}:`, text);
+      };
+
       // ── Archives ──────────────────────────────────────────────────────────
       let archives = [];
       let selected = null;
@@ -3676,7 +3721,7 @@ Every piece is hashed against the ` +
             // torrent client, a feed reader — and following one from here
             // achieves nothing, so they copy.
             //
-            // The style URL is not printed. It is a TileJSON URL carrying a
+            // The source URL is not printed. It is a TileJSON URL carrying a
             // `.torrent` URL and a percent-encoded magnet in its fragment,
             // several hundred characters of it, and it was truncated to 96
             // here — long enough to fill the row and too short to be the
@@ -3715,19 +3760,22 @@ Every piece is hashed against the ` +
                   ends.tileJson
                     ? `<div class="links">
                          ${copyable(ends.tileJson, 'TileJSON')}
+                         ${copyable(ends.xyz, 'XYZ')}
                          ${link(ends.preview, 'preview')}
                          ${link(ends.torrent, '.torrent', true)}
                          ${copyable(ends.magnet, 'magnet', true)}
                          ${copyable(ends.feed, 'RSS')}
                          ${copyable(ends.latestFeed, 'RSS, newest only')}
-                         ${copyable(ends.styleUrl, 'style URL')}
+                         ${copyable(ends.sourceUrl, 'source URL')}
                        </div>
                        <div class="sub" style="margin-top:0.5rem">
-                         The style URL is the TileJSON with the .torrent URL and
-                         the magnet in its fragment, which is never sent to the
-                         server — an ordinary client fetches the TileJSON and
-                         ignores them, a swarm-aware one reads them first, and
-                         both follow the category rather than this build.
+                         The source URL is the TileJSON with the .torrent URL
+                         and the magnet in its fragment, which is never sent to
+                         the server — an ordinary client fetches the TileJSON
+                         and ignores them, a swarm-aware one reads them first,
+                         and both follow the category rather than this build.
+                         It goes in a style's <code>sources</code> block; it is
+                         not itself a style.
                        </div>`
                     : '<div class="sub">not a PMTiles archive, so there is no tile endpoint</div>'
                 }
@@ -3736,15 +3784,32 @@ Every piece is hashed against the ` +
           .join('');
 
         for (const button of list.querySelectorAll('[data-copy-url]')) {
+          // The URL either way, so there is something to select by hand if
+          // both routes to the clipboard are shut.
+          button.title = button.dataset.copyUrl;
+          const label = button.textContent;
+          // Said on the button rather than in a toast: these sit in a row of
+          // seven, and which one was pressed is the thing worth confirming.
+          // The public catalogue page does the same, and the two are the same
+          // control in two places.
+          const said = (text, ok) => {
+            button.textContent = text;
+            if (ok) button.classList.add('done');
+            setTimeout(() => {
+              button.textContent = label;
+              button.classList.remove('done');
+            }, 1200);
+          };
           button.onclick = async () => {
             const url = button.dataset.copyUrl;
-            if (!button.dataset.copyFetch) return copy(url, 'URL');
             try {
               // A plain fetch, not `api`: that one parses JSON, and this
               // endpoint answers text/plain.
-              const response = await fetch(url);
-              if (!response.ok) throw new Error(response.statusText);
-              copy((await response.text()).trim(), 'magnet');
+              const value = button.dataset.copyFetch
+                ? (await (await fetch(url)).text()).trim()
+                : url;
+              const landed = await toClipboard(value);
+              said(landed ? 'copied' : 'it is in the tooltip', landed);
             } catch (error) {
               toast(error.message);
             }

package/src/web/public.html

@@ -290,6 +290,48 @@
 
       // A row of "label: copyable URL". The URLs are the point of the page, so
       // they are shown in full rather than hidden behind link text.
+      /**
+       * Puts text on the clipboard, including where there is no secure context.
+       *
+       * `navigator.clipboard` requires one. This page is usually reached over
+       * HTTPS and so usually has it — but a node answering by IP on a LAN does
+       * not, and there the button did nothing but say so. The selection API
+       * predates the clipboard API and carries no such requirement, so it
+       * works exactly where the other one fails.
+       *
+       * Deliberately duplicated in index.html rather than shared. Both pages
+       * are single files with their script inlined, and one browser shim is a
+       * poor reason to give that up.
+       * @param {string} text - What to put on the clipboard.
+       * @returns {Promise<boolean>} - Whether it landed.
+       */
+      const toClipboard = async (text) => {
+        try {
+          await navigator.clipboard.writeText(text);
+          return true;
+        } catch {
+          // Off-screen rather than hidden: an element with `display:none` or
+          // `visibility:hidden` cannot hold a selection, so the copy would
+          // silently do nothing.
+          const field = document.createElement('textarea');
+          field.value = text;
+          field.setAttribute('readonly', '');
+          field.style.position = 'fixed';
+          field.style.top = '-1000px';
+          document.body.append(field);
+          field.select();
+          // select() alone is not enough on iOS.
+          field.setSelectionRange(0, field.value.length);
+          try {
+            return document.execCommand('copy');
+          } catch {
+            return false;
+          } finally {
+            field.remove();
+          }
+        }
+      };
+
       const urlRow = (label, url) => {
         const row = el('div');
         row.append(el('span', null, label));
@@ -454,9 +496,11 @@
                 const value = fetched
                   ? (await (await fetch(absolute)).text()).trim()
                   : absolute;
-                await navigator.clipboard.writeText(value);
-                button.classList.add('done');
-                button.textContent = 'copied';
+                const landed = await toClipboard(value);
+                button.classList.toggle('done', landed);
+                button.textContent = landed
+                  ? 'copied'
+                  : 'copy failed — it is in the tooltip';
                 setTimeout(() => {
                   button.classList.remove('done');
                   button.textContent = text;
@@ -470,6 +514,11 @@
 
           const ends = entry.endpoints;
           if (ends.tileJson) copy(ends.tileJson, 'TileJSON');
+          // The XYZ template beside the document that carries it. Anything
+          // that takes a URL with braces in it rather than a TileJSON — a
+          // Leaflet layer, an OpenLayers source, a GIS client — wants this
+          // one, and it follows the category, so it survives a rebuild.
+          if (ends.xyz) copy(ends.xyz, 'XYZ');
           // The newest build's preview, which is what "show me this
           // category" means.
           if (ends.preview) add(ends.preview, 'preview');
@@ -486,7 +535,7 @@
           // URL carrying the .torrent URL and a percent-encoded magnet in its
           // fragment. Printed in full it was most of the card — several
           // hundred characters of it — and it is never read, only pasted.
-          if (ends.styleUrl) copy(ends.styleUrl, 'style URL');
+          if (ends.sourceUrl) copy(ends.sourceUrl, 'source URL');
           card.append(links);
 
           host.append(card);