pmtiles-swarm 0.55.2 → 0.56.1

package/CHANGELOG.md

@@ -7,6 +7,65 @@
 ### 🐞 Bug fixes
 - _...Add new stuff here..._
 
+## 0.56.1
+### ✨ Features and improvements
+- **Comments trimmed back to house style, and the reasoning moved into the docs where it belongs.**
+  Recent work had been leaving 15–25 line explanatory blocks in the source; measured across
+  `catalog.js` that was 85 new comment lines against 76 new code lines. The argument for a decision
+  drifts out of date faster in a comment than in prose, and buries the code while it does it.
+
+  Two new sections in [docs/internals.md](docs/internals.md) give the displaced reasoning a home:
+  **Re-reading a summary an older prober wrote**, and **A validator for a URL that stays put** —
+  which is where the ETag story now lives, including why the infohash was the wrong choice and what
+  a PMTiles reader needs to see for `If-Range` and cross-origin `ETag` to work.
+
+  No behaviour change.
+
+## 0.56.0
+### ✨ Features and improvements
+- **MapLibre Tiles are recognised.** PMTiles tile type `6` is MLT, and the tile-type table stopped at
+  `5` — so an MLT archive probed as `unknown` and was refused a tile endpoint it could have served,
+  even though the extension map had known about `.mlt` for a while.
+
+  No new TileJSON key was needed for it. MapLibre spells the tile encoding of a vector source
+  `encoding`, exactly as it does the elevation packing of a raster-dem one, and applies TileJSON
+  members to both after the source is constructed — so `encoding: "mlt"` reaches a vector source the
+  same way `encoding: "terrarium"` reaches a raster one. One key, two meanings, told apart by the
+  source type. The MLT value is read from the header rather than the metadata, because an archive
+  whose tile type says MapLibre Tiles is MLT-encoded and nothing needs to say so twice.
+
+- **A resume-data shortfall is now reported.** The sidecar has been answering with how many torrents
+  were asked to write and how many managed it before the deadline, and both numbers were discarded —
+  by the engine wrapper, and again by the timer that called it. A torrent that does not write is one
+  that gets re-hashed on the next start, which for a 700 GiB archive is the difference between
+  seeding in seconds and seeding in half an hour. That is what "why is everything at 0%" looks like
+  from outside, and the silence here is part of why it was hard to see.
+
+### 🐞 Bug fixes
+- **The two pages disagreed about how large an archive was.** The console rounded to whole units
+  above ten and the public page always kept a decimal, so the same archive read as `81 GiB` on one
+  and `80.6 GiB` on the other. Both now keep a decimal from KiB up — these are mostly archive sizes,
+  and half a gigabyte is worth seeing — and a test holds the two helpers character-for-character
+  identical, since nothing about a duplicated function announces when it stops being a copy.
+
+## 0.55.3
+### 🐞 Bug fixes
+- **A `/latest/` document could never be updated once a client had cached it.** The `ETag` was the
+  infohash, which says which _build_ a category resolved to — and these documents carry more than
+  that. A TileJSON also carries the archive's summary; a magnet also carries its web seeds and
+  trackers; the feed carries both. So enriching a summary (0.55.1 reading an `encoding` an older
+  prober had missed) or adding a web seed changed the body while the infohash stayed put: every cache
+  in the path revalidated, was told `304`, and went on serving the old document indefinitely. That is
+  not a cache being stale for a minute; it is a document that can never change again.
+
+  These are now tagged over what is actually sent. `/latest/<category>/archive.pmtiles` and the
+  `.torrent` redirect keep the infohash, because for those the infohash really is the whole content.
+
+- **A successful metadata re-read was logged as a failure.** The line counting vector layers ran for
+  raster archives too, where there are none, and the `TypeError` went to the catch beside it — which
+  reported a backfill that had already written its result as "no vector layers yet". A misleading log
+  is worse than none when it is what somebody is reading to find out whether the thing works.
+
 ## 0.55.2
 ### 🐞 Bug fixes
 - **The stale-summary re-read never ran for a category URL.** 0.55.1 added it to

package/docs/internals.md

@@ -18,6 +18,8 @@ Operator-facing documentation is elsewhere — see [publishing](publishing.md),
 - [Serving an MBTiles archive](#serving-an-mbtiles-archive)
 - [Answering for a tile that is not there](#answering-for-a-tile-that-is-not-there)
 - [Reading an archive that is still arriving](#reading-an-archive-that-is-still-arriving)
+- [Re-reading a summary an older prober wrote](#re-reading-a-summary-an-older-prober-wrote)
+- [A validator for a URL that stays put](#a-validator-for-a-url-that-stays-put)
 - [The health endpoint](#the-health-endpoint)
 - [The externally visible base URL](#the-externally-visible-base-url)
 - [Scheduled sources](#scheduled-sources)
@@ -310,6 +312,86 @@ nothing to read yet and the same archive at 100% will, so a permanent
 would put a swarm read behind each one. The limiter is in memory on purpose: a
 restart is a reasonable moment to try again.
 
+## Re-reading a summary an older prober wrote
+
+An archive's summary — format, zoom range, bounds, vector layers, encoding — is
+read once, written into the catalog, and never questioned again. That is right
+as far as it goes: re-reading a header out of the swarm is not free, and for a
+given infohash the answer cannot change.
+
+It goes wrong the moment the prober learns to read something new. Every archive
+probed before that keeps a summary with a hole in it, permanently, and the only
+way out is to remove and re-add the archive by hand.
+
+`encoding` was the case that made this plain. The key had been sitting in the
+metadata of archives a node had been serving for months; teaching the prober to
+read it changed nothing at all, because nothing ever asked again.
+
+So `summarize()` stamps a `summaryVersion`, and a summary older than the current
+prober is re-read once in the background — on a request for the archive's
+TileJSON, rate-limited to once a minute per archive. Raise `SUMMARY_VERSION`
+whenever a field is added and every archive in the catalog picks it up on its
+own, with nothing to do on upgrade.
+
+Two things this needs to get right, both learned the hard way:
+
+- **Every route that serves a TileJSON has to trigger it**, not only
+  `/archives/<infohash>/tiles.json`. `/latest/<category>/tiles.json` is the URL
+  a style points at, so leaving it out missed exactly the archives that were
+  being consumed the documented way.
+- **The write-back must not be conditioned on one field.** It began as a
+  vector-layer backfill and returned early unless layers turned up, which would
+  have discarded the encoding it had just gone to fetch.
+
+## A validator for a URL that stays put
+
+Everything under `/archives/` is addressed by infohash: the URL changes when the
+content does, so it can be cached for a year and marked `immutable`. A `/latest/`
+URL is the opposite — stable on purpose, with the content moving underneath it —
+so it needs a validator, and a short TTL alone is a guess.
+
+The obvious validator is the infohash of the archive the category resolved to.
+It is wrong, and it fails in a way that has no bottom: these documents carry more
+than which build they name. A TileJSON also carries the archive's summary; a
+magnet also carries its web seeds and trackers; the feed carries both. Enrich a
+summary or add a web seed and the body changes while the infohash does not — so
+every cache in the path revalidates, is told `304`, and goes on serving the old
+document. Not stale for a minute: unable to be updated again for as long as the
+archive exists.
+
+So these are tagged over the body they are sending. Two nodes answering
+identically still produce identical tags, which is the property the infohash was
+chosen for; two nodes answering differently no longer claim otherwise.
+
+`/latest/<category>/archive.pmtiles` and the `.torrent` redirect keep the
+infohash, because for those the infohash really is the whole content.
+
+### Where the ranges have to be careful
+
+A PMTiles reader does not fetch a file. It fetches a header, then a root
+directory, then leaf directories, then tiles, over minutes or hours. If a
+rebuild lands partway through, the offsets it read from the old build address
+bytes in the new one — which does not fail loudly. It decodes as the wrong tile,
+or as nothing, with no error anywhere naming the cause.
+
+`If-Range` is therefore honoured on the category range endpoint: a range
+conditioned on a build that is no longer current is refused _as a range_ and
+answered in full, which a reader survives. `Last-Modified` is suppressed there so
+a client cannot condition on a date instead — a build restored from a backup can
+be newer while looking older.
+
+The official reader closes the loop from its side, comparing the `ETag` of every
+response against the one it saw first and re-reading the header when they differ.
+That only works if it can see the tag: `ETag` is not exposed to cross-origin
+JavaScript by default, so these routes send `Access-Control-Expose-Headers`.
+Unexposed, the reader compares against `null`, the comparison never fires, and it
+splices two builds in silence.
+
+A weak validator is no better. The reader discards any tag beginning with `W/`,
+which is exactly what Express derives from a file's size and mtime — and that
+tag also differs per node, so two nodes behind a load balancer would hand a
+reader two tags for byte-identical archives.
+
 ## The health endpoint
 
 For a load balancer, which needs three things: no credential, a cheap answer, and

package/docs/serving-tiles.md

@@ -157,14 +157,23 @@ answers 404 for them exactly as the feeds do.
 
 ### How a client knows the build moved
 
-Every one of these carries an `ETag`, and the tag is the infohash of the archive
-it resolved to:
+Every one of these carries an `ETag` over the document it is sending:
 
 ```
-ETag: "913d671f3a28c5b8d605e28cf6bf01e293d36e86"
+ETag: "6b8f1c2d…"
 Cache-Control: public, max-age=60, must-revalidate
 ```
 
+**Over the document, not over the infohash.** The infohash is the obvious
+choice and it is wrong here: it says which _build_ a category resolved to, and
+these documents carry more than that. A TileJSON also carries the archive's
+summary; a magnet also carries its web seeds and trackers. Enrich a summary or
+add a web seed and the body changes while the infohash does not — so every cache
+in the path revalidates, is told `304`, and goes on serving the old document.
+Not stale for a minute: unable to be updated at all. Only
+`/latest/{category}/archive.pmtiles` and the `.torrent` redirect are tagged by
+infohash, because for those the infohash really is the whole content.
+
 A short TTL on its own is a guess. At five minutes, every client and every proxy
 in front of one serves the previous build for up to five minutes after a rollover
 and not one of them can tell it is doing so. The infohash is the honest answer:

package/docs/tilejson.md

@@ -172,7 +172,22 @@ MapLibre applies every TileJSON member to the source after the source is
 constructed, so an `encoding` here **overrides** one written in the style. That
 is the intended direction: the archive is the thing that knows.
 
-Anything other than the three values the style specification defines is dropped
+### `mlt`
+
+The same key carries one more value, for a different kind of source. MapLibre
+spells the tile encoding of a vector source `encoding` too, and `mlt` there means
+the tiles are [MapLibre Tiles](https://maplibre.org/maplibre-tile-spec/) rather
+than MVT — so there was no new key to invent for it, only a value to allow
+through.
+
+This one comes from the header rather than the metadata. PMTiles has a tile type
+for MLT (`6`), and an archive whose tile type says MapLibre Tiles _is_
+MLT-encoded; nothing needs to be written in the metadata to make it so.
+Elevation packing is the opposite case — the header knows the tile is WebP and
+nothing about what its three channels mean — which is why the two are read from
+different places despite sharing a key.
+
+Anything other than the values the style specification defines is dropped
 rather than passed on. A client handed an encoding it does not recognise is
 worse off than one handed nothing, because nothing at least leaves it free to
 use its own default.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "pmtiles-swarm",
-  "version": "0.55.2",
+  "version": "0.56.1",
   "description": "BitTorrent distribution for PMTiles map archives: create torrents, watch folders, publish and subscribe to RSS feeds, and seed through qBittorrent or an embedded client",
   "type": "module",
   "main": "src/index.js",

package/src/api.js

@@ -366,13 +366,8 @@ export function createApp({
   const metadataRetries = new Map();
   const needsReread = (summary, infoHash) => {
     if (!summary) return false;
-    // Two reasons to go back to the archive. The first is the one this began
-    // as: a vector archive whose layers have not been read yet. The second is
-    // the general case it turned out to be — a summary written by an older
-    // prober, missing whatever that prober did not know to look for. Without
-    // it, adding a field to the summary reaches new archives only, and every
-    // archive already in the catalog keeps its hole until somebody removes and
-    // re-adds it by hand.
+    // Either the layers were never read, or an older prober wrote this and did
+    // not know to look for what the summary now carries.
     const stale = summary.summaryVersion !== SUMMARY_VERSION;
     const missingLayers = summary.format === 'pbf' && !summary.vectorLayers;
     if (!stale && !missingLayers) return false;
@@ -404,19 +399,18 @@ export function createApp({
     tiles
       .summarize(entry.infoHash, { timeoutMs })
       .then(async (summary) => {
-        // Written back whenever the read produced something newer than what
-        // was stored. Returning early unless vector layers turned up was right
-        // when layers were the only thing this looked for; it would now throw
-        // away the encoding it went to fetch.
         const fresh = summary.summaryVersion !== entry.pmtiles?.summaryVersion;
         if (!summary.vectorLayers && !fresh) return;
         await catalog.put({
           infoHash: entry.infoHash,
           pmtiles: { ...entry.pmtiles, ...summary },
         });
+        // Raster archives have no layers to count.
         console.log(
-          `[tiles] read ${summary.vectorLayers.length} vector layers for ` +
-            `${entry.name} out of the swarm`,
+          summary.vectorLayers
+            ? `[tiles] read ${summary.vectorLayers.length} vector layers for ` +
+                `${entry.name} out of the swarm`
+            : `[tiles] re-read the metadata for ${entry.name}`,
         );
       })
       .catch((error) => {
@@ -1041,17 +1035,16 @@ export function createApp({
           // `location.origin` there names the one port that is not for the
           // public — and a web seed built from it is handed to every peer in
           // the swarm.
-          // Whatever was actually published for this archive, where one was:
-          // the record is the truth about what peers hold, and it need not be
-          // what this node would build today.
+          // What was actually published, which need not be what this node
+          // would build today.
           url:
             entry.selfWebSeedUrl ??
             `${publishingBase({
               config,
               requestBase: baseUrl(req),
             })}/archives/${entry.infoHash}/archive.pmtiles`,
-          // The base the switch would use, so the console can offer it for
-          // editing before anything permanent is written.
+          // Worked out here: the console runs on the admin listener, whose port
+          // no peer can reach.
           base: publishingBase({ config, requestBase: baseUrl(req) }),
         },
       });
@@ -1177,14 +1170,8 @@ export function createApp({
       const body = req.body ?? {};
       try {
         const result = await library.setPublishing(req.params.infoHash, body, {
-          // Typed into the field beside the switch, where it beats
-          // everything else: this is the one URL a person gets to decide
-          // deliberately, because it is the one that cannot be taken back.
+          // The typed field wins; otherwise the request, on the public port.
           publishingUrl: body.publishingUrl,
-          // Otherwise the request it is being asked on is the best evidence
-          // available, the same reasoning the TileJSON already uses — and
-          // with the public port rather than the admin one, since a seed URL
-          // on a listener bound to localhost reaches no peer at all.
           baseUrl: baseUrl(req),
         });
         res.json(result);
@@ -2052,9 +2039,7 @@ export function createApp({
           magnet: entry.magnet,
           torrent: `${baseUrl(req)}/archives/${entry.infoHash}/archive.torrent`,
           webSeeds: entry.webSeeds ?? [],
-          // Present only when this archive is offered as a download. Absent
-          // otherwise rather than null: a public document should say what is
-          // on offer, not enumerate what is being withheld.
+          // Absent rather than null where it is not on offer.
           ...(publishingFor(entry, config).publicDownload
             ? {
                 archive: `${baseUrl(req)}/archives/${entry.infoHash}/archive.pmtiles`,
@@ -2103,34 +2088,22 @@ export function createApp({
   };
 
   /**
-   * Tags a response as "whichever build of this category is current".
+   * Tags a `/latest/` response so a cache can tell when the build moves.
    *
-   * Everything under `/archives/` is addressed by infohash and may be cached
-   * for a year, because the URL changes when the content does. A `/latest/`
-   * URL is the exact opposite: it is stable on purpose, so the content
-   * underneath it moves and the URL alone gives a cache no way to notice. A
-   * short TTL was the only thing these endpoints had, and a TTL is a guess —
-   * at five minutes, every client and every proxy in front of one serves the
-   * previous build for up to five minutes after a rollover, and not one of
-   * them can tell whether it is doing so.
-   *
-   * The infohash is the honest validator. It changes exactly when the archive
-   * a category resolves to changes, never otherwise, and it is the same value
-   * on every node in the swarm — so two nodes behind a load balancer agree
-   * about what is current, rather than each inventing a tag of its own from a
-   * body hash or an mtime and defeating the cache whenever a client is sent
-   * to the other one.
+   * See docs/serving-tiles.md — "How a client knows the build moved".
    * @param {import('express').Response} res - The response to tag.
-   * @param {object} entry - The archive this URL resolved to.
+   * @param {object|string} content - What is being sent.
    */
-  const tagAsLatest = (res, entry) => {
-    // Quoted, because that is what an entity-tag is. Unquoted, a validator
-    // never matches an If-None-Match and every revalidation misses silently.
-    res.setHeader('etag', `"${entry.infoHash}"`);
-    // must-revalidate because serving this one stale is not merely "slightly
-    // old": a reader holding part of one build and asking for the rest after
-    // a rollover assembles a file that never existed. A minute of cache with
-    // a mandatory check at the end of it is the trade.
+  const tagAsLatest = (res, content) => {
+    // Over the body, not the infohash: these documents carry more than which
+    // build they resolved to, so the infohash can stay put while they change.
+    const text =
+      typeof content === 'string' ? content : JSON.stringify(content);
+    // Quoted, or `fresh` never matches it and every revalidation misses.
+    res.setHeader(
+      'etag',
+      `"${crypto.createHash('sha1').update(text).digest('hex')}"`,
+    );
     res.setHeader('cache-control', 'public, max-age=60, must-revalidate');
   };
 
@@ -2238,12 +2211,8 @@ export function createApp({
     route(async (req, res) => {
       res.setHeader('access-control-allow-origin', '*');
       const categories = describeCategories(req);
-      // Tagged over the categories alone, deliberately. Express would happily
-      // hash the whole body for us, but `generatedAt` is a fresh timestamp on
-      // every request — so that tag would differ every time and no client
-      // could ever revalidate. What a consumer polls this for is whether the
-      // set of categories or the build each resolves to has moved, and that
-      // is exactly what this covers.
+      // Over the categories alone: `generatedAt` changes every request, so a
+      // tag covering it could never match.
       res.setHeader(
         'etag',
         `"${crypto.createHash('sha1').update(JSON.stringify(categories)).digest('hex')}"`,
@@ -2274,10 +2243,7 @@ export function createApp({
         });
       }
 
-      // The same background re-read the per-archive route does. Left out
-      // here, it missed exactly the archives that matter most: /latest/ is the
-      // URL a style points at, so a category anyone actually consumes through
-      // the documented path was the one place a stale summary never healed.
+      // /latest/ is the URL a style points at, so it has to heal too.
       startMetadataBackfill(entry);
 
       res.setHeader('access-control-allow-origin', '*');
@@ -2287,8 +2253,7 @@ export function createApp({
       // makes that re-read cheap — a client already holding the current build
       // gets a 304 and no body, which Express does on its own once an ETag is
       // set before the response goes out.
-      tagAsLatest(res, entry);
-      res.json({
+      const doc = {
         ...buildTileJson(entry, baseUrl(req)),
         // Names what it resolved to, so a consumer can tell one build from the
         // next without diffing the tile URLs.
@@ -2298,7 +2263,9 @@ export function createApp({
           name: entry.name,
           createdAt: entry.createdAt,
         },
-      });
+      };
+      tagAsLatest(res, doc);
+      res.json(doc);
     }),
   );
 
@@ -2319,34 +2286,23 @@ export function createApp({
     // this one says so, and names what it resolved to while it is at it. The
     // tag is the infohash being redirected to, which is precisely the thing
     // that changes when this redirect starts pointing somewhere else.
-    tagAsLatest(res, entry);
-    res.redirect(
-      302,
-      `${baseUrl(req)}/archives/${entry.infoHash}/archive.torrent`,
-    );
+    const target = `${baseUrl(req)}/archives/${entry.infoHash}/archive.torrent`;
+    tagAsLatest(res, target);
+    res.redirect(302, target);
   });
 
   app.get('/latest/:category/magnet', (req, res) => {
     const entry = newestIn(req.params.category, req);
     if (!entry) return res.status(404).json({ error: 'no such category' });
-    tagAsLatest(res, entry);
-    res.type('text/plain').send(entry.magnet ?? '');
+    const magnet = entry.magnet ?? '';
+    tagAsLatest(res, magnet);
+    res.type('text/plain').send(magnet);
   });
 
   /**
-   * The headers a browser must be told it may read from a range response.
+   * Headers a cross-origin PMTiles reader needs and would not otherwise see.
    *
-   * Only a handful of response headers reach JavaScript on a cross-origin
-   * fetch, and not one of the headers a PMTiles reader depends on is among
-   * them. The official reader records the ETag of the first response and
-   * compares every response after it against that value — which is how it
-   * notices that the archive moved underneath a read in progress. Cross-origin
-   * and unexposed, it reads `null` instead, a comparison against null never
-   * fires, and it would splice two builds together in silence: the exact
-   * failure the tag exists to prevent. Content-Range earns its place through a
-   * narrower case — an archive smaller than the reader's first 16 KiB request
-   * is answered 416, and it parses the real length out of that header before
-   * retrying.
+   * See docs/serving-tiles.md — "How a client knows the build moved".
    */
   const RANGE_CORS_HEADERS = {
     'access-control-allow-origin': '*',
@@ -2357,24 +2313,8 @@ export function createApp({
   /**
    * Serves a byte range for an archive this node does not hold whole.
    *
-   * **Experimental.** It closes the loop the cache mode was built for: the
-   * node holds no bytes, a reader asks for some, and the pieces covering them
-   * are pulled out of the swarm on demand — the same path the tile endpoint
-   * has always taken internally, one HTTP layer further out and sharing the
-   * same piece cache and open handle.
-   *
-   * It is off by default and should stay off for anything public, for reasons
-   * that are properties of the arrangement rather than of this code:
-   *
-   * - Every byte is somebody else's upload. A node in cache mode is not an
-   *   origin; putting one behind a URL that looks like an origin turns each
-   *   request into swarm traffic that this node neither paid for nor holds.
-   * - A piece read takes as long as the swarm takes. That is fine for a tile,
-   *   which the reader asked for and will wait on, and poor for an HTTP client
-   *   with its own patience.
-   * - There is no way to answer a request for the whole file that is not
-   *   "download 700 GiB through BitTorrent and stream it out". So a range is
-   *   required, and a large one is refused.
+   * Experimental, off by default, and bounded: a Range is required and a large
+   * one refused. See docs/configuration.md — `serveArchiveFromSwarm`.
    * @param {import('express').Request} req - The request, for its Range.
    * @param {import('express').Response} res - The response.
    * @param {object} entry - The archive.
@@ -2396,17 +2336,12 @@ export function createApp({
       });
     }
 
-    // A range, and only a range. Without one the answer is the whole archive,
-    // and the whole archive is not on this disk — serving it would mean
-    // pulling every piece through the swarm to stream it back out, which is
-    // both enormous and somebody else's bandwidth.
+    // Without a Range the answer is the whole archive, pulled piece by piece
+    // through the swarm to stream back out.
     const asked = req.headers.range;
     if (!asked) {
       res.setHeader('accept-ranges', 'bytes');
-      // 409 rather than 411. The request is well formed -- 411 is about a
-      // missing Content-Length on the way in -- and what is wrong is the state
-      // of the thing being asked for: the whole file is not here, and fetching
-      // it through the swarm to stream back out is not an answer.
+      // 409, not 411: the request is well formed, the resource is not here.
       return res.status(409).json({
         error:
           'this node does not hold this archive, so it can only answer a ' +
@@ -2415,8 +2350,7 @@ export function createApp({
     }
 
     const ranges = parseRange(size, asked, { combine: true });
-    // -1 is unsatisfiable, -2 is malformed, and more than one range would mean
-    // a multipart response that nothing reading PMTiles has ever asked for.
+    // -1 unsatisfiable, -2 malformed; multipart is not worth supporting here.
     if (ranges === -1) {
       res.setHeader('content-range', `bytes */${size}`);
       return res.status(416).json({ error: 'range not satisfiable' });
@@ -2437,9 +2371,8 @@ export function createApp({
       });
     }
 
-    // A deadline of its own, shorter than the piece timeout underneath. An
-    // HTTP client gives up long before libtorrent does, and a request left
-    // hanging on a piece nobody is seeding holds a connection open for nothing.
+    // Shorter than the piece timeout underneath: an HTTP client gives up long
+    // before libtorrent does.
     const controller = new AbortController();
     const deadline = setTimeout(
       () => controller.abort(),
@@ -2456,8 +2389,7 @@ export function createApp({
       for (const [name, value] of Object.entries(RANGE_CORS_HEADERS)) {
         res.setHeader(name, value);
       }
-      // The same tag a complete copy would answer with, because it is the same
-      // content: the infohash names these bytes wherever they were read from.
+      // The same tag a complete copy answers with: same bytes, same name.
       res.setHeader('etag', `"${entry.infoHash}"`);
       res.setHeader('cache-control', 'public, max-age=31536000, immutable');
       res.send(body);
@@ -2476,30 +2408,13 @@ export function createApp({
   /**
    * The current build of a category, as a file, by byte range.
    *
-   * The same thing `/archives/<infohash>/archive.pmtiles` is, addressed the
-   * way a style or a long-lived configuration wants to address it: by what it
-   * is rather than by which build it happens to be. Point any PMTiles reader
-   * at this and it keeps working across rebuilds, with no infohash to chase.
-   *
-   * The hazard is the one the immutable route never has to think about. A
-   * PMTiles reader does not fetch a file; it fetches a header, then a root
-   * directory, then leaf directories, then tiles, over minutes or hours. If a
-   * rebuild lands partway through, the offsets it read from the old build
-   * address bytes in the new one. That does not fail loudly — it decodes as
-   * the wrong tile, or as nothing, with no error anywhere that names the
-   * cause. So the validator is the infohash and `If-Range` is honoured: a
-   * range conditioned on a build that is no longer current is refused as a
-   * range and answered in full, which a reader survives, instead of being
-   * spliced, which it does not.
+   * `If-Range` is honoured so a rebuild landing mid-read cannot splice two
+   * builds. See docs/serving-tiles.md — "How a client knows the build moved".
    */
   app.get('/latest/:category/archive.pmtiles', (req, res) => {
     const entry = newestIn(req.params.category, req);
     if (!entry) return res.status(404).json({ error: 'no such category' });
 
-    // Whether this node hands out whole archives at all is the operator's
-    // call, not a consequence of having the file. Everything else published
-    // here is either small or metered by the request; this is up to 700 GiB to
-    // anyone who knows an infohash, so it is asked for rather than assumed.
     if (!publishingFor(entry, config).serveArchive) {
       return res.status(403).json({
         error:
@@ -2508,10 +2423,8 @@ export function createApp({
       });
     }
 
-    // Not on this disk. Either the node can read it out of the swarm, which is
-    // what cache mode is for, or it says so — but it never answers a range
-    // from a sparse file, where the unwritten space reads as zeroes and looks
-    // exactly like data.
+    // Never answered from the sparse file, where unwritten space reads as
+    // zeroes and looks exactly like data.
     if (entry.complete === false) return serveFromSwarm(req, res, entry);
     const file = entry.savePath ? path.join(entry.savePath, entry.name) : null;
     if (!file) return res.status(404).json({ error: 'no file for it here' });
@@ -2566,16 +2479,15 @@ export function createApp({
     // validator: a tag meaning "still empty" is indistinguishable from one
     // meaning "still this build", and a reader would cache the emptiness past
     // the arrival of the thing it was waiting for.
-    if (entry) tagAsLatest(res, entry);
-    res.type('application/rss+xml').send(
-      renderFeed(entry ? [entry] : [], {
-        title: `${config.feedTitle ?? 'PMTiles archives'} — ${category}, latest`,
-        baseUrl: baseUrl(req),
-        copyright: config.feedCopyright,
-        category,
-        maxItems: 1,
-      }),
-    );
+    const body = renderFeed(entry ? [entry] : [], {
+      title: `${config.feedTitle ?? 'PMTiles archives'} — ${category}, latest`,
+      baseUrl: baseUrl(req),
+      copyright: config.feedCopyright,
+      category,
+      maxItems: 1,
+    });
+    if (entry) tagAsLatest(res, body);
+    res.type('application/rss+xml').send(body);
   });
 
   app.get('/feed.xml', (req, res) => {
@@ -2839,10 +2751,6 @@ export function createApp({
     const entry = catalog.get(req.params.infoHash);
     if (!entry) return res.status(404).json({ error: 'not found' });
 
-    // Whether this node hands out whole archives at all is the operator's
-    // call, not a consequence of having the file. Everything else published
-    // here is either small or metered by the request; this is up to 700 GiB to
-    // anyone who knows an infohash, so it is asked for rather than assumed.
     if (!publishingFor(entry, config).serveArchive) {
       return res.status(403).json({
         error:

package/src/catalog.js

@@ -58,42 +58,18 @@ export function normalizeCategories(source) {
 }
 
 /**
- * What this node offers of an archive's own bytes, over plain HTTP.
+ * What this node offers of an archive's own bytes over HTTP.
  *
- * Three separate decisions, because they are three separate exposures and
- * conflating them takes the choice away from whoever runs the node:
- *
- * - `serveArchive` — whether `/archives/<hash>/archive.pmtiles` answers at
- *   all. This is the one that decides whether a stranger who knows an infohash
- *   can pull 700 GiB off the box.
- * - `selfWebSeed` — whether this node's own URL is written into the torrent's
- *   `url-list`, so every peer in the swarm fetches from it.
- * - `publicDownload` — whether the public catalogue page offers it as a
- *   download. Serving a file to a reader that already knows the URL and
- *   advertising it on a page are not the same act.
- *
- * The last two are ANDed with the first rather than merely defaulting from it.
- * A node cannot be a web seed for a file it will not serve, and a download link
- * that 403s is worse than no link — so a catalog edited by hand into that state
- * is read as the safe thing rather than obeyed into an incoherent one.
+ * Three separate exposures, resolved per archive then per node. See
+ * docs/configuration.md — "Offering the archive file itself".
  * @param {object} [entry] - A catalog entry, whose fields win where set.
  * @param {object} [config] - The node's defaults.
  * @returns {{serveArchive: boolean, selfWebSeed: boolean, publicDownload: boolean}} - Resolved.
  */
 export function publishingFor(entry, config) {
   const serveArchive = entry?.serveArchive ?? config?.serveArchive ?? false;
-  // Whether the whole file is here. `serveArchive` does not depend on it —
-  // `serveArchiveFromSwarm` exists precisely so a node holding nothing can
-  // still answer a bounded range — but a download link does. A link labelled
-  // "download" on a public page has to hand over the file, and a cache-mode
-  // node cannot: it answers 409, or refuses a request with no Range, which
-  // reads as a broken node rather than as a deliberate limit.
-  //
-  // `selfWebSeed` is guarded separately, in setPublishing, because it has a
-  // lifecycle rather than a state: the URL is written into a .torrent and has
-  // to be added and withdrawn at the right moments, not merely reported as
-  // off. Resolving it to false here would withdraw a published seed the first
-  // time an archive was rechecked.
+  // A download link has to hand over the file; a bounded range need not.
+  // selfWebSeed is guarded in setPublishing instead — it has a lifecycle.
   const held = entry?.complete !== false;
   return {
     serveArchive,
@@ -275,20 +251,10 @@ export class Catalog {
   }
 }
 /**
- * Whether a URL is one other people could plausibly fetch.
- *
- * A web seed is not a setting; it is written into the `.torrent` and the
- * magnet, and served to everyone who asks for either. Nothing rewrites it
- * afterwards — the file is handed out byte for byte — so a URL that names
- * this machine's own loopback interface is not a mistake that gets corrected
- * later. It is distributed, followed, and retried by every peer in the swarm
- * for as long as the torrent exists.
+ * Whether a URL is one other peers could plausibly fetch.
  *
- * Loopback is the only case refused outright, because it cannot be right for
- * anybody: `127.0.0.1` means the peer's own machine, not this one. A private
- * address is a different matter — a node syncing to its own peers across a LAN
- * is a real arrangement, and this is not the place to overrule it — so that is
- * reported rather than blocked.
+ * Loopback is refused; a private address is reported and allowed. See
+ * docs/configuration.md — `selfWebSeed`.
  * @param {string} url - The candidate web seed.
  * @returns {{ok: boolean, why?: string, warning?: string}} - Whether to
  *   publish it, and what to say about it either way.
@@ -343,24 +309,10 @@ export function reachability(url) {
 }
 
 /**
- * The base a URL gets when it is going to outlive the request that made it.
- *
- * Almost every URL this node emits is worked out per request, on purpose: a
- * node answering on several domains should name itself as whichever one was
- * asked, and `publicUrl` is deliberately left unset to allow that. That is the
- * right answer for a TileJSON, a `.torrent` link or a style URL — read once,
- * by whoever asked, and correct for them.
- *
- * A web seed is not that. It is written into the `.torrent` and the magnet and
- * served byte for byte to everyone who asks for either, so it has to be one
- * address rather than whichever the last request happened to arrive on. This
- * is where that address comes from, in order of how deliberate it is:
- *
- * 1. Given outright — the field beside the switch in the console.
- * 2. `publishingUrl`, the node's answer for exactly this question.
- * 3. `publicUrl`, if a node has overridden everything anyway.
- * 4. The request, which is a guess, but the same guess every other URL makes.
+ * The base for a URL that will outlive the request that made it.
  *
+ * Given outright, then `publishingUrl`, then `publicUrl`, then the request.
+ * See docs/configuration.md — `publishingUrl`.
  * @param {object} [options] - `explicit`, `config` and `requestBase`.
  * @returns {string} - A base with no trailing slash, or an empty string.
  */
@@ -372,9 +324,7 @@ export function publishingBase({ explicit, config, requestBase } = {}) {
     requestBase,
   ];
   for (const candidate of candidates) {
-    // An empty or whitespace value means "not set", not "use an empty base" —
-    // the same reading `publicUrl` gets, and for the same reason: clearing a
-    // key by emptying it is what an operator naturally does to a JSON file.
+    // Empty means "not set", not "use an empty base".
     const value = String(candidate ?? '').trim();
     if (value) return value.replace(/\/+$/, '');
   }

package/src/config.js

@@ -166,82 +166,29 @@ const DEFAULTS = {
    */
   md5: false,
   /**
-   * Answer `/archives/<infohash>/archive.pmtiles` — the archive itself, by
-   * byte range, which is what every PMTiles reader actually wants.
-   *
-   * Off by default, and deliberately so: this is the whole file, and an
-   * archive here can be 700 GiB. Everything else this node publishes is either
-   * small (TileJSON, a .torrent, a feed) or metered by the request (one tile),
-   * so turning a node on has never meant offering its disk to anyone who knows
-   * an infohash. This would, so it is asked for rather than assumed.
-   *
-   * The node's answer, not the only one: a watched folder, a scheduled source
-   * or an individual archive may carry its own, and is obeyed either way.
+   * Answer `/archives/<infohash>/archive.pmtiles`. Off by default: it is the
+   * whole file. See docs/configuration.md — `serveArchive`.
    */
   serveArchive: false,
   /**
-   * Write this node's own archive URL into the torrent's `url-list`, so every
-   * peer in the swarm can fetch from it over HTTP.
-   *
-   * A web seed is the difference between a cold tile taking tens of seconds
-   * and taking under one, and it is what makes an archive usable before it has
-   * any peers at all. It is also an open invitation: a seed URL is followed by
-   * everyone who holds the torrent, not only by people who visit this node.
-   *
-   * Means nothing without `serveArchive`, and is read as off without it — a
-   * web seed URL that refuses the request is worse than none, because a client
-   * spends its retries on it.
+   * Publish this node as a web seed for the archives it holds. Read as off
+   * without `serveArchive`. See docs/configuration.md — `selfWebSeed`.
    */
   selfWebSeed: false,
   /**
-   * Offer the archive as a download on the public catalogue page.
-   *
-   * Separate from `serveArchive` on purpose. Serving a file to a reader that
-   * was given the URL and advertising it on a page are different acts, and a
-   * node can reasonably want the first without the second: the endpoint exists
-   * for a style or a peer, and putting a 700 GiB link in front of every casual
-   * visitor is a different decision. Read as off without `serveArchive`, since
-   * a link that answers 403 is worse than no link.
+   * Offer the archive as a download on the public catalogue page. Needs
+   * `serveArchive` and a complete copy. See docs/configuration.md.
    */
   publicDownload: false,
   /**
-   * Answer a byte range for an archive this node does not hold, by pulling the
-   * covering pieces out of the swarm on demand.
-   *
-   * **Experimental, and not recommended for anything public.** It closes the
-   * loop cache mode was built for — the node holds no bytes, a reader asks for
-   * some, and the pieces arrive from peers — over the same path the tile
-   * endpoint has always used internally, sharing its piece cache and its open
-   * handle. As a way to point an ordinary PMTiles reader at an archive nothing
-   * here has a copy of, it works.
-   *
-   * The reservations are properties of the arrangement rather than of the
-   * implementation:
-   *
-   * - Every byte is somebody else's upload. A cache-mode node is not an origin,
-   *   and putting one behind a URL that looks like one turns each request into
-   *   swarm traffic it neither paid for nor holds.
-   * - A piece read takes as long as the swarm takes. Acceptable for a tile,
-   *   which a reader asked for and will wait on; poor for an HTTP client with
-   *   its own patience and its own timeout.
-   * - There is no honest answer to a request for the whole file, so a Range
-   *   header is required and a large one is refused.
+   * Answer a byte range for an archive this node does not hold, from the
+   * swarm. Experimental, and not for anything public — see
+   * docs/configuration.md — `serveArchiveFromSwarm`.
    */
   serveArchiveFromSwarm: false,
-  /**
-   * The largest range `serveArchiveFromSwarm` will fetch at once, in bytes.
-   *
-   * Sized for what a PMTiles reader actually asks for — a 16 KiB header, a
-   * directory, a tile — rather than for bulk transfer, which is the thing this
-   * must not quietly become.
-   */
+  /** The largest range `serveArchiveFromSwarm` will fetch at once, in bytes. */
   swarmRangeLimitBytes: 8 * 1024 * 1024,
-  /**
-   * How long to wait for the swarm before giving up on a range, in
-   * milliseconds. Shorter than the piece timeout underneath on purpose: an
-   * HTTP client gives up long before libtorrent does, and a request left
-   * hanging on a piece nobody is seeding holds a connection open for nothing.
-   */
+  /** How long to wait for the swarm before giving up on a range, in ms. */
   swarmRangeTimeoutMs: 30000,
   /**
    * How long an unfinished download is kept before startup treats it as
@@ -274,23 +221,9 @@ const DEFAULTS = {
   /** Public base URL, used to build absolute links in the RSS feed and TileJSON. */
   publicUrl: undefined,
   /**
-   * The one address to use for URLs that outlive the request that made them.
-   *
-   * Almost every URL this node emits is worked out per request, deliberately: a
-   * node answering on several domains should name itself as whichever one was
-   * asked for, and leaving `publicUrl` unset is what allows that. Read once, by
-   * whoever asked, a per-request answer is the correct answer.
-   *
-   * A web seed is not read once. It is written into the `.torrent` and the
-   * magnet, served byte for byte to everyone who fetches either, and never
-   * rewritten — so it has to be one address rather than whichever the last
-   * request happened to arrive on. Set this to that address.
-   *
-   * Narrower than `publicUrl` on purpose. `publicUrl` overrides every URL the
-   * node emits and so gives up the multi-domain behaviour; this overrides only
-   * the ones that have to be permanent. Unset falls back to `publicUrl`, then
-   * to the request — and the console lets the address be typed at the moment
-   * the switch is turned on, which beats both.
+   * The address for URLs that outlive the request that made them — today, web
+   * seeds. Narrower than `publicUrl`, which overrides every URL and so gives
+   * up answering on several domains. See docs/configuration.md.
    */
   publishingUrl: undefined,
   /**

package/src/engines/composite.js

@@ -430,20 +430,30 @@ export class CompositeEngine {
    * therefore re-hashed its whole store on every start, which for 800 GB is
    * half an hour of disk before it serves anything.
    * @param {string} [infoHash] - One archive, or all of them when omitted.
-   * @returns {Promise<void>} - Resolves once every engine has been asked.
+   * @returns {Promise<{written: number, asked: number}>} - Totals across every
+   *   engine that keeps resume data.
    */
   async saveResume(infoHash) {
+    // Summed rather than dropped, so a caller can tell the difference between
+    // "every torrent wrote" and "half of them will be re-hashed on the next
+    // start". An engine that fails outright contributes nothing to either
+    // total, which is right: it was never asked in a way that counted.
+    let written = 0;
+    let asked = 0;
     for (const engine of [this.#primary, ...this.#secondaries]) {
       // WebTorrent keeps none, and says so by not offering the method.
       if (!engine.saveResume) continue;
       try {
-        await engine.saveResume(infoHash);
+        const result = (await engine.saveResume(infoHash)) ?? {};
+        written += Number(result.written) || 0;
+        asked += Number(result.asked) || 0;
       } catch (error) {
         console.warn(
           `[composite] ${engine.name} could not save resume data: ${error.message}`,
         );
       }
     }
+    return { written, asked };
   }
 
   /**

package/src/engines/libtorrent.js

@@ -759,10 +759,17 @@ export class LibtorrentEngine {
   /**
    * Persists resume data, so the next start skips re-hashing the store.
    * @param {string} [infoHash] - One torrent, or all when omitted.
-   * @returns {Promise<void>} - Resolves once saved.
+   * @returns {Promise<{written: number, asked: number}>} - How many torrents
+   *   were told to write resume data, and how many actually did before the
+   *   deadline.
    */
   async saveResume(infoHash) {
-    await this.#call('save_resume', { infoHash });
+    // Returned rather than discarded. The sidecar reports both numbers, and
+    // the gap between them is the thing worth knowing: a torrent that did not
+    // write is one that gets re-hashed on the next start, which for a 700 GiB
+    // archive is the difference between seeding in seconds and seeding in half
+    // an hour. That answer was being thrown away here.
+    return this.#call('save_resume', { infoHash });
   }
 
   /**

package/src/index.js

@@ -529,6 +529,21 @@ PMTILES_SWARM_PUBLIC_URL
     resumeTimer = setInterval(() => {
       engine
         .saveResume()
+        .then((result) => {
+          // Said out loud, because the shortfall is the thing that costs.
+          // A torrent that did not write inside the deadline is one that gets
+          // re-hashed on the next start, and for a 700 GiB archive that is the
+          // difference between seeding in seconds and seeding in half an hour
+          // -- which is exactly what "why is everything at 0%" looks like from
+          // the outside. Silence here is what made that hard to see.
+          const { written = 0, asked = 0 } = result ?? {};
+          if (asked > 0 && written < asked) {
+            console.warn(
+              `[resume] ${written} of ${asked} torrents wrote resume data ` +
+                'in time; the rest will be re-hashed on the next start',
+            );
+          }
+        })
         .catch((error) =>
           console.warn(`[resume] could not save: ${error.message}`),
         );

package/src/library.js

@@ -528,17 +528,8 @@ export class Library {
   async finalize(infoHash) {
     const settled = await this.#finalizeOnce(infoHash);
 
-    // Only now, and never at import. A web seed URL for an archive that is
-    // still arriving answers 409, and a peer handed a URL that refuses spends
-    // its retries on it — worse than no web seed at all, and unfixable
-    // afterwards, because the URL is in the .torrent every peer holds. So a
-    // subscription records the intention when it joins and it is acted on
-    // here, at the first moment this node actually holds the whole file.
-    //
-    // Safe to reach on an archive that was already complete: setPublishing
-    // works from what is on record rather than from a transition, so this does
-    // nothing the second time — and does the right thing the first time for an
-    // archive that finished before the setting existed.
+    // Only now: a web seed for an archive still arriving answers 409, and the
+    // URL cannot be taken back out of the torrents peers already hold.
     if (settled && publishingFor(settled, this.#config).selfWebSeed) {
       try {
         await this.setPublishing(infoHash, {});
@@ -2713,13 +2704,9 @@ export class Library {
   /**
    * Rewrites an archive's web seed list, in the .torrent and everywhere else.
    *
-   * This is safe on a torrent already in circulation. BEP 19's `url-list` is a
-   * top-level key in the metainfo and the infohash covers only the `info`
-   * dictionary, so changing one leaves the infohash untouched — every magnet,
-   * peer and published reference stays valid. The check below asserts that
-   * rather than trusting it: if a rewrite ever did change the infohash, the
-   * result would be a different torrent wearing the old one's name, which is
-   * worth refusing loudly.
+   * Safe on a torrent in circulation: `url-list` sits outside the info
+   * dictionary, so the infohash is untouched. Asserted below rather than
+   * trusted.
    * @param {object} entry - The catalog entry to rewrite.
    * @param {string[]} wanted - The complete new list.
    * @returns {Promise<{webSeeds: string[], parsed: object}>} - The new list and
@@ -2821,13 +2808,10 @@ export class Library {
   }
 
   /**
-   * Drops web seeds from an archive, leaving everything else about it alone.
+   * Drops web seeds from an archive, leaving everything else alone.
    *
-   * The running engine is not told. libtorrent has no "forget this url seed"
-   * that every version answers to, and the consequence of it keeping one is
-   * bounded — it retries a URL that refuses and gives up on it. What matters
-   * is that the .torrent and the magnet stop handing the URL to anybody new,
-   * which is what this does.
+   * The running engine is not told: there is no portable way to retract a url
+   * seed, and what matters is that nobody new is handed it.
    * @param {string} infoHash - The archive.
    * @param {string[]} urls - Web seed URLs to drop. Unknown ones are ignored.
    * @returns {Promise<{webSeeds: string[], dropped: string[]}>} - What is left,
@@ -2857,17 +2841,9 @@ export class Library {
   /**
    * Sets what this node offers of one archive's own bytes over HTTP.
    *
-   * Three separate switches — see `publishingFor` in catalog.js for why they
-   * are separate — and one side effect: turning `selfWebSeed` on writes this
-   * node's own archive URL into the torrent's `url-list`, and turning it off
-   * takes that URL back out. The URL used is remembered, because the base can
-   * change underneath a node and removing "whatever we would build today"
-   * would leave yesterday's URL in the torrent for ever.
-   *
-   * Turning `serveArchive` off takes the other two with it, and that is not a
-   * quiet tidy-up: it means a URL already handed to every peer holding the
-   * torrent stops answering. The caller is told what was withdrawn so it can
-   * say so.
+   * `selfWebSeed` has a side effect: it writes this node's URL into the
+   * torrent, and takes it out again. See docs/configuration.md — "Offering the
+   * archive file itself".
    * @param {string} infoHash - The archive.
    * @param {object} changes - Any of the three, as booleans. Anything else is
    *   ignored, so a caller may send only what it is changing.
@@ -2878,20 +2854,14 @@ export class Library {
     const entry = this.#catalog.get(infoHash);
     if (!entry) throw new Error('unknown archive');
 
-    // Only what was actually asked for is recorded. An archive that says
-    // nothing about a setting goes on deferring to the node, which is what
-    // makes changing the node's answer reach the archives that never had one
-    // of their own.
+    // Only what was asked for: an archive that says nothing defers to the node.
     const wanted = { ...entry };
     for (const key of ['serveArchive', 'selfWebSeed', 'publicDownload']) {
       if (typeof changes[key] === 'boolean') wanted[key] = changes[key];
     }
 
-    // With serving off the other two cannot be true, and writing that down
-    // matters more than it looks: left as a latent `true`, either would spring
-    // back the moment serving was turned on again — re-publishing this node as
-    // a web seed, or re-listing a 700 GiB download, as a side effect of a
-    // decision about something else.
+    // Written down rather than left latent, or either would spring back the
+    // next time serving was turned on.
     if (!publishingFor(wanted, this.#config).serveArchive) {
       wanted.selfWebSeed = false;
       wanted.publicDownload = false;
@@ -2899,21 +2869,14 @@ export class Library {
     const after = publishingFor(wanted, this.#config);
     await this.#catalog.put(wanted);
 
-    // Driven by what is on record rather than by the transition, so calling
-    // this twice does nothing the second time and calling it on an archive
-    // that was created with the setting already on still writes the seed. A
-    // transition test looked equivalent and was not: an import that inherits
-    // `selfWebSeed: true` from the node has no "before" in which it was off,
-    // so nothing would ever have published it.
+    // Driven by what is on record, not by a transition: an import that
+    // inherits the setting has no "before" in which it was off.
     const published = entry.selfWebSeedUrl ?? null;
     let webSeed = published;
     let warning = null;
 
-    // A node cannot be a web seed for bytes it does not have. It might be able
-    // to *answer* for them, where serveArchiveFromSwarm is on -- but answering
-    // by fetching from the swarm and then advertising that to the swarm is a
-    // loop with an amplifier in it: every peer that takes the seed makes this
-    // node download the piece again to serve it.
+    // Answering from the swarm and advertising that to the swarm is a loop
+    // with an amplifier in it.
     if (after.selfWebSeed && !published && entry.complete === false) {
       throw new Error(
         'this node does not hold a complete copy of this archive, so it ' +
@@ -2937,10 +2900,7 @@ export class Library {
         );
       }
       webSeed = `${base}/archives/${infoHash}/archive.pmtiles`;
-      // Checked before it goes anywhere. Nothing rewrites a web seed once it
-      // is in a .torrent -- the file is served byte for byte to everyone who
-      // asks -- so a URL that cannot work is not a mistake that gets corrected
-      // on the next request. It is distributed and then retried for ever.
+      // Nothing rewrites a web seed once it is in a .torrent.
       const reach = reachability(webSeed);
       if (!reach.ok) throw new Error(reach.why);
       if (reach.warning) console.warn(`[web seed] ${reach.warning}`);
@@ -2952,10 +2912,7 @@ export class Library {
       // and then vanish in the same call.
       await this.#catalog.put({ infoHash, selfWebSeedUrl: webSeed });
     } else if (!after.selfWebSeed && published) {
-      // Whatever was actually published, which is not necessarily the URL this
-      // node would build for itself today: the base can change underneath a
-      // node, and removing "whatever we would say now" would leave yesterday's
-      // URL in the torrent for ever.
+      // What was published, not what this node would build today.
       await this.removeWebSeeds(infoHash, [published]);
       await this.#catalog.put({ infoHash, selfWebSeedUrl: null });
       webSeed = null;
@@ -2964,12 +2921,8 @@ export class Library {
     return {
       ...after,
       webSeed,
-      // Published anyway, and said out loud. A node syncing to its own peers
-      // across a LAN is a real arrangement and not this code's to overrule.
       warning,
-      // Named so a caller can warn about the one change that is not merely a
-      // setting moving: this URL is already in the hands of every peer that
-      // holds the torrent, and they will go on trying it for a while.
+      // Peers holding the torrent keep trying the URL until they refresh.
       withdrewWebSeed: Boolean(published) && !after.selfWebSeed,
     };
   }
@@ -3165,12 +3118,8 @@ export class Library {
       stale: false,
     });
 
-    // Publishing this node as a web seed is the one of the three with an
-    // effect beyond a stored boolean: it writes a URL into the .torrent and
-    // the magnet. Done here so a watched folder and an RSS subscription get it
-    // without each having to know, and warned rather than thrown because an
-    // import that produced a good archive should not be failed by a seed URL
-    // that could not be built.
+    // Here rather than in each caller, and warned rather than thrown: a good
+    // archive should not be failed by a seed URL that could not be built.
     if (publishingFor(entry, this.#config).selfWebSeed) {
       try {
         await this.setPublishing(entry.infoHash, {});

package/src/pmtiles-probe.js

@@ -18,6 +18,8 @@ const TILE_TYPES = {
   3: { format: 'jpeg', contentType: 'image/jpeg' },
   4: { format: 'webp', contentType: 'image/webp' },
   5: { format: 'avif', contentType: 'image/avif' },
+  // No MIME is registered for MLT, and nothing selects a tile by content type.
+  6: { format: 'mlt', contentType: 'application/octet-stream' },
 };
 
 /**
@@ -59,34 +61,23 @@ export function metadataFlag(value) {
 }
 
 /**
- * What a `raster-dem` archive says about how its elevations are packed.
+ * The tile encoding an archive declares, if it is one MapLibre understands.
  *
- * Nothing in a PMTiles header says this — the header knows the tile is WebP,
- * not what the three channels mean — so the only place it can come from is the
- * archive's own metadata. Without it a consumer falls back to a default, and
- * the default is wrong for exactly the archives that most need to say
- * something: a terrarium-packed DEM read as `mapbox` decodes every mountain
- * into noise, silently, with a plausible-looking map on screen.
- *
- * The values are the ones the style specification defines, and anything else
- * is dropped rather than passed on. A tile client handed an encoding it does
- * not recognise is worse off than one handed nothing, because nothing at least
- * leaves it free to use its own default.
+ * See docs/tilejson.md — `encoding`.
  * @param {unknown} value - Whatever the metadata held.
  * @returns {string | undefined} - A known encoding, or undefined.
  */
 export function metadataEncoding(value) {
   if (typeof value !== 'string') return undefined;
   const text = value.trim().toLowerCase();
-  return ['terrarium', 'mapbox', 'custom'].includes(text) ? text : undefined;
+  // An unrecognised value is dropped: it would cost a client its own default.
+  return ['terrarium', 'mapbox', 'custom', 'mlt'].includes(text)
+    ? text
+    : undefined;
 }
 
 /**
- * The numbers a `custom` encoding is meaningless without.
- *
- * `encoding: "custom"` says "the channels mean what these four factors say",
- * so carrying the word and not the factors publishes an archive nobody can
- * read. They travel together or not at all.
+ * The four factors a `custom` encoding is unreadable without. All or nothing.
  * @param {object} metadata - The archive's metadata.
  * @returns {object | undefined} - The four factors, or undefined.
  */
@@ -111,22 +102,11 @@ export function customEncodingFactors(metadata) {
  * @returns {Promise<PMTilesSummary>} - The summary.
  */
 /**
- * What this prober reads, as a number that goes up when that changes.
- *
- * A summary is stored in the catalog and never read again, which is right --
- * re-reading a header out of the swarm is not free, and the answer does not
- * change for a given infohash. It goes wrong the moment the prober learns to
- * read something new: every archive probed before that keeps a summary with a
- * hole in it, for ever, and the only way out was to remove and re-add it.
- *
- * `encoding` was the case that made this obvious. It sat in the metadata of
- * archives this node had been serving for months, and adding the code to read
- * it changed nothing at all, because nothing ever asked again.
- *
- * So: stamp what the prober knew at the time, and re-read once when that is
- * behind. Raise this whenever a field is added to the summary below.
+ * What this prober reads. Raise it whenever a field is added to the summary,
+ * and archives probed by an older build are re-read once. See
+ * docs/internals.md — "Re-reading a summary an older prober wrote".
  */
-export const SUMMARY_VERSION = 2;
+export const SUMMARY_VERSION = 3;
 
 export function summarize(header, metadata = {}) {
   const type = TILE_TYPES[header.tileType] ?? TILE_TYPES[0];
@@ -165,12 +145,9 @@ export function summarize(header, metadata = {}) {
     // the same key, so an archive built to be served there carries the answer
     // with it and does not have to be configured again here.
     sparse: metadataFlag(metadata.sparse),
-    // How a raster-dem archive packs elevation into pixels: terrarium, mapbox
-    // or custom. Read from the archive rather than configured here, for the
-    // same reason `sparse` is — the archive knows, and a mirror of it should
-    // not have to be told again.
-    encoding: metadataEncoding(metadata.encoding),
-    // Only where the encoding is custom, since they mean nothing otherwise.
+    // The header settles MLT; only the metadata can settle elevation packing.
+    encoding:
+      type.format === 'mlt' ? 'mlt' : metadataEncoding(metadata.encoding),
     encodingFactors:
       metadataEncoding(metadata.encoding) === 'custom'
         ? customEncodingFactors(metadata)

package/src/web/index.html

@@ -889,17 +889,30 @@
     <script type="module">
       const $ = (id) => document.getElementById(id);
 
-      /** Formats a byte count for people rather than machines. */
+      /** What this page shows in place of a size it has not got. */
+      const EMPTY = '—';
+      /**
+       * Formats a byte count for people rather than machines.
+       *
+       * Kept character-for-character identical to the same helper on the other
+       * page, and held there by a test. They drifted once: this one rounded to
+       * whole units above ten and the other always kept a decimal, so the same
+       * archive read as 81 GiB on one page and 80.6 GiB on the other. Two
+       * numbers for one fact is worse than either number.
+       *
+       * One decimal from KiB up, because these are mostly archive sizes and
+       * half a gigabyte of difference is worth seeing.
+       */
       const bytes = (n) => {
-        if (!Number.isFinite(n) || n <= 0) return '—';
+        if (!Number.isFinite(n) || n <= 0) return EMPTY;
         const units = ['B', 'KiB', 'MiB', 'GiB', 'TiB'];
         let value = n;
         let unit = 0;
         while (value >= 1024 && unit < units.length - 1) {
           value /= 1024;
-          unit++;
+          unit += 1;
         }
-        return `${value.toFixed(value < 10 && unit > 0 ? 1 : 0)} ${units[unit]}`;
+        return `${value.toFixed(unit === 0 ? 0 : 1)} ${units[unit]}`;
       };
 
       const rate = (n) => (n > 0 ? `${bytes(n)}/s` : '—');

package/src/web/public.html

@@ -262,8 +262,22 @@
         return node;
       };
 
+      /** What this page shows in place of a size it has not got. */
+      const EMPTY = null;
+      /**
+       * Formats a byte count for people rather than machines.
+       *
+       * Kept character-for-character identical to the same helper on the other
+       * page, and held there by a test. They drifted once: this one rounded to
+       * whole units above ten and the other always kept a decimal, so the same
+       * archive read as 81 GiB on one page and 80.6 GiB on the other. Two
+       * numbers for one fact is worse than either number.
+       *
+       * One decimal from KiB up, because these are mostly archive sizes and
+       * half a gigabyte of difference is worth seeing.
+       */
       const bytes = (n) => {
-        if (!Number.isFinite(n) || n <= 0) return null;
+        if (!Number.isFinite(n) || n <= 0) return EMPTY;
         const units = ['B', 'KiB', 'MiB', 'GiB', 'TiB'];
         let value = n;
         let unit = 0;