pmtiles-swarm 0.14.2 → 0.15.1

package/CHANGELOG.md

@@ -7,6 +7,58 @@
 ### 🐞 Bug fixes
 - _...Add new stuff here..._
 
+## 0.15.1
+### 🐞 Bug fixes
+- **The downloaded bar showed nothing on an archive that was plainly downloading.** A column on
+  that bar covers many pieces, and the sidecar reduced "held" by `all` — so a column lit only when
+  every piece beneath it had arrived, and an archive 18% complete showed an empty bar. Fixed in
+  pmtiles-torrent 0.4.4, which reports a proportion; the console now shades those columns by it,
+  with a floor so the first few percent of a download are visible rather than indistinguishable
+  from none.
+
+  Peer bars had the opposite fault and are fixed the same way, so a peer holding a little no longer
+  reads as a seed.
+
+  Renders correctly against either sidecar: the newer one never rounds a non-empty column below 2,
+  so values above 1 identify the new encoding.
+
+## 0.15.0
+### ✨ Features and improvements
+- **The DHT routing table is remembered between runs**, which is the difference between publishing
+  reliably and gambling on each start. Field measurements on a domestic connection: a fresh socket
+  usually found one node and never recovered, while roughly one start in seven found sixteen within
+  five seconds — and no amount of retrying rescued a bad one.
+
+  This is what libtorrent does, and why the libtorrent engine's DHT works on hosts where a fresh
+  bittorrent-dht socket does not: it saves its table and reloads it rather than bootstrapping cold
+  every time. Saved to `dht-nodes.json` in the data directory (`mutable.statePath`), written every
+  five minutes and on shutdown, and never overwritten with an empty table — a bad run must not
+  replace a good table with its own nothing.
+
+  The bootstrap list also now matches libtorrent's rather than the library default, adding
+  `dht.libtorrent.org:25401` and `router.bitcomet.com:6881`. Remembered nodes are tried first and
+  the hostnames stay behind them, since a saved table can be entirely stale.
+- **A DHT socket that cannot reach the network is replaced rather than retried.** Retrying on a bad
+  socket failed at 30s, 60s, 2m and 4m in the field while a restart succeeded, so after two futile
+  cycles the publisher now opens a new socket itself instead of waiting for someone to restart the
+  service. A socket that is finding peers is never replaced, whatever its puts are doing.
+
+## 0.14.3
+### 🐞 Bug fixes
+- **The publisher waits for a usable routing table rather than a single node.** A freshly
+  bootstrapped table holds one entry — the bootstrap host, which stores nothing — so publishing
+  against it failed once per category with "No nodes to query" before any retry could help. It now
+  waits for eight, for up to two minutes, and says what it is waiting for as it goes.
+
+  Two minutes rather than longer because this turned out to be bimodal rather than slow: a socket
+  that can reach the DHT fills its table in a few seconds, and one that cannot is still empty ten
+  minutes later. Waiting past that buys nothing and delays saying so.
+
+  Worth knowing where that comes from, since the log looks like a swarm problem and is not: on a
+  multi-WAN router each new UDP socket is assigned a gateway by the load balancer and then keeps
+  it, so a socket that lands on a WAN with no working return path never recovers — which is why
+  retrying could not rescue a bad run and only a restart changed the outcome.
+
 ## 0.14.2
 ### 🐞 Bug fixes
 - **Waiting for the DHT's `ready` event was not enough.** It fires when the bootstrap lookup

package/README.md

@@ -115,8 +115,31 @@ subscribers forward, and they fail differently, so publishing both is cheap insu
 
 - **RSS** — easy to consume, understood by existing clients, needs a server that stays up.
 - **BEP 46** — an ed25519-signed DHT record naming the current infohash, addressed by public key
-  rather than infohash (`magnet:?xs=urn:btpk:…`). No server needed, but the record expires and must
-  be republished. See [src/mutable.js](src/mutable.js).
+  rather than infohash (`magnet:?xs=urn:btpk:…`). No server needed at all.
+
+Publishing those records is built in. The node that builds gets a key — `pmtiles-swarm
+publisher-key` — and announces the newest archive in each category, salted by category name so one
+keypair addresses all of them. Records expire from the DHT after about two hours, so it republishes
+on a timer; that timer is the feature rather than an optimisation. Serving nodes need nothing: they
+receive the public half on the catalog entry and hand it out in the TileJSON, which is why a
+serving tier can be compromised without anyone being able to publish.
+
+```json
+{ "mutable": { "publish": true, "keyPath": "/etc/pmtiles-swarm/publisher.pem" } }
+```
+
+The routing table is remembered between runs (`dht-nodes.json` in the data directory), which
+matters more than it sounds: bootstrapping from hostnames alone was measured working about one
+start in seven on a domestic connection, and a saved table turns that into every start. It is the
+same thing libtorrent does, and why its DHT works on hosts where a fresh socket does not.
+
+**What a style should point at** is then the category's TileJSON URL with that magnet in its
+fragment — the console's Categories page has a **For a style** row that gives you the whole
+string. A fragment is never sent in an HTTP request, so ordinary clients fetch the TileJSON and
+ignore it, while a swarm-aware one reads the magnet before making any call and can start when the
+server cannot answer. With `xs=urn:btpk:` rather than an infohash it does not go stale on the next
+build either. See [docs/serving-tiles.md](docs/serving-tiles.md) and
+[docs/security.md](docs/security.md), because that key behaves unlike every other secret here.
 
 ## Configuration
 
@@ -742,9 +765,13 @@ another node, watched web locations including directory listings and date templa
 limits and what is left of them, in-place settings reload and process restart, and two-node
 subscription sync in both directions over both RSS and the catalog API.
 
-Not yet exercised: the qBittorrent engine against a real instance, watch-folder imports, and
-BEP 46 publish/resolve against a live DHT (the crypto and magnet handling are tested; interop
-with libtorrent's encoding is not).
+Running in production: watch-folder imports feeding a nightly planet build, an 18-archive library
+of roughly 2.5 TB behind HAProxy, and BEP 46 records published to a live DHT — 54 nodes stored the
+last one.
+
+Not yet exercised: the qBittorrent engine against a real instance, and **resolving** a BEP 46
+record from outside the publishing network. Nodes accept the records and report storing them, which
+is not the same claim as a stranger reading one back, and only the second matters to a subscriber.
 
 ## License and attribution
 

package/docs/publishing.md

@@ -779,4 +779,12 @@ rebuild, which fail differently, so publishing both is cheap insurance:
   public key rather than infohash (`magnet:?xs=urn:btpk:…`). No server required, but the
   record expires and must be republished.
 
-See [subscribing.md](subscribing.md).
+Publishing those records is built in: give the building node a key with `pmtiles-swarm
+publisher-key`, set `mutable.publish`, and it announces the newest archive in each category
+and republishes on a timer. Serving nodes need nothing — they receive the public half on
+the catalog entry and hand it out in the TileJSON.
+
+See [subscribing.md](subscribing.md) for the shape of it,
+[serving-tiles.md](serving-tiles.md#a-fragment-that-survives-a-rebuild) for what a style
+should then point at, and [security.md](security.md#the-publisher-key-is-not-a-credential)
+for why that key wants treating like a signing key rather than an API token.

package/docs/subscribing.md

@@ -419,9 +419,29 @@ RSS and BEP 46 fail in opposite ways, which is the argument for publishing both:
 | RSS | a server that stays up | the server goes away |
 | BEP 46 | periodic republishing | the record expires (hours, not days) |
 
-**Status:** the crypto and magnet handling are tested — 32-byte keys, 64-byte signatures
-verifying, roundtrip stable. Publishing and resolving against a live DHT, and interop
-with libtorrent's exact value encoding, are **not yet verified**.
+**This is built in.** The code above is what a host application would call; a node only
+needs a key and a setting:
+
+```sh
+pmtiles-swarm publisher-key > /etc/pmtiles-swarm/publisher.pem
+chmod 400 /etc/pmtiles-swarm/publisher.pem
+```
+
+```json
+{ "mutable": { "publish": true, "keyPath": "/etc/pmtiles-swarm/publisher.pem" } }
+```
+
+It then announces the newest archive in each category, salted by category name so one key
+addresses all of them, and republishes before the records expire. Only the node that builds
+needs the key — see
+[running-as-a-service.md](running-as-a-service.md#the-publisher-key) for setting it up and
+[security.md](security.md#the-publisher-key-is-not-a-credential) for why it is not a
+credential.
+
+**Status:** publishing is verified against a live DHT — records stored by 54 nodes at the
+last check. **Resolving** one from outside the publishing network is not yet verified:
+nodes accepting a record is not the same claim as a stranger reading it back, and only the
+second matters to a subscriber.
 
 ## Making the archives serveable
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "pmtiles-swarm",
-  "version": "0.14.2",
+  "version": "0.15.1",
   "description": "BitTorrent distribution for PMTiles map archives: create torrents, watch folders, publish and subscribe to RSS feeds, and seed through qBittorrent or an embedded client",
   "type": "module",
   "main": "src/index.js",

package/src/config.js

@@ -438,6 +438,15 @@ const DEFAULTS = {
      * own, and two sockets cannot hold one port.
      */
     dhtPort: 0,
+    /**
+     * Where to remember the DHT routing table between runs.
+     *
+     * Defaults to `dht-nodes.json` in the data directory. Bootstrapping from
+     * hostnames alone is unreliable enough that a node doing it on every start
+     * is gambling each time; libtorrent saves its table for the same reason,
+     * which is why its DHT works on hosts where a fresh socket does not.
+     */
+    statePath: undefined,
   },
   /**
    * What this node has served, at `GET /api/stats`.

package/src/dht-state.js

@@ -0,0 +1,76 @@
+/**
+ * Keeping a DHT routing table across restarts.
+ *
+ * Bootstrapping is not reliably quick. Measured on a domestic connection, a
+ * fresh socket usually found one node and never recovered, while roughly one
+ * attempt in seven found sixteen within five seconds — and no amount of
+ * retrying rescued a bad one. A node that has to bootstrap from nothing on
+ * every start is therefore gambling on each start.
+ *
+ * libtorrent does not gamble: it saves its routing table and reloads it, which
+ * is why the libtorrent engine's DHT works on hosts where a fresh
+ * bittorrent-dht socket does not. This does the same thing — a table that
+ * worked once is remembered, so later starts begin with peers that answered
+ * recently rather than with three hostnames.
+ */
+
+import fs from 'node:fs/promises';
+
+/**
+ * Bootstrap hosts, as libtorrent uses rather than the shorter default.
+ *
+ * More names is more chances that one answers, and they cost nothing when the
+ * saved table already works. `dht.libtorrent.org` in particular is absent from
+ * bittorrent-dht's defaults.
+ */
+export const BOOTSTRAP = [
+  'router.bittorrent.com:6881',
+  'dht.transmissionbt.com:6881',
+  'router.utorrent.com:6881',
+  'dht.libtorrent.org:25401',
+  'router.bitcomet.com:6881',
+];
+
+/**
+ * Nodes to start from: whatever was saved, plus the bootstrap hosts.
+ * @param {string} [path] - Where the table was saved.
+ * @returns {Promise<Array>} - Addresses for the `bootstrap` option.
+ */
+export async function loadNodes(path) {
+  if (!path) return [...BOOTSTRAP];
+  try {
+    const saved = JSON.parse(await fs.readFile(path, 'utf8'));
+    const nodes = (saved.nodes ?? [])
+      .filter((node) => node?.host && node?.port)
+      .map((node) => `${node.host}:${node.port}`);
+    // The hostnames stay in the list. A saved table can be entirely stale —
+    // a laptop that moved networks, a node that was off for a week — and
+    // falling back to bootstrapping is better than starting nowhere.
+    return [...nodes, ...BOOTSTRAP];
+  } catch {
+    // No file yet, or an unreadable one. Neither is worth reporting: the
+    // bootstrap hosts are a complete answer on their own.
+    return [...BOOTSTRAP];
+  }
+}
+
+/**
+ * Saves the current routing table.
+ * @param {string} [path] - Where to write it.
+ * @param {object} dht - The DHT to read.
+ * @returns {Promise<number>} - How many nodes were saved.
+ */
+export async function saveNodes(path, dht) {
+  if (!path || !dht?.toJSON) return 0;
+  const nodes = dht.toJSON().nodes ?? [];
+  // An empty table is not worth writing, and writing it would replace a good
+  // table from a previous run with the results of a bad one.
+  if (nodes.length === 0) return 0;
+
+  const body = JSON.stringify({ savedAt: new Date().toISOString(), nodes });
+  // Written then renamed, so a crash mid-write cannot leave a truncated file
+  // that the next start has to fail on.
+  await fs.writeFile(`${path}.tmp`, body);
+  await fs.rename(`${path}.tmp`, path);
+  return nodes.length;
+}

package/src/index.js

@@ -1,5 +1,6 @@
 #!/usr/bin/env node
 import fs from 'node:fs/promises';
+import path from 'node:path';
 import { parseArgs } from 'node:util';
 import { createApp } from './api.js';
 import { assertSafeToListen, createAuth } from './auth.js';
@@ -272,29 +273,60 @@ PMTILES_SWARM_PUBLIC_URL
         import('bittorrent-dht').then((m) => m.default),
       ]);
       const pem = await fs.readFile(config.mutable.keyPath, 'utf8');
-      const dht = new DHT();
-      // Bound explicitly rather than left to bind implicitly on first send,
-      // so the port is predictable and can be forwarded if you want this to
-      // be a reachable DHT node. 0 takes an ephemeral one, which is all
-      // publishing needs.
-      await new Promise((resolve, reject) => {
-        dht.once('error', reject);
-        dht.listen(config.mutable.dhtPort ?? 0, resolve);
-      });
-      console.log(`[mutable] DHT on UDP ${dht.address().port}`);
+      // A factory rather than an instance, so the publisher can replace a
+      // socket that proves unable to reach the DHT. Bound explicitly rather
+      // than left to bind implicitly on first send, so the port is
+      // predictable and can be forwarded; 0 takes an ephemeral one, which is
+      // all publishing needs — and is what lets a replacement differ from
+      // what it replaced.
+      const { loadNodes, saveNodes } = await import('./dht-state.js');
+      const statePath =
+        config.mutable.statePath ?? path.join(config.dataDir, 'dht-nodes.json');
+
+      const openDht = async () => {
+        // Started from a table that worked before where there is one, the way
+        // libtorrent does. Bootstrapping from hostnames alone is unreliable
+        // enough that a node doing it on every start is gambling each time.
+        const bootstrap = await loadNodes(statePath);
+        const dht = new DHT({ bootstrap });
+        await new Promise((resolve, reject) => {
+          dht.once('error', reject);
+          dht.listen(config.mutable.dhtPort ?? 0, resolve);
+        });
+        console.log(
+          `[mutable] DHT on UDP ${dht.address().port} ` +
+            `(${bootstrap.length} bootstrap addresses)`,
+        );
+        return dht;
+      };
+
       publisher = new MutablePublisher({
         catalog,
-        dht,
+        createDht: openDht,
         key: publisherKeyFromPem(pem),
         intervalMs: (config.mutable.republishSeconds ?? 1800) * 1000,
       });
+      // Saved periodically rather than only on the way out, because the run
+      // that finds a good table is often the one that is later killed rather
+      // than stopped, and a table nobody wrote down is a table nobody keeps.
+      const saveTable = setInterval(() => {
+        publisher
+          .saveTable((dht) => saveNodes(statePath, dht))
+          .catch(() => {});
+      }, 5 * 60_000);
+      saveTable.unref?.();
+
       stoppers.unshift({
         label: 'mutable publisher',
-        stop: () => {
+        stop: async () => {
+          clearInterval(saveTable);
+          const saved = await publisher
+            .saveTable((dht) => saveNodes(statePath, dht))
+            .catch(() => 0);
+          if (saved) console.log(`[mutable] remembered ${saved} DHT nodes`);
           publisher.stop();
-          dht.destroy();
         },
-        ms: 2000,
+        ms: 3000,
       });
       publisher.start();
     } catch (error) {

package/src/publisher.js

@@ -50,12 +50,48 @@ import { mutableMagnet, publishInfoHash } from './mutable.js';
 /** Republish well inside the ~2h a DHT keeps an item. */
 const DEFAULT_INTERVAL_MS = 30 * 60 * 1000;
 
-/** How long to wait for the DHT to bootstrap before publishing anyway. */
-const DEFAULT_READY_MS = 60_000;
+/**
+ * How long to wait for the DHT to find peers before publishing anyway.
+ *
+ * Minutes rather than seconds, because bootstrapping is not uniformly fast and
+ * publishing early does not fail gracefully -- it fails once per category with
+ * a message that reads like something is broken.
+ *
+ * Not longer, because in practice this is bimodal rather than slow: a socket
+ * that can reach the DHT fills its table in a few seconds, and one that cannot
+ * is still empty ten minutes later. Observed on a multi-WAN router, where each
+ * new socket gets a gateway assigned by the load balancer and is then stuck
+ * with it -- so retrying never rescued a bad socket, and only a restart
+ * re-rolled it. Waiting past a couple of minutes buys nothing and delays
+ * saying so.
+ */
+const DEFAULT_READY_MS = 2 * 60_000;
+
+/** How often to say that it is still waiting, so a slow start is legible. */
+const WAITING_LOG_MS = 60_000;
 
 /** First retry delay after an attempt where nothing was published. */
 const RETRY_MS = 30_000;
 
+/**
+ * Failed cycles before opening a new DHT socket.
+ *
+ * A socket that cannot reach the DHT never recovers: retrying on it failed at
+ * 30s, 60s, 2m and 4m in the field, while a restart -- a new socket, a new
+ * source port, a new NAT state -- worked roughly one attempt in seven. So when
+ * retrying is demonstrably futile, re-roll rather than wait for a human.
+ */
+const REROLL_AFTER = 2;
+
+/**
+ * Nodes wanted before a first publish is worth attempting.
+ *
+ * One is what a freshly bootstrapped table holds, and a put against it fails
+ * with "No nodes to query" -- the entry is the bootstrap host, not a peer that
+ * will store anything.
+ */
+const MINIMUM_NODES = 8;
+
 export class MutablePublisher {
   #catalog;
   #dht;
@@ -64,6 +100,10 @@ export class MutablePublisher {
   #timer = null;
   #retryTimer = null;
   #stopped = false;
+  #createDht = null;
+  #ownsDht = false;
+  /** Consecutive cycles that published nothing, for deciding to re-roll. */
+  #failedCycles = 0;
   #log;
   /** Last published infohash per category, so an unchanged build stays quiet. */
   #published = new Map();
@@ -71,14 +111,19 @@ export class MutablePublisher {
   /**
    * @param {object} options - Wiring.
    * @param {object} options.catalog - Catalog to read categories from.
-   * @param {object} options.dht - A bittorrent-dht instance.
+   * @param {object} [options.dht] - A bittorrent-dht instance to use as-is.
+   * @param {Function} [options.createDht] - Opens a fresh one, already listening.
+   *   Given this, the publisher owns the socket and replaces it when it proves
+   *   unable to reach the DHT.
    * @param {object} options.key - Keypair from `publisherKeyFromPem`.
    * @param {number} [options.intervalMs] - Republish interval.
    * @param {Function} [options.log] - Where to report.
    */
-  constructor({ catalog, dht, key, intervalMs, log = console.log }) {
+  constructor({ catalog, dht, createDht, key, intervalMs, log = console.log }) {
     this.#catalog = catalog;
-    this.#dht = dht;
+    this.#dht = dht ?? null;
+    this.#createDht = createDht ?? null;
+    this.#ownsDht = Boolean(createDht) && !dht;
     this.#key = key;
     this.#intervalMs = intervalMs ?? DEFAULT_INTERVAL_MS;
     this.#log = log;
@@ -149,7 +194,10 @@ export class MutablePublisher {
       }
     }
     if (done.length === 0 && this.#current().size > 0) {
+      this.#failedCycles += 1;
       this.#scheduleRetry(options.retryDelayMs ?? RETRY_MS);
+    } else if (done.length > 0) {
+      this.#failedCycles = 0;
     }
     return done;
   }
@@ -175,6 +223,7 @@ export class MutablePublisher {
    * Starts publishing, and keeps republishing before the records expire.
    * @param {object} [options] - Timing.
    * @param {number} [options.readyMs] - Grace before the first publish.
+   * @param {number} [options.retryMs] - First retry delay, for tests.
    * @returns {void}
    */
   start(options = {}) {
@@ -183,7 +232,7 @@ export class MutablePublisher {
     // bet on how long bootstrapping takes -- one this lost in the field, where
     // fifteen seconds was not enough and every category failed on the first
     // attempt.
-    this.#firstPublish(options.readyMs ?? DEFAULT_READY_MS).catch((error) =>
+    this.#firstPublish(options.readyMs ?? DEFAULT_READY_MS, options.retryMs).catch((error) =>
       this.#log(`[mutable] first publish failed: ${error.message}`),
     );
 
@@ -205,23 +254,24 @@ export class MutablePublisher {
    * @param {number} readyMs - How long to wait before going ahead regardless.
    * @returns {Promise<void>} - Resolves once the first attempt is done.
    */
-  async #firstPublish(readyMs) {
+  async #firstPublish(readyMs, retryMs) {
+    if (!this.#dht && this.#createDht) this.#dht = await this.#createDht();
     const nodes = await this.#whenDhtReady(readyMs);
     if (this.#stopped) return;
-    if (nodes === 0) {
+    if (nodes !== null && nodes < MINIMUM_NODES) {
       // Named rather than left to be inferred. A routing table that is still
       // empty after a minute means the bootstrap queries are not being
       // answered, and every "No nodes to query" after this is that same fact
       // reported once per category.
       this.#log(
-        `[mutable] the DHT found no peers in ${Math.round(readyMs / 1000)}s. Publishing ` +
-          'will fail until it does — check that outbound UDP is not blocked, and that ' +
-          'the bootstrap hosts resolve',
+        `[mutable] the DHT found only ${nodes} nodes in ${Math.round(readyMs / 60000)} ` +
+          'minutes. Publishing will keep retrying — if it never succeeds, check that ' +
+          'outbound UDP is not blocked and that the bootstrap hosts resolve',
       );
     } else if (nodes) {
       this.#log(`[mutable] DHT ready with ${nodes} nodes`);
     }
-    await this.publishAll({ force: true });
+    await this.publishAll({ force: true, retryDelayMs: retryMs });
   }
 
   /**
@@ -248,10 +298,25 @@ export class MutablePublisher {
       });
     }
 
+    // Waited out rather than given up on. A table with a single bootstrap
+    // entry is not enough to store a record -- the put needs somewhere to put
+    // it -- so this holds until there are a few, saying so as it goes.
     let count = this.#nodeCount();
-    while (count === 0 && Date.now() < deadline && !this.#stopped) {
+    let lastSaid = Date.now();
+    while (count !== null && count < MINIMUM_NODES && Date.now() < deadline && !this.#stopped) {
+      if (Date.now() - lastSaid >= WAITING_LOG_MS) {
+        lastSaid = Date.now();
+        this.#log(
+          `[mutable] waiting for the DHT (${count} nodes so far). ` +
+            'Bootstrapping can take several minutes on a home connection',
+        );
+      }
+      // Never past the deadline: sleeping a flat two seconds overshoots a
+      // short wait by the whole interval, which makes the wait unpredictable
+      // and the behaviour hard to test.
+      const remaining = deadline - Date.now();
       await new Promise((resolve) => {
-        const timer = setTimeout(resolve, 1000);
+        const timer = setTimeout(resolve, Math.max(10, Math.min(2000, remaining)));
         timer.unref?.();
       });
       count = this.#nodeCount();
@@ -281,17 +346,64 @@ export class MutablePublisher {
    */
   #scheduleRetry(delayMs) {
     if (this.#stopped || this.#retryTimer) return;
-    const timer = setTimeout(() => {
+    const timer = setTimeout(async () => {
       this.#retryTimer = null;
       if (this.#stopped) return;
-      this.publishAll({ retryDelayMs: Math.min(delayMs * 2, this.#intervalMs) }).catch(
-        (error) => this.#log(`[mutable] retry failed: ${error.message}`),
-      );
+      try {
+        await this.#rerollIfHopeless();
+        await this.publishAll({
+          retryDelayMs: Math.min(delayMs * 2, this.#intervalMs),
+        });
+      } catch (error) {
+        this.#log(`[mutable] retry failed: ${error.message}`);
+      }
     }, delayMs);
     timer.unref?.();
     this.#retryTimer = timer;
   }
 
+  /**
+   * Lets a caller read the current DHT, whichever socket that now is.
+   *
+   * Needed because this may replace the socket it was given, so a caller
+   * holding the original would save a table belonging to a socket that has
+   * been closed.
+   * @param {Function} save - Receives the live DHT.
+   * @returns {Promise<*>} - Whatever `save` returns, or 0 with no socket.
+   */
+  async saveTable(save) {
+    if (!this.#dht) return 0;
+    return save(this.#dht);
+  }
+
+  /**
+   * Replaces the DHT socket when retrying on it is demonstrably futile.
+   *
+   * Only when this publisher opened the socket, and only while its table is
+   * still unusable — a socket that is finding peers is working, whatever the
+   * puts are doing, and swapping it would throw away a good one.
+   * @returns {Promise<void>} - Resolves once any replacement is ready.
+   */
+  async #rerollIfHopeless() {
+    if (!this.#ownsDht || this.#failedCycles < REROLL_AFTER) return;
+    const count = this.#nodeCount();
+    if (count === null || count >= MINIMUM_NODES) return;
+
+    this.#log(
+      `[mutable] this DHT socket has found ${count} nodes and is not recovering — ` +
+        'opening a new one',
+    );
+    try {
+      this.#dht?.destroy?.();
+    } catch {
+      // A socket being replaced is not worth reporting if it objects to closing.
+    }
+    this.#dht = await this.#createDht();
+    this.#failedCycles = 0;
+    const found = await this.#whenDhtReady(DEFAULT_READY_MS);
+    if (found !== null) this.#log(`[mutable] new DHT socket has ${found} nodes`);
+  }
+
   /**
    * Stops republishing.
    * @returns {void}
@@ -302,5 +414,13 @@ export class MutablePublisher {
     if (this.#retryTimer) clearTimeout(this.#retryTimer);
     this.#timer = null;
     this.#retryTimer = null;
+    // Only what this publisher opened: a caller that supplied a socket owns it.
+    if (this.#ownsDht) {
+      try {
+        this.#dht?.destroy?.();
+      } catch {
+        // Shutting down; nothing useful to do about a socket that will not close.
+      }
+    }
   }
 }

package/src/web/index.html

@@ -2894,9 +2894,17 @@
         };
         const [r, g, b] = rgb(accent);
 
-        // Availability is a count, so it shades: the rarer the piece, the
-        // fainter the column. The other two are yes or no.
-        const ceiling = kind === 'availability' ? Math.max(1, ...bytesIn) : 1;
+        // Availability is a count, so it shades by rarity. The other two are
+        // proportions now: a column covers many pieces — 178,000 of them
+        // across a bar a thousand wide on a 698 GiB archive — so "held" is a
+        // fraction of its slice rather than a yes or no. Reduced any other way
+        // the bar only tells the truth at 0% and 100%.
+        //
+        // A sidecar older than pmtiles-torrent 0.4.4 still sends 0 or 1, and
+        // is recognised by nothing exceeding 1: the newer one never rounds a
+        // non-empty bucket below 2, precisely so this test works.
+        const ceiling = kind === 'availability' ? Math.max(1, ...bytesIn) : 255;
+        const proportional = kind !== 'availability' && Math.max(0, ...bytesIn) > 1;
 
         for (let index = 0; index < bytesIn.length; index += 1) {
           const value = bytesIn[index];
@@ -2905,7 +2913,15 @@
             image.data[at + 3] = 0;
             continue;
           }
-          const strength = kind === 'availability' ? 0.35 + 0.65 * (value / ceiling) : 1;
+          // A floor, because a column holding a little is worth seeing: without
+          // one, the first few percent of a download are indistinguishable
+          // from none at all.
+          const strength =
+            kind === 'availability'
+              ? 0.35 + 0.65 * (value / ceiling)
+              : proportional
+                ? Math.max(0.18, value / 255)
+                : 1;
           image.data[at] = r;
           image.data[at + 1] = g;
           image.data[at + 2] = b;