pmtiles-swarm 0.8.0 → 0.9.0

package/CHANGELOG.md

@@ -7,6 +7,45 @@
 ### 🐞 Bug fixes
 - _...Add new stuff here..._
 
+## 0.9.0
+### ✨ Features and improvements
+- **`pmtiles-swarm status`**, which asks a running node what it is doing and reads the answer out
+  loud. It takes the same config file the node runs with, so the address, the port and the
+  credential come from one place rather than being remembered and retyped. That is the whole
+  point of it: the API is on `adminPort` rather than the public port, the node binds where `host`
+  says and that is usually not loopback, and it accepts `authorization: Bearer` and not
+  `x-api-key`. Get any one of those wrong by hand and the answer is a refused connection or a 401,
+  both of which read as a broken node rather than as a mistyped command — which is exactly how
+  they were read while diagnosing the archive fixed below.
+
+  It names the case that is otherwise silent: an archive the catalog holds and the engine does
+  not, which through `curl` is a row of empty columns and looks like a corrupt archive. Just after
+  a start it is normal and passes; persisting, the engine refused it and the log says why. Exits
+  non-zero when the node does not answer or its engine is down, so it can be the last step of a
+  deployment script, and `--json` hands back the raw replies for anything that would rather parse.
+
+  Also warns when `--config` names a file that is not there. Startup ignores that on purpose, so
+  a first run can write one — but for a question about a running node the silence is
+  misleading, since the answer then describes the default address and looks entirely real.
+
+- **[docs/haproxy.md](docs/haproxy.md) now covers the backend pool**: why round robin rather than
+  the plugin's default of Source-IP Hash, which fails quietly behind a CDN by pinning nearly all
+  traffic to one node while the rest sit idle and healthy; when least-connections or URI hash are
+  worth having instead; and what HTTP/2 on the frontend does and does not change about balancing.
+
+### 🐞 Bug fixes
+- **An archive built from a watched folder no longer sits at 0%, seeding nobody, for a quarter of
+  an hour.** The libtorrent engine dropped `seedOnly` on its way to the sidecar, so libtorrent
+  re-hashed an 81 GiB archive that had been read end to end moments earlier to produce its
+  torrent. Everything else already handled it — the library sets it in five places, the
+  composite engine checks it against what the primary reports, qBittorrent has its own flag for
+  it — and this one engine silently did not pass it on. Needs pmtiles-torrent 0.4.1, which the
+  existing dependency range picks up on a fresh install.
+- **`docs/running-as-a-service.md` no longer suggests checking a node with `curl localhost:8091`.**
+  It names loopback and sends no credential, so on a node bound to its LAN address with a key
+  configured it fails twice over, in the two ways that look most like a broken node. It now uses
+  the status command.
+
 ## 0.8.0
 ### ✨ Features and improvements
 - **`GET /health`**, for a load balancer: 200 when this node can serve, 503 when its engine

package/README.md

@@ -589,6 +589,38 @@ network equipment: peers request 16 KiB blocks whatever the piece size. The sett
 matters there is `maxConnections`, since every peer holds a NAT table entry. See
 [docs/publishing.md](docs/publishing.md).
 
+## Asking a running node what it is doing
+
+```sh
+node src/index.js status --config /etc/pmtiles-swarm/swarm.config.json
+```
+
+```
+engine  libtorrent  ready
+version 0.8.0
+17 archives, 1 the engine does not know about
+
+NAME                                             SIZE  STATE       PROGRESS
+planetiler-openmaptiles-260803.pmtiles         81 GiB  seeding         100%
+planet-260803.osm.pbf                          94 GiB  downloading      37%
+planetiler-openmaptiles-260810.pmtiles         83 GiB  —                 —
+```
+
+It reads the same config file the node runs with, so the address, the port and the credential come
+from one place rather than being remembered and retyped. That matters more than it sounds: the API
+is on `adminPort`, not the public port; the node binds where `host` says, which is usually not
+loopback; and it accepts `authorization: Bearer`, not `x-api-key`. Get any one of those wrong with
+`curl` and the answer is a refused connection or a 401 — both of which read as a broken node rather
+than a mistyped command.
+
+An archive with a state of `—` is one the catalog holds and the engine is not. Directly after a
+restart that is normal and passes. Persisting, it means the engine refused it, and the log says
+why.
+
+Exit status is 0 when the node answered and its engine is up, 1 when it did not or is not — so it
+works in a deployment script. `--json` gives the raw `/api/status` and `/api/torrents` replies for
+anything that wants to parse rather than read.
+
 ## API
 
 | Method | Path | Purpose |

package/docs/haproxy.md

@@ -81,6 +81,47 @@ more often.
 Note also that the node comes *back* on `rise` successful checks — 2 by
 default — so a flapping node re-enters rotation quickly whichever you choose.
 
+## The backend pool
+
+**Settings → Backend Pools → Add**, mode **HTTP (Layer 7)**.
+
+### Balancing algorithm
+
+**Round Robin.** Tile requests are stateless, numerous and roughly the same
+size, which is the case round robin is for.
+
+**Not Source-IP Hash, even though it is the default here.** It exists for
+sticky sessions, and tiles have no session to be sticky about. Worse, it fails
+quietly in exactly the setup this document assumes: behind a CDN every request
+arrives from a handful of edge addresses, so hashing on the source pins almost
+all traffic to one node while the others sit idle and healthy. A NAT'd office
+or a mobile carrier does the same thing on a smaller scale.
+
+Two others are worth knowing about, for arrangements this is not:
+
+**Least Connections**, when long transfers share the backend with short ones.
+A web-seed range request can run for minutes while a tile takes milliseconds,
+and round robin will happily queue tiles behind a transfer. If web seeds are
+served from a different host — an ordinary web server in front of the published
+directory, which is the usual arrangement — that variance is not here and round
+robin is simpler.
+
+**URI Hash**, for a tier of **cache-mode** nodes. Such a node holds only the
+pieces it has read, so sending the same region to different nodes makes each of
+them pay the cold read separately — the one real cost of scaling a cache-mode
+tier horizontally. `balance uri depth 2` hashes on `/archives/<infohash>` and
+gives archive affinity, at the price of concentrating one archive on one node.
+For nodes holding complete copies there is no cold read to avoid, so this is a
+cost with no benefit.
+
+### HTTP/2
+
+Enable it on the frontend and leave *HTTP/2 without TLS* unchecked: the client
+gets HTTP/2, the node is spoken to over HTTP/1.1, which is what it speaks.
+Balancing in HTTP mode is per request rather than per connection, so a client
+multiplexing a hundred tile requests over one HTTP/2 connection still has them
+spread across the pool.
+
 ## Real servers
 
 **Settings → Real Servers → Add**, one per node, port **8090**.

package/docs/running-as-a-service.md

@@ -441,9 +441,23 @@ Then check it is actually serving:
 
 ```sh
 curl -fsS localhost:8090/feed.xml >/dev/null && echo "public surface ok"
-curl -fsS localhost:8091/api/status | head -c 200
+node /opt/pmtiles-swarm/src/index.js status \
+  --config /etc/pmtiles-swarm/swarm.config.json
 ```
 
+Ask through the status command rather than with `curl`. Reaching the API by hand
+means getting the bind address, the admin port and the credential right in one
+go, and each of them fails in a way that looks like a broken node: a node bound
+to its LAN address refuses a request to `localhost`, and the header it accepts
+is `authorization: Bearer`, so anything else is a 401. The status command reads
+all three out of the config file the service is running with. It exits non-zero
+when the node does not answer or its engine is down, so it also works as the
+last step of a deployment script.
+
+An archive listed with a state of `—` is one the catalog holds and the engine is
+not. Just after a start that is normal and passes within a minute or so. If it
+persists, the engine refused it, and the journal says why.
+
 And that it can write where it is supposed to, which nothing above proves:
 
 ```sh

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "pmtiles-swarm",
-  "version": "0.8.0",
+  "version": "0.9.0",
   "description": "BitTorrent distribution for PMTiles map archives: create torrents, watch folders, publish and subscribe to RSS feeds, and seed through qBittorrent or an embedded client",
   "type": "module",
   "main": "src/index.js",

package/src/engines/libtorrent.js

@@ -254,6 +254,12 @@ export class LibtorrentEngine {
       savePath: request.savePath ?? this.#options.savePath,
       mode: request.mode,
       paused: request.paused,
+      // The caller's claim that the data is already on disk, which for an
+      // archive created here it is: the file was read end to end a moment ago
+      // to produce the torrent. Without passing it on, libtorrent hashes the
+      // whole archive again before seeding a byte — a quarter of an hour for
+      // an 81 GiB build, during which it reads as 0% and serves nobody.
+      seedOnly: request.seedOnly,
     });
     return result.infoHash;
   }

package/src/index.js

@@ -92,20 +92,26 @@ function createOneEngine(name, config) {
  * @returns {Promise<void>} - Resolves once listening.
  */
 async function main() {
-  const { values } = parseArgs({
+  const { values, positionals } = parseArgs({
     options: {
       config: { type: 'string', short: 'c' },
       port: { type: 'string', short: 'p' },
       help: { type: 'boolean', short: 'h' },
+      json: { type: 'boolean' },
     },
-    allowPositionals: false,
+    allowPositionals: true,
   });
 
   if (values.help) {
     console.log(`pmtiles-swarm — BitTorrent distribution for PMTiles archives
 
+Usage:
+  pmtiles-swarm [--config FILE]          start the node
+  pmtiles-swarm status [--config FILE]   ask a running node what it is doing
+
   --config, -c   path to a JSON config file
   --port,   -p   override the listen port
+  --json         machine-readable output, for the status command
   --help,   -h   this message
 
 Environment: PMTILES_SWARM_PORT, PMTILES_SWARM_DATA_DIR, PMTILES_SWARM_ENGINE,
@@ -118,6 +124,23 @@ PMTILES_SWARM_PUBLIC_URL
   const config = await loadConfig(values.config);
   if (values.port) config.port = Number(values.port);
 
+  // Asking rather than starting. Everything it needs — which address the admin
+  // listener is on, which port, and the credential — comes from the same
+  // configuration the node runs with, so there is nothing to pass and nothing
+  // to get wrong.
+  if (positionals[0] === 'status') {
+    const { runStatus } = await import('./status-command.js');
+    process.exitCode = await runStatus(config, { json: values.json });
+    return;
+  }
+
+  if (positionals.length > 0) {
+    console.error(`unknown command: ${positionals[0]}`);
+    console.error('try: pmtiles-swarm status');
+    process.exitCode = 2;
+    return;
+  }
+
   // Everything that has to be stopped, in the order it should be stopped,
   // filled in as startup proceeds.
   //

package/src/status-command.js

@@ -0,0 +1,233 @@
+/**
+ * `pmtiles-swarm status` — asking a running node what it is doing.
+ *
+ * This exists because interrogating a node meant getting four separate things
+ * right at once: which address the admin listener is bound to, which port,
+ * which header carries the credential, and where in the JSON the answer lives.
+ * Getting any one wrong produces something that looks like a broken archive —
+ * a refused connection, a 401, or a row of nulls — rather than like a mistyped
+ * command. Every one of those is derivable from the configuration file the
+ * node is already running with, so nothing here needs to be passed or
+ * remembered.
+ */
+
+import { access } from 'node:fs/promises';
+
+/**
+ * Whether a path can be read.
+ * @param {string} path - The path.
+ * @returns {Promise<boolean>} - True when it is there.
+ */
+async function readable(path) {
+  try {
+    await access(path);
+    return true;
+  } catch {
+    return false;
+  }
+}
+
+/** Columns, and how wide the name column may grow before it is cut. */
+const NAME_WIDTH = 44;
+
+/**
+ * A size in bytes, as a person would write it.
+ * @param {number} value - Bytes.
+ * @returns {string} - e.g. "81 GiB".
+ */
+export function bytes(value) {
+  if (!Number.isFinite(value) || value <= 0) return '—';
+  const units = ['B', 'KiB', 'MiB', 'GiB', 'TiB'];
+  let size = value;
+  let unit = 0;
+  while (size >= 1024 && unit < units.length - 1) {
+    size /= 1024;
+    unit += 1;
+  }
+  return `${size < 10 && unit > 0 ? size.toFixed(1) : Math.round(size)} ${units[unit]}`;
+}
+
+/**
+ * Where this node's API is, according to its own configuration.
+ *
+ * The admin listener where there is one, since that is where the API lives on
+ * a node that separates them. A wildcard bind is reported as loopback: `::` is
+ * what the node listens on, not an address anything can connect to.
+ * @param {object} config - Resolved configuration.
+ * @returns {string} - An origin, e.g. "http://172.16.1.49:8091".
+ */
+export function adminUrl(config) {
+  const host = config.adminHost ?? config.host ?? '127.0.0.1';
+  const port = config.adminPort ?? config.port ?? 8090;
+  const reachable =
+    host === '0.0.0.0' || host === '::' || host === '' ? '127.0.0.1' : host;
+  // A bare IPv6 address needs brackets before it is a URL.
+  const bracketed =
+    reachable.includes(':') && !reachable.startsWith('[')
+      ? `[${reachable}]`
+      : reachable;
+  return `http://${bracketed}:${port}`;
+}
+
+/**
+ * The header a request to this node's API needs, if any.
+ *
+ * `authorization: Bearer`, which is the only form the node accepts — not
+ * `x-api-key`, whatever the convention elsewhere.
+ * @param {object} config - Resolved configuration.
+ * @returns {object} - Headers to send.
+ */
+export function authHeaders(config) {
+  const key = config.auth?.apiKey;
+  return key ? { authorization: `Bearer ${key}` } : {};
+}
+
+/**
+ * One line per archive, plus what the engine says about the node.
+ * @param {object} answer - `{ status, torrents }` as the API returned them.
+ * @returns {string} - The report.
+ */
+export function formatStatus({ status, torrents }) {
+  const lines = [];
+  const engine = status?.engine;
+  lines.push(
+    `engine  ${engine?.name ?? 'unknown'}` +
+      (engine?.ok === false ? `  UNAVAILABLE — ${engine.error ?? ''}` : '  ready'),
+  );
+  if (status?.version) lines.push(`version ${status.version}`);
+
+  const rows = torrents ?? [];
+  // An archive the engine has never heard of is the case worth naming. It is
+  // in the catalog, it has a size, and every live column is empty — which
+  // reads as a broken archive and is usually a node that has not finished
+  // starting, or one that could not add it.
+  const unknown = rows.filter((row) => !row.status).length;
+  lines.push(
+    `${rows.length} archive${rows.length === 1 ? '' : 's'}` +
+      (unknown > 0 ? `, ${unknown} the engine does not know about` : ''),
+  );
+  lines.push('');
+
+  if (rows.length === 0) return `${lines.join('\n')}\n`;
+
+  const head =
+    'NAME'.padEnd(NAME_WIDTH) +
+    'SIZE'.padStart(9) +
+    '  ' +
+    'STATE'.padEnd(12) +
+    'PROGRESS'.padStart(8);
+  lines.push(head);
+
+  for (const row of rows) {
+    const name =
+      row.name.length > NAME_WIDTH - 1
+        ? `${row.name.slice(0, NAME_WIDTH - 2)}…`
+        : row.name;
+    const state = row.status?.state ?? (row.paused ? 'paused' : '—');
+    const progress =
+      typeof row.status?.progress === 'number'
+        ? `${Math.round(row.status.progress * 100)}%`
+        : '—';
+    lines.push(
+      name.padEnd(NAME_WIDTH) +
+        bytes(row.size).padStart(9) +
+        '  ' +
+        String(state).padEnd(12) +
+        progress.padStart(8),
+    );
+  }
+
+  if (unknown > 0) {
+    lines.push('');
+    lines.push(
+      'An archive with no state is one the engine is not holding. If the node',
+    );
+    lines.push(
+      'has just started it may still be handing them back; if it persists, the',
+    );
+    lines.push('log will say why it could not be added.');
+  }
+
+  return `${lines.join('\n')}\n`;
+}
+
+/**
+ * Asks a running node for its status and prints it.
+ * @param {object} config - Resolved configuration.
+ * @param {object} [options] - Injectable fetch and output, for testing.
+ * @returns {Promise<number>} - The exit code.
+ */
+export async function runStatus(config, options = {}) {
+  const {
+    fetch: get = globalThis.fetch,
+    out = (text) => process.stdout.write(text),
+    err = (text) => process.stderr.write(text),
+    json = false,
+  } = options;
+
+  // A named config file that is not there is silently ignored on startup, so
+  // that a first run can write one. Here that silence is misleading: a typo in
+  // --config means this reports on the default address with no key, which is a
+  // different node from the one that was asked about, and the answer looks
+  // real. Say it, rather than letting it be discovered later.
+  if (config.configPath && !(await readable(config.configPath))) {
+    err(
+      `no config file at ${config.configPath} — using defaults, ` +
+        'which is probably not the node you meant.\n',
+    );
+  }
+
+  const base = adminUrl(config);
+  const headers = authHeaders(config);
+
+  let status;
+  let torrents;
+  try {
+    const [statusReply, torrentsReply] = await Promise.all([
+      get(`${base}/api/status`, { headers }),
+      get(`${base}/api/torrents`, { headers }),
+    ]);
+
+    // Said plainly, because a 401 here means the key in this configuration is
+    // not the key the node is running with — which is a different problem from
+    // the node being down, and looks identical without being told.
+    if (statusReply.status === 401 || statusReply.status === 403) {
+      err(
+        `${base} refused the credential in this configuration file.\n` +
+          'The node is running, but with a different auth.apiKey.\n',
+      );
+      return 1;
+    }
+    if (!statusReply.ok) {
+      err(`${base}/api/status answered ${statusReply.status}\n`);
+      return 1;
+    }
+
+    status = await statusReply.json();
+    torrents = torrentsReply.ok ? await torrentsReply.json() : [];
+  } catch (error) {
+    // A refused connection is the commonest failure and the least obvious: the
+    // node binds where the configuration says, which is often not loopback.
+    // Node's fetch reports every one of them as "fetch failed" and puts the
+    // part worth reading — refused, timed out, no such host — in `cause`.
+    const reason = error.cause?.code
+      ? `${error.message} (${error.cause.code})`
+      : error.message;
+    err(
+      `could not reach ${base}: ${reason}\n` +
+        'That address comes from adminHost and adminPort in this configuration ' +
+        'file.\nIs the node running, and bound where this says?\n',
+    );
+    return 1;
+  }
+
+  if (json) {
+    out(`${JSON.stringify({ status, torrents }, null, 2)}\n`);
+  } else {
+    out(formatStatus({ status, torrents }));
+  }
+
+  // Usable from a script: the engine being unreachable is the thing worth
+  // failing on, and it is what /health reports to a load balancer.
+  return status?.engine?.ok === false ? 1 : 0;
+}