pmtiles-swarm 0.49.0 → 0.50.0

package/CHANGELOG.md

@@ -7,6 +7,38 @@
 ### 🐞 Bug fixes
 - _...Add new stuff here..._
 
+## 0.50.0
+### ✨ Features and improvements
+- **Three switches for what a node offers of an archive's own bytes**, on the node, on a watched
+  folder, on a scheduled source, and on any individual archive from **HTTP sources** in its details.
+  They are separate because they are three different exposures, and a node can reasonably want any
+  one of them without the others:
+
+  - `serveArchive` — whether `/archives/<infohash>/archive.pmtiles` answers at all. This is the one
+    that decides whether a stranger who knows an infohash can pull 700 GiB off the box.
+  - `selfWebSeed` — whether this node's own URL goes into the torrent's `url-list`, so every peer
+    holding the torrent fetches from here. Turning it on writes the URL into the `.torrent` and the
+    magnet; turning it off takes that URL back out.
+  - `publicDownload` — whether the public catalogue page offers it as a download. Serving a file to a
+    reader that was handed the URL and advertising it to every visitor are different decisions.
+
+  The last two are read as off wherever the first is, whatever the record says. A web seed URL that
+  answers `403` is worse than no web seed, because a client spends its retries on it, and a download
+  link that `403`s is worse than no link. An archive that says nothing about a setting goes on
+  following the node, so changing the node's answer reaches everything that never had one of its own.
+
+- **`DELETE /api/torrents/<infohash>/webseeds`** drops web seeds, the same rewrite as adding them and
+  safe for the same reason: `url-list` sits outside the info dictionary, so the infohash — and every
+  magnet and peer depending on it — is untouched.
+
+### 🐞 Bug fixes
+- **Serving whole archives over HTTP is no longer on for everyone.** 0.48.0 added
+  `/archives/<infohash>/archive.pmtiles` and left it answering for every complete archive, on every
+  node, to anyone who knew an infohash. Everything else this node publishes is either small — TileJSON,
+  a `.torrent`, a feed — or metered by the request, one tile at a time, so turning a node on had never
+  meant offering its disk to strangers. It does not now either: `serveArchive` defaults to off and
+  both range endpoints answer `403` until it is set.
+
 ## 0.49.0
 ### ✨ Features and improvements
 - **The current build of a category can be read as a file.** `GET /latest/<category>/archive.pmtiles`

package/README.md

@@ -756,6 +756,8 @@ which the endpoint answers 501.
 | `POST`                | `/api/torrents/:infoHash/recheck`                           | Hash what is on disk again and believe the result — for an archive whose progress and whose files disagree                                                                                                                                |
 | `POST`                | `/api/torrents/:infoHash/pause`, `/resume`                  | Stop offering it, without forgetting it                                                                                                                                                                                                   |
 | `POST`                | `/api/torrents/:infoHash/webseeds`                          | Add web seeds — does not change the infohash                                                                                                                                                                                              |
+| `DELETE`              | `/api/torrents/:infoHash/webseeds`                          | Drop web seeds — likewise                                                                                                                                                                                                                 |
+| `POST`                | `/api/torrents/:infoHash/publish`                           | What this node offers of the archive itself: `serveArchive`, `selfWebSeed`, `publicDownload`                                                                                                                                              |
 | `POST`                | `/api/torrents/:infoHash/warm`                              | Pre-fetch a region (`GET` for progress, `DELETE` to cancel)                                                                                                                                                                               |
 | `DELETE`              | `/api/torrents/:infoHash/cache`                             | Reclaim cached pieces, keep the archive                                                                                                                                                                                                   |
 | `POST`                | `/api/torrents/:infoHash/check`                             | Has the source changed since the torrent was made?                                                                                                                                                                                        |
@@ -783,11 +785,11 @@ which the endpoint answers 501.
 | `GET`                 | `/health`                                                   | 200 when this node can serve, 503 when its engine cannot — **public**, no credential, for a load balancer                                                                                                                                 |
 | `GET`                 | `/archives/:infoHash/ready`                                 | Whether this node can serve _this_ archive yet: 200 ready, 503 not yet, 415 never — **public**                                                                                                                                            |
 | `GET`                 | `/archives/:infoHash/archive.torrent`                       | The `.torrent` a torrent-aware client joins with — **public**                                                                                                                                                                             |
-| `GET`                 | `/archives/:infoHash/archive.pmtiles`                       | The archive itself, by byte range — what any PMTiles reader wants, and a valid web seed. Complete archives only — **public**                                                                                                              |
+| `GET`                 | `/archives/:infoHash/archive.pmtiles`                       | The archive itself, by byte range — what any PMTiles reader wants, and a valid web seed. Complete archives only, and only where [`serveArchive`](docs/configuration.md#servearchive) is on — **public**                                   |
 | `GET`                 | `/archives/:infoHash/preview`                               | Map preview for one archive — admin, not public                                                                                                                                                                                           |
 | `GET`                 | `/latest/`                                                  | Every category and the endpoints that resolve to its newest build — **public**, the index of everything below                                                                                                                             |
 | `GET`                 | `/latest/:category/tiles.json`, `/:name.torrent`, `/magnet` | The newest build in a category — **public**. The torrent name is yours to choose, so a link can read `planetiler-openmaptiles-latest.torrent`; it redirects to the immutable URL, which names the download after the build it actually is |
-| `GET`                 | `/latest/:category/archive.pmtiles`                         | The newest build in a category as a file, by byte range — **public**. What to point a PMTiles reader at when you want "whichever is current" rather than one build                                                                        |
+| `GET`                 | `/latest/:category/archive.pmtiles`                         | The newest build in a category as a file, by byte range — **public**, and gated by [`serveArchive`](docs/configuration.md#servearchive). What to point a PMTiles reader at when you want "whichever is current" rather than one build     |
 | `GET`                 | `/feed.xml`, `/feed/:category.xml`, `/latest/:category.xml` | RSS — **public**                                                                                                                                                                                                                          |
 
 Everything under `/api/` is guarded once a credential is configured; tiles, TileJSON and the feeds

package/docs/configuration.md

@@ -13,6 +13,7 @@ Some settings only take effect on restart. Those are marked **restart**.
 - [Storage](#storage)
 - [Engines](#engines)
 - [Creating torrents](#creating-torrents)
+- [Offering the archive file itself](#offering-the-archive-file-itself)
 - [Trackers](#trackers)
 - [Feeds](#feeds)
 - [Watched folders](#watched-folders)
@@ -312,6 +313,77 @@ idled. Use **Pause all** for the other half.
 It lives in the configuration rather than in memory, so a node taken out of
 rotation stays out across the restart you were probably about to do.
 
+## Offering the archive file itself
+
+| setting          | default |                                                       |
+| ---------------- | ------- | ----------------------------------------------------- |
+| `serveArchive`   | `false` | answer `/archives/<infohash>/archive.pmtiles`         |
+| `selfWebSeed`    | `false` | publish this node as a web seed for archives it holds |
+| `publicDownload` | `false` | offer them as downloads on the public catalogue page  |
+
+Three switches rather than one, because they are three different exposures and a
+node can reasonably want any of them without the others.
+
+### `serveArchive`
+
+Off by default. This is the archive as a file, by byte range — which is what
+every PMTiles reader actually wants, and what makes this node usable as an origin
+without a copy of the file somewhere else.
+
+It is off by default because it is the only thing a node publishes that is not
+either small or metered by the request. TileJSON, a `.torrent`, a feed: kilobytes.
+A tile: one tile. This is up to 700 GiB to anyone who knows an infohash, so
+turning a node on has never meant offering its disk to strangers, and still does
+not.
+
+With it on, both range endpoints answer:
+
+```
+GET /archives/<infohash>/archive.pmtiles   one build, immutable, cached for a year
+GET /latest/<category>/archive.pmtiles     whichever is current, with an ETag
+```
+
+With it off, both answer `403`.
+
+### `selfWebSeed`
+
+Writes this node's own `archive.pmtiles` URL into the torrent's `url-list`, so
+every peer holding the torrent fetches from here over HTTP. A web seed is the
+difference between a cold tile taking tens of seconds and taking under one, and
+it is what makes a brand-new archive usable before it has any peers at all.
+
+It is also an open invitation, which is why it is separate: a seed URL is
+followed by everyone who holds the torrent, not only by people who came to this
+node. Turning it off takes the URL back out of the `.torrent` and the magnet —
+but peers already holding either keep trying it until they refresh, so this
+withdraws an advertisement rather than closing a door.
+
+The node has to know what it is called. Set [`publicUrl`](#publicurl), or turn
+the switch on from the console, where the request itself names the node. Without
+either, the setting is refused rather than guessed at: a guessed web seed URL is
+published and then followed.
+
+### `publicDownload`
+
+Adds a **download** link to the public catalogue page. Separate from
+`serveArchive` because serving a file to a reader that was handed the URL and
+advertising it to every visitor are different decisions — the endpoint can exist
+for a style or a peer without being put in front of a browser.
+
+Both `selfWebSeed` and `publicDownload` are read as off wherever `serveArchive`
+is off, whatever the file says. A web seed URL that answers `403` is worse than no
+web seed, because a client spends its retries on it, and a download link that
+`403`s is worse than no link.
+
+### Per archive, per folder, per source
+
+The same three-level rule as [`md5`](#md5). The node's setting is the default; a
+[watched folder](#watched-folders) or a [scheduled source](#scheduled-sources)
+may carry its own; and any individual archive can be switched in the console,
+under **HTTP sources** in its details. An archive that says nothing goes on
+following the node, so changing the node's answer reaches every archive that
+never had one of its own.
+
 ## Trackers
 
 `trackers` is baked into every torrent this node creates. It defaults to the
@@ -372,7 +444,8 @@ their own name. Archives with no category are excluded whenever this is set.
 ## Watched folders
 
 `watch` is a list of `{ path, category, match, webSeedBase, publishDir, sparse,
-latestLink, latestLinkType, keep, keepDays, md5 }`.
+latestLink, latestLinkType, keep, keepDays, md5, serveArchive, selfWebSeed,
+publicDownload }`.
 
 | field               |                                                                                       |
 | ------------------- | ------------------------------------------------------------------------------------- |
@@ -383,6 +456,9 @@ latestLink, latestLinkType, keep, keepDays, md5 }`.
 | `latestLinkType`    | `'symbolic'` (default) or `'hard'`                                                    |
 | `keep` / `keepDays` | retire what the folder has outgrown                                                   |
 | `md5`               | overrides the node's [`md5`](#md5) for this folder alone                              |
+| `serveArchive`      | overrides [`serveArchive`](#servearchive) for archives from this folder               |
+| `selfWebSeed`       | overrides [`selfWebSeed`](#selfwebseed) for them                                      |
+| `publicDownload`    | overrides [`publicDownload`](#publicdownload) for them                                |
 
 `publishDir` and `webSeedBase` together give every imported archive a working web
 seed, which is what makes a brand-new archive usable before any peer has a copy of
@@ -420,14 +496,15 @@ remove the newest build however old it gets. See
 `sources` is a list of upstreams that publish a new archive on a schedule. Each
 entry gives either a `url` template or an `index` directory:
 
-| field        |                                                                                                               |
-| ------------ | ------------------------------------------------------------------------------------------------------------- |
-| `url`        | a template with the date in it — `{YYYYMMDD}`, `{YYYY-MM-DD}`, `{YYYY}`, `{MM}`, `{DD}` — expanded and probed |
-| `index`      | a directory URL, listed and filtered                                                                          |
-| `newest`     | how many listed files an index source will consider. Defaults to 1                                            |
-| `at`         | a time of day in UTC, or a list of them — `"03:30"`                                                           |
-| `everyHours` | an interval instead                                                                                           |
-| `md5`        | overrides the node's [`md5`](#md5) for this source alone                                                      |
+| field                                           |                                                                                                               |
+| ----------------------------------------------- | ------------------------------------------------------------------------------------------------------------- |
+| `url`                                           | a template with the date in it — `{YYYYMMDD}`, `{YYYY-MM-DD}`, `{YYYY}`, `{MM}`, `{DD}` — expanded and probed |
+| `index`                                         | a directory URL, listed and filtered                                                                          |
+| `newest`                                        | how many listed files an index source will consider. Defaults to 1                                            |
+| `at`                                            | a time of day in UTC, or a list of them — `"03:30"`                                                           |
+| `everyHours`                                    | an interval instead                                                                                           |
+| `md5`                                           | overrides the node's [`md5`](#md5) for this source alone                                                      |
+| `serveArchive`, `selfWebSeed`, `publicDownload` | override what this node offers of the archives this source fetches                                            |
 
 Prefer a template where the naming is predictable: it asks a direct question,
 gets a direct answer, and needs the upstream to publish no listing at all.

package/docs/serving-tiles.md

@@ -121,6 +121,12 @@ GET /latest/{category}.xml               a feed holding only the current build
 Point a style at `/latest/basemaps/tiles.json` and it keeps working across every
 rebuild, with no edit.
 
+`archive.pmtiles` is the one of these that has to be asked for. It is off until
+[`serveArchive`](configuration.md#servearchive) is set, on the node or on the
+archive, because it is the only thing here that is neither small nor metered by
+the request — everything else on this page is kilobytes or one tile, and this is
+the whole file.
+
 **The tiles it names are still infohash URLs.** That is the whole point of the
 layering: this document is the only mutable thing in the system, and everything
 it refers to stays content-addressed and cached for a year. Pointing the tile

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "pmtiles-swarm",
-  "version": "0.49.0",
+  "version": "0.50.0",
   "description": "BitTorrent distribution for PMTiles map archives: create torrents, watch folders, publish and subscribe to RSS feeds, and seed through qBittorrent or an embedded client",
   "type": "module",
   "main": "src/index.js",

package/src/api.js

@@ -13,7 +13,7 @@ import {
   hashToken,
   isPublicSurface,
 } from './auth.js';
-import { normalizeCategories } from './catalog.js';
+import { normalizeCategories, publishingFor } from './catalog.js';
 import { mutableMagnet, trackersFromMagnet } from './mutable.js';
 import { guessKind } from './library.js';
 import { QBittorrentEngine } from './engines/qbittorrent.js';
@@ -1005,7 +1005,17 @@ export function createApp({
       // being read through the swarm while serving thousands of tiles is a
       // different situation from one doing neither.
       const served = stats?.forArchive(entry.infoHash) ?? null;
-      res.json({ ...entry, status, reading, diskBytes, served });
+      // Resolved rather than raw, because an entry that says nothing is not
+      // "off" — it defers to the node, and the console has to show what is
+      // actually happening rather than what this record happens to store.
+      res.json({
+        ...entry,
+        status,
+        reading,
+        diskBytes,
+        served,
+        publishing: publishingFor(entry, config),
+      });
     }),
   );
 
@@ -1097,6 +1107,54 @@ export function createApp({
     }),
   );
 
+  // Removes web seeds from an archive already in circulation. The same
+  // rewrite as adding one, and safe for the same reason: url-list sits outside
+  // the info dictionary, so the infohash is untouched.
+  app.delete(
+    '/api/torrents/:infoHash/webseeds',
+    route(async (req, res) => {
+      const body = req.body ?? {};
+      const urls = body.webSeeds ?? body.urls ?? body.url;
+      try {
+        const result = await library.removeWebSeeds(
+          req.params.infoHash,
+          Array.isArray(urls) ? urls : [urls],
+        );
+        res.json(result);
+      } catch (error) {
+        const status = /unknown archive/.test(error.message) ? 404 : 400;
+        res.status(status).json({ error: error.message });
+      }
+    }),
+  );
+
+  // What this node offers of one archive's own bytes over HTTP: whether it
+  // serves the file at all, whether it publishes itself as a web seed for it,
+  // and whether the public page offers it as a download. Three switches
+  // because they are three exposures — see publishingFor in catalog.js.
+  app.post(
+    '/api/torrents/:infoHash/publish',
+    route(async (req, res) => {
+      const body = req.body ?? {};
+      try {
+        const result = await library.setPublishing(
+          req.params.infoHash,
+          body,
+          // A node with no publicUrl still has to name itself to be a web
+          // seed, and the request it is being asked on is the best evidence
+          // available — the same reasoning the TileJSON already uses. Given
+          // the public port, never the admin one: a seed URL on a listener
+          // bound to localhost is a URL no peer can reach.
+          { baseUrl: baseUrl(req) },
+        );
+        res.json(result);
+      } catch (error) {
+        const status = /unknown archive/.test(error.message) ? 404 : 400;
+        res.status(status).json({ error: error.message });
+      }
+    }),
+  );
+
   // Stops a torrent without forgetting it. "Not right now" is a different
   // intention from "not any more", and remove was the only way to express
   // either.
@@ -1601,6 +1659,9 @@ export function createApp({
         webSeedBase: body.webSeedBase,
         allowUnknown: body.allowUnknown,
         md5: body.md5,
+        serveArchive: body.serveArchive,
+        selfWebSeed: body.selfWebSeed,
+        publicDownload: body.publicDownload,
         webSeed: body.webSeed,
       };
 
@@ -1951,6 +2012,14 @@ export function createApp({
           magnet: entry.magnet,
           torrent: `${baseUrl(req)}/archives/${entry.infoHash}/archive.torrent`,
           webSeeds: entry.webSeeds ?? [],
+          // Present only when this archive is offered as a download. Absent
+          // otherwise rather than null: a public document should say what is
+          // on offer, not enumerate what is being withheld.
+          ...(publishingFor(entry, config).publicDownload
+            ? {
+                archive: `${baseUrl(req)}/archives/${entry.infoHash}/archive.pmtiles`,
+              }
+            : {}),
           pmtiles: entry.pmtiles,
           kind: entry.kind,
           md5: entry.md5,
@@ -2262,6 +2331,18 @@ export function createApp({
     const entry = newestIn(req.params.category, req);
     if (!entry) return res.status(404).json({ error: 'no such category' });
 
+    // Whether this node hands out whole archives at all is the operator's
+    // call, not a consequence of having the file. Everything else published
+    // here is either small or metered by the request; this is up to 700 GiB to
+    // anyone who knows an infohash, so it is asked for rather than assumed.
+    if (!publishingFor(entry, config).serveArchive) {
+      return res.status(403).json({
+        error:
+          'this node does not serve archive files over HTTP. Set ' +
+          '`serveArchive` on the node, or turn it on for this archive.',
+      });
+    }
+
     if (entry.complete === false) {
       return res.status(409).json({
         error:
@@ -2595,6 +2676,18 @@ export function createApp({
     const entry = catalog.get(req.params.infoHash);
     if (!entry) return res.status(404).json({ error: 'not found' });
 
+    // Whether this node hands out whole archives at all is the operator's
+    // call, not a consequence of having the file. Everything else published
+    // here is either small or metered by the request; this is up to 700 GiB to
+    // anyone who knows an infohash, so it is asked for rather than assumed.
+    if (!publishingFor(entry, config).serveArchive) {
+      return res.status(403).json({
+        error:
+          'this node does not serve archive files over HTTP. Set ' +
+          '`serveArchive` on the node, or turn it on for this archive.',
+      });
+    }
+
     if (entry.complete === false) {
       return res.status(409).json({
         error:

package/src/catalog.js

@@ -21,6 +21,12 @@ import path from 'node:path';
  * @property {string} torrentPath - Generated .torrent on disk.
  * @property {string} magnet - Magnet URI for the current infohash.
  * @property {string[]} webSeeds - BEP 19 url-list entries.
+ * @property {boolean} [serveArchive] - Answer /archives/<hash>/archive.pmtiles
+ *   for this archive. Unset defers to the node.
+ * @property {boolean} [selfWebSeed] - Publish this node's own archive URL in
+ *   the torrent's url-list. Unset defers to the node.
+ * @property {boolean} [publicDownload] - Offer it as a download on the public
+ *   catalogue page. Unset defers to the node.
  * @property {object} [pmtiles] - Header and metadata summary.
  * @property {object} [mutable] - BEP 46 identity: {publicKey, salt, seq}.
  * @property {string} [originMtime] - The archive's mtime on the node that built
@@ -51,6 +57,41 @@ export function normalizeCategories(source) {
   return [...new Set(clean)].sort();
 }
 
+/**
+ * What this node offers of an archive's own bytes, over plain HTTP.
+ *
+ * Three separate decisions, because they are three separate exposures and
+ * conflating them takes the choice away from whoever runs the node:
+ *
+ * - `serveArchive` — whether `/archives/<hash>/archive.pmtiles` answers at
+ *   all. This is the one that decides whether a stranger who knows an infohash
+ *   can pull 700 GiB off the box.
+ * - `selfWebSeed` — whether this node's own URL is written into the torrent's
+ *   `url-list`, so every peer in the swarm fetches from it.
+ * - `publicDownload` — whether the public catalogue page offers it as a
+ *   download. Serving a file to a reader that already knows the URL and
+ *   advertising it on a page are not the same act.
+ *
+ * The last two are ANDed with the first rather than merely defaulting from it.
+ * A node cannot be a web seed for a file it will not serve, and a download link
+ * that 403s is worse than no link — so a catalog edited by hand into that state
+ * is read as the safe thing rather than obeyed into an incoherent one.
+ * @param {object} [entry] - A catalog entry, whose fields win where set.
+ * @param {object} [config] - The node's defaults.
+ * @returns {{serveArchive: boolean, selfWebSeed: boolean, publicDownload: boolean}} - Resolved.
+ */
+export function publishingFor(entry, config) {
+  const serveArchive = entry?.serveArchive ?? config?.serveArchive ?? false;
+  return {
+    serveArchive,
+    selfWebSeed:
+      serveArchive && (entry?.selfWebSeed ?? config?.selfWebSeed ?? false),
+    publicDownload:
+      serveArchive &&
+      (entry?.publicDownload ?? config?.publicDownload ?? false),
+  };
+}
+
 /**
  * Orders two entries newest first.
  *

package/src/config.js

@@ -165,6 +165,45 @@ const DEFAULTS = {
    * whether a second full read is worth it depends on what is being read.
    */
   md5: false,
+  /**
+   * Answer `/archives/<infohash>/archive.pmtiles` — the archive itself, by
+   * byte range, which is what every PMTiles reader actually wants.
+   *
+   * Off by default, and deliberately so: this is the whole file, and an
+   * archive here can be 700 GiB. Everything else this node publishes is either
+   * small (TileJSON, a .torrent, a feed) or metered by the request (one tile),
+   * so turning a node on has never meant offering its disk to anyone who knows
+   * an infohash. This would, so it is asked for rather than assumed.
+   *
+   * The node's answer, not the only one: a watched folder, a scheduled source
+   * or an individual archive may carry its own, and is obeyed either way.
+   */
+  serveArchive: false,
+  /**
+   * Write this node's own archive URL into the torrent's `url-list`, so every
+   * peer in the swarm can fetch from it over HTTP.
+   *
+   * A web seed is the difference between a cold tile taking tens of seconds
+   * and taking under one, and it is what makes an archive usable before it has
+   * any peers at all. It is also an open invitation: a seed URL is followed by
+   * everyone who holds the torrent, not only by people who visit this node.
+   *
+   * Means nothing without `serveArchive`, and is read as off without it — a
+   * web seed URL that refuses the request is worse than none, because a client
+   * spends its retries on it.
+   */
+  selfWebSeed: false,
+  /**
+   * Offer the archive as a download on the public catalogue page.
+   *
+   * Separate from `serveArchive` on purpose. Serving a file to a reader that
+   * was given the URL and advertising it on a page are different acts, and a
+   * node can reasonably want the first without the second: the endpoint exists
+   * for a style or a peer, and putting a 700 GiB link in front of every casual
+   * visitor is a different decision. Read as off without `serveArchive`, since
+   * a link that answers 403 is worse than no link.
+   */
+  publicDownload: false,
   /**
    * How long an unfinished download is kept before startup treats it as
    * abandoned. Until then, re-adding the same URL resumes it.
@@ -313,7 +352,8 @@ const DEFAULTS = {
   /**
    * Folders scanned for new archives. Each entry is `{ path, categories,
    * match, webSeedBase, publishDir, latestLink, latestLinkType, keep,
-   * keepDays, sparse, md5 }` — see docs/configuration.md, "Watched folders".
+   * keepDays, sparse, md5, serveArchive, selfWebSeed, publicDownload }` — see
+   * docs/configuration.md, "Watched folders".
    */
   watch: [],
   /**

package/src/library.js

@@ -11,6 +11,7 @@ import {
   promote,
   suffixFor,
 } from './incomplete.js';
+import { publishingFor } from './catalog.js';
 import { checkOrigin, fingerprintOrigin } from './origin.js';
 import { probePMTiles } from './pmtiles-probe.js';
 import {
@@ -333,6 +334,9 @@ export class Library {
         kind: identified.kind,
         sparse: options.sparse,
         md5: created.md5,
+        serveArchive: options.serveArchive,
+        selfWebSeed: options.selfWebSeed,
+        publicDownload: options.publicDownload,
         webSeeds: [...new Set(webSeeds)],
         seedOnly: true,
       });
@@ -1004,6 +1008,9 @@ export class Library {
       kind: identified.kind,
       sparse: options.sparse,
       md5: created.md5,
+      serveArchive: options.serveArchive,
+      selfWebSeed: options.selfWebSeed,
+      publicDownload: options.publicDownload,
       // Whatever was asked for, plus the source when it may be published.
       // Falling back to the source alone dropped a caller's own list — which
       // is precisely the case where the source must not be published and a
@@ -2666,41 +2673,21 @@ export class Library {
   }
 
   /**
-   * Adds web seeds to an archive that already exists.
+   * Rewrites an archive's web seed list, in the .torrent and everywhere else.
    *
    * This is safe on a torrent already in circulation. BEP 19's `url-list` is a
    * top-level key in the metainfo and the infohash covers only the `info`
-   * dictionary, so adding one leaves the infohash untouched — every magnet,
+   * dictionary, so changing one leaves the infohash untouched — every magnet,
    * peer and published reference stays valid. The check below asserts that
    * rather than trusting it: if a rewrite ever did change the infohash, the
    * result would be a different torrent wearing the old one's name, which is
    * worth refusing loudly.
-   *
-   * Retrofitting matters because a web seed is the difference between a cold
-   * tile taking tens of seconds and taking well under one — and an archive
-   * published without one can be given one at any time.
-   * @param {string} infoHash - The archive.
-   * @param {string[]} urls - Web seed URLs to add.
-   * @param {object} [options] - Set `replace` to discard the existing list.
-   * @returns {Promise<{webSeeds: string[], live: boolean}>} - The new list, and
-   *   whether the running torrent took them without a restart.
+   * @param {object} entry - The catalog entry to rewrite.
+   * @param {string[]} wanted - The complete new list.
+   * @returns {Promise<{webSeeds: string[], parsed: object}>} - The new list and
+   *   the parsed torrent, for a caller that needs its infohash.
    */
-  async addWebSeeds(infoHash, urls, options = {}) {
-    const entry = this.#catalog.get(infoHash);
-    if (!entry) throw new Error('unknown archive');
-    if (!entry.torrentPath) {
-      throw new Error('no .torrent is stored for this archive');
-    }
-
-    const wanted = (Array.isArray(urls) ? urls : [urls]).filter(Boolean);
-    if (wanted.length === 0) throw new Error('no web seeds given');
-
-    for (const url of wanted) {
-      if (!/^https?:\/\//i.test(url)) {
-        throw new Error(`web seed must be an http(s) URL: ${url}`);
-      }
-    }
-
+  async #writeWebSeeds(entry, wanted) {
     const parseTorrentModule = await import('parse-torrent');
     const parse = parseTorrentModule.default;
     const { toTorrentFile } = parseTorrentModule;
@@ -2708,9 +2695,7 @@ export class Library {
     const original = await fs.readFile(entry.torrentPath);
     const parsed = await parse(new Uint8Array(original));
 
-    const merged = options.replace
-      ? [...new Set(wanted)]
-      : [...new Set([...(parsed.urlList ?? []), ...wanted])];
+    const merged = [...new Set(wanted)];
     parsed.urlList = merged;
 
     const rebuilt = toTorrentFile(parsed);
@@ -2745,6 +2730,42 @@ export class Library {
       magnet: magnetFor(parsed, this.#config.trackers, merged),
     });
 
+    return { webSeeds: merged, parsed };
+  }
+
+  /**
+   * Adds web seeds to an archive that already exists.
+   *
+   * Retrofitting matters because a web seed is the difference between a cold
+   * tile taking tens of seconds and taking well under one — and an archive
+   * published without one can be given one at any time.
+   * @param {string} infoHash - The archive.
+   * @param {string[]} urls - Web seed URLs to add.
+   * @param {object} [options] - Set `replace` to discard the existing list.
+   * @returns {Promise<{webSeeds: string[], live: boolean}>} - The new list, and
+   *   whether the running torrent took them without a restart.
+   */
+  async addWebSeeds(infoHash, urls, options = {}) {
+    const entry = this.#catalog.get(infoHash);
+    if (!entry) throw new Error('unknown archive');
+    if (!entry.torrentPath) {
+      throw new Error('no .torrent is stored for this archive');
+    }
+
+    const wanted = (Array.isArray(urls) ? urls : [urls]).filter(Boolean);
+    if (wanted.length === 0) throw new Error('no web seeds given');
+
+    for (const url of wanted) {
+      if (!/^https?:\/\//i.test(url)) {
+        throw new Error(`web seed must be an http(s) URL: ${url}`);
+      }
+    }
+
+    const { webSeeds } = await this.#writeWebSeeds(
+      entry,
+      options.replace ? wanted : [...(entry.webSeeds ?? []), ...wanted],
+    );
+
     // Where the engine can take a seed at runtime, the node benefits now;
     // where it cannot, peers still get it from the rewritten .torrent and this
     // node picks it up when the torrent is next added.
@@ -2758,7 +2779,132 @@ export class Library {
       live = results.every(Boolean);
     }
 
-    return { webSeeds: merged, live };
+    return { webSeeds, live };
+  }
+
+  /**
+   * Drops web seeds from an archive, leaving everything else about it alone.
+   *
+   * The running engine is not told. libtorrent has no "forget this url seed"
+   * that every version answers to, and the consequence of it keeping one is
+   * bounded — it retries a URL that refuses and gives up on it. What matters
+   * is that the .torrent and the magnet stop handing the URL to anybody new,
+   * which is what this does.
+   * @param {string} infoHash - The archive.
+   * @param {string[]} urls - Web seed URLs to drop. Unknown ones are ignored.
+   * @returns {Promise<{webSeeds: string[], dropped: string[]}>} - What is left,
+   *   and what went.
+   */
+  async removeWebSeeds(infoHash, urls) {
+    const entry = this.#catalog.get(infoHash);
+    if (!entry) throw new Error('unknown archive');
+    if (!entry.torrentPath) {
+      throw new Error('no .torrent is stored for this archive');
+    }
+
+    const going = new Set(
+      (Array.isArray(urls) ? urls : [urls]).filter(Boolean),
+    );
+    const held = entry.webSeeds ?? [];
+    const dropped = held.filter((url) => going.has(url));
+    if (dropped.length === 0) return { webSeeds: held, dropped: [] };
+
+    const { webSeeds } = await this.#writeWebSeeds(
+      entry,
+      held.filter((url) => !going.has(url)),
+    );
+    return { webSeeds, dropped };
+  }
+
+  /**
+   * Sets what this node offers of one archive's own bytes over HTTP.
+   *
+   * Three separate switches — see `publishingFor` in catalog.js for why they
+   * are separate — and one side effect: turning `selfWebSeed` on writes this
+   * node's own archive URL into the torrent's `url-list`, and turning it off
+   * takes that URL back out. The URL used is remembered, because the base can
+   * change underneath a node and removing "whatever we would build today"
+   * would leave yesterday's URL in the torrent for ever.
+   *
+   * Turning `serveArchive` off takes the other two with it, and that is not a
+   * quiet tidy-up: it means a URL already handed to every peer holding the
+   * torrent stops answering. The caller is told what was withdrawn so it can
+   * say so.
+   * @param {string} infoHash - The archive.
+   * @param {object} changes - Any of the three, as booleans. Anything else is
+   *   ignored, so a caller may send only what it is changing.
+   * @param {object} [options] - `baseUrl` for building the web seed URL.
+   * @returns {Promise<object>} - The resolved settings and what changed.
+   */
+  async setPublishing(infoHash, changes = {}, options = {}) {
+    const entry = this.#catalog.get(infoHash);
+    if (!entry) throw new Error('unknown archive');
+
+    // Only what was actually asked for is recorded. An archive that says
+    // nothing about a setting goes on deferring to the node, which is what
+    // makes changing the node's answer reach the archives that never had one
+    // of their own.
+    const wanted = { ...entry };
+    for (const key of ['serveArchive', 'selfWebSeed', 'publicDownload']) {
+      if (typeof changes[key] === 'boolean') wanted[key] = changes[key];
+    }
+
+    // With serving off the other two cannot be true, and writing that down
+    // matters more than it looks: left as a latent `true`, either would spring
+    // back the moment serving was turned on again — re-publishing this node as
+    // a web seed, or re-listing a 700 GiB download, as a side effect of a
+    // decision about something else.
+    if (!publishingFor(wanted, this.#config).serveArchive) {
+      wanted.selfWebSeed = false;
+      wanted.publicDownload = false;
+    }
+    const after = publishingFor(wanted, this.#config);
+    await this.#catalog.put(wanted);
+
+    // Driven by what is on record rather than by the transition, so calling
+    // this twice does nothing the second time and calling it on an archive
+    // that was created with the setting already on still writes the seed. A
+    // transition test looked equivalent and was not: an import that inherits
+    // `selfWebSeed: true` from the node has no "before" in which it was off,
+    // so nothing would ever have published it.
+    const published = entry.selfWebSeedUrl ?? null;
+    let webSeed = published;
+
+    if (after.selfWebSeed && !published) {
+      const base = String(options.baseUrl ?? this.#config.publicUrl ?? '')
+        .trim()
+        .replace(/\/+$/, '');
+      if (!base) {
+        throw new Error(
+          'cannot publish this node as a web seed without knowing its own ' +
+            'URL: set publicUrl, or pass a base URL',
+        );
+      }
+      webSeed = `${base}/archives/${infoHash}/archive.pmtiles`;
+      await this.addWebSeeds(infoHash, [webSeed]);
+      // Only the field. put() merges, so spreading the entry captured before
+      // addWebSeeds would write the old webSeeds and the old magnet back over
+      // the ones it had just rewritten -- the seed would appear to be added
+      // and then vanish in the same call.
+      await this.#catalog.put({ infoHash, selfWebSeedUrl: webSeed });
+    } else if (!after.selfWebSeed && published) {
+      // Whatever was actually published, which is not necessarily the URL this
+      // node would build for itself today: the base can change underneath a
+      // node, and removing "whatever we would say now" would leave yesterday's
+      // URL in the torrent for ever.
+      await this.removeWebSeeds(infoHash, [published]);
+      await this.#catalog.put({ infoHash, selfWebSeedUrl: null });
+      webSeed = null;
+    }
+
+    return {
+      ...after,
+      webSeed,
+      // Named so a caller can warn about the one change that is not merely a
+      // setting moving: this URL is already in the hands of every peer that
+      // holds the torrent, and they will go on trying it for a while.
+      withdrewWebSeed: Boolean(published) && !after.selfWebSeed,
+    };
   }
 
   /**
@@ -2919,7 +3065,7 @@ export class Library {
       mode: details.mode ?? 'mirror',
     });
 
-    return this.#catalog.put({
+    const entry = await this.#catalog.put({
       infoHash: created.infoHash,
       name: created.name,
       size: created.size,
@@ -2939,12 +3085,37 @@ export class Library {
       // Left undefined unless asked for, so the format-based default applies
       // and a later change to that default reaches existing archives.
       sparse: details.sparse,
+      // Left undefined unless the import asked for one, so the node's own
+      // answer applies and a later change to it reaches every archive that
+      // never had an opinion.
+      serveArchive: details.serveArchive,
+      selfWebSeed: details.selfWebSeed,
+      publicDownload: details.publicDownload,
       mode: details.mode ?? 'mirror',
       retainedAt: created.retainedAt,
       origin,
       originMtime,
       stale: false,
     });
+
+    // Publishing this node as a web seed is the one of the three with an
+    // effect beyond a stored boolean: it writes a URL into the .torrent and
+    // the magnet. Done here so a watched folder and an RSS subscription get it
+    // without each having to know, and warned rather than thrown because an
+    // import that produced a good archive should not be failed by a seed URL
+    // that could not be built.
+    if (publishingFor(entry, this.#config).selfWebSeed) {
+      try {
+        await this.setPublishing(entry.infoHash, {});
+        return this.#catalog.get(entry.infoHash) ?? entry;
+      } catch (error) {
+        console.warn(
+          `[web seed] ${created.name} is not published as a web seed by this ` +
+            `node: ${error.message}`,
+        );
+      }
+    }
+    return entry;
   }
 }
 

package/src/sources.js

@@ -413,6 +413,9 @@ export class ScheduledSourceManager {
           // and one publishing a 700 GiB planet nightly is not worth reading
           // twice. Unset inherits the node's `md5`.
           md5: source.md5,
+          serveArchive: source.serveArchive,
+          selfWebSeed: source.selfWebSeed,
+          publicDownload: source.publicDownload,
           retain: source.retain !== false,
           // Left undefined the library decides, which is to publish the URL
           // unless it carries credentials. Set explicitly it is obeyed either
@@ -514,6 +517,9 @@ export class ScheduledSourceManager {
           addTrackers: source.addTrackers,
           pieceLength: source.pieceLength,
           md5: source.md5,
+          serveArchive: source.serveArchive,
+          selfWebSeed: source.selfWebSeed,
+          publicDownload: source.publicDownload,
           retain: source.retain !== false,
           webSeed: source.webSeed,
           webSeeds: source.webSeeds,

package/src/watch.js

@@ -224,6 +224,10 @@ export class WatchManager {
         // taking a nightly planet build cannot, whatever the node's default
         // says. Left unset the node's `md5` decides, as it always did.
         md5: folder.md5,
+        // Same three-level rule as md5: unset here means the node decides.
+        serveArchive: folder.serveArchive,
+        selfWebSeed: folder.selfWebSeed,
+        publicDownload: folder.publicDownload,
         comment: folder.comment,
         // Marks this as the folder's, so retention below has a family to work
         // within and nothing outside it can be caught up in one.

package/src/web/index.html

@@ -377,6 +377,21 @@
       button.state { border-color: var(--line); color: var(--muted); }
       button.speed.on,
       button.state.on { border-color: var(--warn); color: var(--warn); }
+      /* The three switches on the HTTP sources tab. Decisions about exposure
+         rather than settings, so each gets room to say what it does instead of
+         sitting in a row of bare labels. Ordinary .choice labels underneath —
+         these rules only undo the parts of that written for a one-line one. */
+      .publish { display: grid; gap: 0.7rem; margin-bottom: 1.25rem; }
+      /* .choice centres the box against its label, which is right for one line
+         and wrong for four: the box drifts down beside the middle of the hint. */
+      .publish label { align-items: flex-start; }
+      .publish input { margin-top: 0.25rem; }
+      .publish .sub { display: block; margin-top: 0.15rem; }
+      .publish code { word-break: break-all; }
+      /* A switch whose gate is off. Dimmed rather than removed: taking it out
+         would rearrange the panel under a click, and would leave no sign of
+         why the option had gone. */
+      .publish input:disabled + span { opacity: 0.55; }
     </style>
   </head>
   <body>
@@ -1802,7 +1817,61 @@
 
           if (name === 'sources') {
             const seeds = entry.webSeeds ?? [];
+            // Resolved by the node, not read off the record: an archive that
+            // says nothing defers to the node's own answer, and what belongs
+            // in a checkbox is what is actually happening.
+            const on = entry.publishing ?? {};
+            const archiveUrl = `${base}/archives/${entry.infoHash}/archive.pmtiles`;
+
+            // Three switches rather than one, because they are three separate
+            // exposures. Serving the file to a reader that was handed the URL,
+            // putting that URL in front of every peer in the swarm, and
+            // advertising it to every visitor of the public page are different
+            // decisions, and a node can reasonably want any one without the
+            // others.
+            const toggle = (key, label, hint, disabled) => `
+              <label class="choice">
+                <input
+                  type="checkbox"
+                  data-publish="${key}"
+                  ${on[key] ? 'checked' : ''}
+                  ${disabled ? 'disabled' : ''}
+                />
+                <span>
+                  <b>${label}</b>
+                  <span class="sub">${hint}</span>
+                </span>
+              </label>`;
+
             pane.innerHTML =
+              `<div class="publish">
+                 ${toggle(
+                   'serveArchive',
+                   'Serve this archive over HTTP',
+                   `<code>${escapeHtml(archiveUrl)}</code> — the file itself,
+                    by byte range, which is what any PMTiles reader wants.
+                    Off means this node answers 403 for it.`,
+                   false,
+                 )}
+                 ${toggle(
+                   'selfWebSeed',
+                   'Publish this node as a web seed for it',
+                   `Writes that URL into the torrent, so every peer holding it
+                    fetches from here. Turning it off takes the URL back out,
+                    but peers already holding the torrent keep trying it for a
+                    while.`,
+                   !on.serveArchive,
+                 )}
+                 ${toggle(
+                   'publicDownload',
+                   'Offer it as a download on the public page',
+                   `A link on the catalogue page anyone can reach. Separate
+                    from serving it: the endpoint can exist for a style or a
+                    peer without being advertised to every visitor.`,
+                   !on.serveArchive,
+                 )}
+               </div>` +
+              `<h2>Web seeds</h2>` +
               table(
                 ['URL'],
                 seeds.map((url) => [escapeHtml(url)]),
@@ -1830,6 +1899,50 @@
             // Attached here rather than in the panel, because this pane is
             // rebuilt whenever it is shown and the handler has to survive that.
             // Safe: sources is not one of the panes the refresh redraws.
+            for (const box of pane.querySelectorAll('[data-publish]')) {
+              box.onchange = async () => {
+                const key = box.dataset.publish;
+                const want = box.checked;
+                // Asked before the request, not after: this is the one that
+                // changes what strangers can pull off the disk, and a checkbox
+                // is a very quiet way to hand out 700 GiB.
+                if (
+                  key === 'serveArchive' &&
+                  want &&
+                  !window.confirm(
+                    'Serve this archive over HTTP.\n\n' +
+                      'Anyone who knows the infohash can then download the ' +
+                      'whole file — ' +
+                      (bytes(entry.size) || 'the entire archive') +
+                      ' — from this node.',
+                  )
+                ) {
+                  box.checked = false;
+                  return;
+                }
+                box.disabled = true;
+                try {
+                  const result = await api(
+                    `/api/torrents/${infoHash}/publish`,
+                    { method: 'POST', body: { [key]: want } },
+                  );
+                  // The withdrawal is worth saying out loud. The others are
+                  // visible in the checkbox that was just clicked.
+                  if (result.withdrewWebSeed) {
+                    toast(
+                      'web seed withdrawn; peers holding the torrent will ' +
+                        'keep trying it until they refresh it',
+                    );
+                  }
+                  renderDetail(infoHash);
+                } catch (error) {
+                  toast(error.message);
+                  box.checked = !want;
+                  box.disabled = false;
+                }
+              };
+            }
+
             const add = async () => {
               const url = pane.querySelector('#seed-url').value.trim();
               if (!url) return;

package/src/web/public.html

@@ -334,12 +334,27 @@
           }
           add(`${root}/archive.torrent`, '.torrent');
           if (archive.magnet) add(archive.magnet, 'magnet');
+          // Only where the node offers this archive as a download. The
+          // catalogue names it rather than the page deciding, because whether
+          // a 700 GiB file is advertised to every visitor is the operator's
+          // call and not something a page can infer from the file existing.
+          if (archive.archive) {
+            const download = el('a', null, 'download');
+            download.href = archive.archive;
+            // Named after the archive rather than after the route, which for
+            // every archive on the node is the same word.
+            download.download = archive.name ?? '';
+            links.append(download);
+          }
           card.append(links);
 
           const urls = el('div', 'urls');
           if (servable) {
             urls.append(urlRow('TileJSON', new URL(`${root}/tiles.json`, location.href).href));
           }
+          if (archive.archive) {
+            urls.append(urlRow('archive', archive.archive));
+          }
           for (const seed of archive.webSeeds ?? []) {
             urls.append(urlRow('web seed', seed));
           }