pmtiles-swarm 0.56.1 → 0.59.0

package/CHANGELOG.md

@@ -7,6 +7,150 @@
 ### 🐞 Bug fixes
 - _...Add new stuff here..._
 
+## 0.59.0
+### ✨ Features and improvements
+- **Requires pmtiles-torrent 0.10.2**, which is what actually ends the re-checking: a
+  `seedOnly` add now discards resume data that would cancel the claim, and the periodic save
+  leaves a hashing torrent alone. Everything below only stops the node making more of it.
+- **`pmtiles-swarm init` writes a first configuration, and optionally the unit to run it.**
+  Every path it writes is absolute, which is the one mistake it exists to make impossible: a
+  relative path resolves against the config file, the documented layout puts that file in
+  `/etc`, and so `./data` there means a catalog, a resume directory and potentially a 700 GiB
+  archive on the configuration partition. State under `/etc` is refused rather than warned
+  about — at the moment a config is written there is nothing to migrate, and the same mistake
+  found later costs a stopped service and a careful move.
+
+  `--systemd` also writes `pmtiles-swarm.service` beside it, with `ReadWritePaths` **derived
+  from the configuration it just wrote**. That derivation is the point. A unit and a config
+  that disagree is every systemd failure this project has diagnosed, and none of them look
+  like what they are: a `savePath` missing from that line is refused inside the unit's
+  namespace before any permission bit is read, so the directory's owner and mode are both
+  perfect and the write still fails. Two files generated from one source cannot drift.
+
+  It installs nothing. The unit is written next to the configuration and the `cp` into
+  `/etc/systemd/system` is one of the commands it prints, alongside an `install -d` for every
+  directory involved with the right owner and mode.
+
+  `--password` is hashed before it is written, and left out entirely when none is given.
+  There is deliberately no placeholder: `auth.password` accepts plaintext, so `REPLACE-ME` in
+  that field is a working password until somebody notices — a credential that looks set and
+  is not.
+
+- **Two scripts for diagnosing a library that re-checks on every start**, in `tools/`.
+  `resume-doctor.py` reads a node's real configuration, catalog, stored `.torrent` files and
+  resume directory and says, for each archive, what libtorrent will do on the next start and
+  why — applying libtorrent's own rules and citing the file and line each came from. It also
+  reads the unit through `systemctl show`, does the arithmetic on every deadline that can cut
+  a resume save short, and hashes pieces rather than trusting the catalog. `resume-experiment.py`
+  proves the five behaviours involved on whatever libtorrent is actually installed, because
+  1.2, 2.0 and 2.1 differ enough that a claim verified on one is not a claim about the other.
+
+### 🐞 Bug fixes
+- **A stop no longer abandons the sidecar mid-write, which is where resume data was going.**
+  Three separate bounds decided how long the engine step had, and the smallest won: eight
+  seconds for the step, fifteen for the whole shutdown, fifteen for the shutdown RPC. The
+  sidecar allows each torrent two seconds of its resume-save budget, so past four archives the
+  node gave up first and every torrent it had not persisted re-hashed its whole store on the
+  way back up. That is the state a library gets stuck in: checking, on every start, for hours.
+
+  The engine step is now worked out from the catalog — two seconds a torrent, over a floor —
+  and the shutdown watchdog is derived from the steps it is meant to contain rather than being
+  a second deadline kept in agreement with them by hand. It was not in agreement.
+
+  The same fixed 60s applied to the periodic save, so past thirty archives the call gave up
+  before the sidecar finished, and the written/asked counts never came back — which is why the
+  shortfall this reports could not be seen from outside. **`TimeoutStopSec` in the documented
+  unit rises from 45 to 300 seconds**, and existing installs need it raised by hand.
+
+- **A `.torrent` that moved with `dataDir` is found again instead of silently becoming a
+  magnet.** `torrentPath` is recorded absolute, so moving state out of `/etc` — which
+  `docs/running-as-a-service.md` tells you to do — left every catalog entry naming a directory
+  that no longer existed. Nothing repointed them and nothing complained, because an unreadable
+  `.torrent` was treated as "use the magnet instead".
+
+  That fallback is the damage rather than a graceful degradation. A magnet carries no metadata
+  and neither does resume data, so the archive waits on BEP 9 for a file list that only a peer
+  can supply — and for an archive this node originated there is nobody to ask. Seen in the
+  field: twenty archives at 0% in `downloading_metadata`, indefinitely, after one documented
+  migration. The current `dataDir` is now tried second, the recorded path is corrected in
+  place so the warning is printed once rather than for ever, and falling back to a magnet at
+  all now says so.
+
+## 0.58.1
+### 🐞 Bug fixes
+- **The sample configuration put every piece of state under the config file.** `"dataDir": "./data"`,
+  `"savePath": "./data/torrents-data"` and `"resumeDir": "./data/resume"` all resolve against the
+  config file — and the service guide puts that file in `/etc`. Anyone following both documents ended
+  up with a catalog, a resume directory and their archives on the partition meant for configuration,
+  having done nothing wrong. The sample now uses absolute paths, a test enforces it, and the node
+  warns at startup if state resolves under `/etc` anyway.
+
+  It is also shorter. A first config should get a node running, not demonstrate the whole surface —
+  `docs/configuration.md` is where the rest lives.
+
+- **Moving state out of `/etc` had a trap in the instructions.** `mv OLD/data NEW/data` nests when the
+  destination exists, which it does after the documented setup — so the real directory ends up one
+  level too deep, the node writes a fresh empty catalog beside it, and an intact library reads as
+  lost. `docs/running-as-a-service.md` now gives a form that cannot nest, says to repoint
+  `libtorrent.resumeDir` as well as `dataDir`, and has you count catalog entries before and after.
+
+## 0.58.0
+### ✨ Features and improvements
+- **A stable name for every kind of import, not just watched folders.** `latestLink` and
+  `latestLinkType` are now offered on watched web locations, RSS feeds and remote nodes as well — one
+  path a consumer can hold while the build behind it changes.
+
+  A scheduled source had the feature all along and no way to ask for it: the console never showed the
+  column. It was also ignoring `latestLinkType`, so a source asking for a hard link quietly got a
+  symlink. A subscription could not ask at all.
+
+  For a subscription the name is pointed at the archive **when the download finishes**. Until then
+  there is a marker file, or a sparse one still filling in, and a name resolving to either is worse
+  than no name: whatever opens it reads zeroes rather than failing.
+
+- **The node says something when its state has landed in `/etc`.** Nobody chooses that — the
+  documented service layout puts the config file there, every path resolves relative to that file,
+  and the sample reads `"dataDir": "./data"`. So the catalog and the resume directory end up on the
+  partition meant for configuration. Warned rather than corrected: it is a real path that works, and
+  moving a running node's data would be worse than saying so.
+
+### 🐞 Bug fixes
+
+## 0.57.0
+### ✨ Features and improvements
+- **The three publishing switches are one **Local file** column on the import tables.** Three
+  dropdowns per row cost more width than the choice is worth, on tables that already scroll
+  sideways — and as an import default these are almost always decided together:
+
+  | option | serves the file | web seed | on the public page |
+  | --- | --- | --- | --- |
+  | `node` | — | — | — |
+  | `off` | no | — | — |
+  | `http` | yes | no | no |
+  | `http + catalog` | yes | no | yes |
+  | `http + web seed` | yes | yes | no |
+  | `http + web seed + catalog` | yes | yes | yes |
+
+  `http + catalog` is there rather than left out to keep a tidier ladder: a dropdown that could not
+  say it would round that state up to the nearest option it had, turning a web seed on — and
+  publishing the node to the swarm — because somebody re-saved an unrelated row. Every combination
+  survives being written and read back, which is what the round-trip test checks.
+
+  The archive details panel still offers the three separately. A row here sets a policy for what
+  arrives; the panel is where one archive gets picked over.
+
+## 0.56.2
+### 🐞 Bug fixes
+- **Adding a save location in the console did nothing.** The row was read correctly and then thrown
+  away: the settings pane below renders every config key it does not explicitly skip as a raw-JSON
+  textarea, the skip list named `watch`, `sources` and `subscriptions` but not `locations`, and in
+  `saveSettings` the textarea loop runs after the row editors — so a copy of the list as it was when
+  the pane was drawn overwrote the one with the new row in it.
+
+  The skip is now derived from the registered row editors rather than listed by hand, so this cannot
+  happen again to the next editor somebody adds. The list was the bug; keeping a list and adding one
+  more name to it would have been the same bug waiting.
+
 ## 0.56.1
 ### ✨ Features and improvements
 - **Comments trimmed back to house style, and the reasoning moved into the docs where it belongs.**

package/docs/configuration.md

@@ -488,9 +488,19 @@ moment a download finishes.
 
 The node's setting is the default. A [watched folder](#watched-folders), a
 [scheduled source](#scheduled-sources), an [RSS feed](#subscriptions) and a
-remote node may each carry their own — the **Serve file**, **Web seed** and
-**Listed** columns on those tables, where `node` means "no opinion" rather than
-"off". And any individual archive can be switched in the console, under **HTTP
+remote node may each carry their own — the **Local file** column on those
+tables, which offers the three as one choice:
+
+| option                      | `serveArchive` | `selfWebSeed` | `publicDownload` |
+| --------------------------- | -------------- | ------------- | ---------------- |
+| `node`                      | unset          | unset         | unset            |
+| `off`                       | `false`        | —             | —                |
+| `http`                      | `true`         | —             | —                |
+| `http + catalog`            | `true`         | —             | `true`           |
+| `http + web seed`           | `true`         | `true`        | —                |
+| `http + web seed + catalog` | `true`         | `true`        | `true`           |
+
+`node` means "no opinion" rather than "off". And any individual archive can be switched in the console, under **HTTP
 sources** in its details.
 
 An archive that says nothing goes on following the node, so changing the node's
@@ -586,6 +596,17 @@ it, and what turns a cold tile read from tens of seconds into well under one.
 `webSeedBase` on its own assumes the watched folder is already the web root, since
 nothing is moved.
 
+**Every table that imports an archive offers this**, not just watched folders:
+a [scheduled source](#scheduled-sources), an [RSS feed](#subscriptions) and a
+remote node each take `latestLink` and `latestLinkType` too. The point is the
+same in all four — a consumer holds one path and never learns the name of any
+particular build.
+
+For a subscription the name is pointed at the archive **when the download
+finishes**, not when it is joined: until then there is a marker file or a sparse
+one still filling in, and a name resolving to either is worse than no name at
+all, because whatever opens it reads zeroes rather than failing.
+
 `latestLinkType: 'hard'` is for a name something reads the archive _through_. A
 hard link still resolves after the build it names is retired, where a symlink is
 left pointing at nothing. The other kind stays the fallback in both directions,
@@ -625,6 +646,7 @@ entry gives either a `url` template or an `index` directory:
 | `everyHours`                                    | an interval instead                                                                                           |
 | `md5`                                           | overrides the node's [`md5`](#md5) for this source alone                                                      |
 | `serveArchive`, `selfWebSeed`, `publicDownload` | override what this node offers of the archives this source fetches                                            |
+| `latestLink`, `latestLinkType`                  | a stable second name for the newest build, as on a watched folder                                             |
 
 Prefer a template where the naming is predictable: it asks a direct question,
 gets a direct answer, and needs the upstream to publish no listing at all.

package/docs/running-as-a-service.md

@@ -151,8 +151,50 @@ sudo apt-get install -y python3-libtorrent
 sudo -u pmtiles-swarm python3 -c "import libtorrent; print(libtorrent.__version__)"
 ```
 
+## The configuration, and a unit for it
+
+`init` writes both, and the unit's `ReadWritePaths` is derived from the
+configuration it just wrote. That derivation is the reason to use it: a unit and
+a configuration that disagree is the failure this whole document is arranged
+around, and two files generated from one source cannot.
+
+```sh
+sudo -u pmtiles-swarm -H /var/lib/pmtiles-swarm/node_modules/.bin/pmtiles-swarm   init --config /etc/pmtiles-swarm/swarm.config.json   --data-dir /var/lib/pmtiles-swarm/data   --save-path /mnt/store/torrent-data   --systemd --password 'the console password'
+```
+
+That writes the configuration, generates `pmtiles-swarm.service` beside it, and
+prints a command for every directory involved — each with the right owner and
+mode. Nothing privileged happens and nothing is installed: the unit is written
+next to the configuration, and copying it into `/etc/systemd/system` is one of
+the lines it hands you.
+
+Two refusals worth knowing before you type it. State under `/etc` is rejected
+rather than warned about, because at the moment a configuration is written there
+is nothing to migrate — the same mistake found six months later costs a stopped
+service and a careful move. And a configuration that already exists is left
+alone unless `--force` says otherwise, since the credentials in it are not
+recoverable.
+
+`--password` is hashed before it is written. Leave it off and there is no
+console password at all: the generated API key is the way in until you set one.
+There is deliberately no placeholder, because `auth.password` accepts plaintext
+and a placeholder in that field is a working password until somebody notices.
+
+**Re-run it after adding a watched folder**, or a subscription with a save path
+of its own. Both are directories the node writes to, and neither reaches
+`ReadWritePaths` by itself. `tools/resume-doctor.py` checks a running node's
+unit against its live configuration and says which directories are missing.
+
+The unit below is what it generates, with your paths in it. Worth reading
+either way — a generated file you do not understand is a hand-written one you
+have not written yet.
+
 ## The unit
 
+Annotated, because every directive here has cost somebody a diagnosis.
+`init --systemd` writes this file with your paths already in it; what
+follows is what those lines are for.
+
 ```ini
 [Unit]
 Description=pmtiles-swarm
@@ -177,8 +219,12 @@ Restart=always
 RestartSec=5
 
 # Stopping announces "stopped" to every tracker, releases the data directory
-# lock and cancels downloads in flight. Worst case is about 20 seconds.
-TimeoutStopSec=45
+# lock, cancels downloads in flight and writes resume data for every archive.
+# That last part is what scales: the node allows its engine two seconds per
+# torrent, so a library of fifty wants well over a minute and systemd has to
+# outlast it. Too low here and the process is killed mid-save, which costs a
+# re-hash of everything unwritten on the way back up.
+TimeoutStopSec=300
 
 # The node stops the sidecar itself, and needs it alive to do so. The default
 # signals both at once, which kills the sidecar before it can write its resume
@@ -233,6 +279,17 @@ itself, in order, and waits for the resume data to be written. Nothing is left
 running: anything still alive when `TimeoutStopSec` expires is killed, sidecar
 included.
 
+**`TimeoutStopSec`, and why it is not a round number you pick once.** Stopping
+writes resume data for every archive, and the node allows its engine two seconds
+per torrent to do it — so the figure that matters grows with the library. Three
+hundred seconds covers a hundred and forty archives; a larger node needs more.
+Set it too low and systemd kills the process mid-save, and every archive that
+had not been written re-hashes its whole store on the way back up, which for a
+700 GiB archive is hours.
+
+`tools/resume-doctor.py` prints the arithmetic for your library and says whether
+the unit's value covers it.
+
 **No `ExecStop=`.** systemd already sends `SIGTERM`, and the node handles it
 from the moment it starts. `ExecStop=/bin/kill -15 $MAINPID` is redundant, and
 becomes wrong under `KillMode=process` — that one leaves the rest of the unit
@@ -256,8 +313,10 @@ directory:
   -> /etc/pmtiles-swarm/data
 ```
 
-That is rarely what you want for a service. Use absolute paths for anything
-holding data:
+That is rarely what you want for a service, and the node says so on startup —
+`/etc` is for configuration, and a catalog, a resume directory and possibly an
+archive do not belong on that partition. The shipped sample uses absolute paths
+for exactly this reason. Use them for anything holding data:
 
 ```json
 {
@@ -267,6 +326,33 @@ holding data:
 }
 ```
 
+#### Moving state that already landed in the wrong place
+
+Repoint **every** path that lived under the directory being moved, not just
+`dataDir`: `libtorrent.resumeDir` is the one that is easy to miss, because it is
+absolute and sits inside it.
+
+And move with a destination that cannot nest. `mv /etc/pmtiles-swarm/data
+/var/lib/pmtiles-swarm/data` puts the source _inside_ the destination when the
+destination already exists — which it does, since the setup above creates it —
+leaving `/var/lib/pmtiles-swarm/data/data`. The node then finds no catalog where
+it was told to look, writes a fresh empty one, and every archive appears to have
+been lost.
+
+```bash
+sudo systemctl stop pmtiles-swarm
+sudo python3 -c "import json;print(len(json.load(open('OLD/catalog.json'))['entries']))"
+
+sudo mv /etc/pmtiles-swarm/data /var/lib/pmtiles-swarm/   # note: no second 'data'
+# repoint dataDir *and* libtorrent.resumeDir, then
+
+sudo systemctl start pmtiles-swarm
+sudo python3 -c "import json;print(len(json.load(open('NEW/catalog.json'))['entries']))"
+```
+
+The entry count before and after is the check that matters: an empty new catalog
+is indistinguishable from a lost library until you count.
+
 ### 2. Every one of them in `ReadWritePaths`
 
 `ProtectSystem=strict` presents the whole filesystem as read-only inside the
@@ -750,3 +836,111 @@ once rather than diagnosing later:
   `journalctl -u pmtiles-swarm | grep -i onComplete` — and a hook redirecting its
   own output to a file will have nothing for the journal to show, which is not
   the same as not having run.
+
+## When archives re-check on every start
+
+The symptom is always the same shape — a restart, and then a library sitting at
+0% or grinding through a re-hash of archives that are whole on the disk. The
+causes are not the same shape at all, so guessing between them is expensive:
+re-hashing a 700 GiB archive is half an hour during which it serves nobody.
+
+Two scripts in `tools/` answer it without guessing. Neither starts anything,
+neither writes anything, and both are safe against a running node.
+
+```sh
+sudo -u pmtiles-swarm python3 tools/resume-doctor.py \
+  -c /etc/pmtiles-swarm/swarm.config.json
+```
+
+**`resume-doctor.py`** reads your configuration, catalog, stored `.torrent`
+files and resume directory, and says for each archive what libtorrent will do on
+the next start and why — seed at once, stat its files, or re-hash the store. It
+applies libtorrent's own rules rather than a guess at them, and cites the file
+and line each one came from. It also reads the unit through `systemctl show`,
+does the arithmetic on every deadline that can cut a resume save short, and
+scans the journal for the failures that leave a trace.
+
+```sh
+sudo -u pmtiles-swarm python3 tools/resume-experiment.py
+```
+
+**`resume-experiment.py`** proves the behaviour on the libtorrent you actually
+have, rather than the one the documentation was written against. It builds a
+real torrent over a temporary file, saves real resume data, adds it again, and
+reports what happened. 2.0 and 2.1 differ enough to be worth the thirty seconds.
+
+Three things they exist to tell apart, because the first two look identical from
+outside and want opposite fixes:
+
+- **No resume data is not the problem.** An archive recorded complete is added
+  with `seed_mode`, and that claim stands on its own: libtorrent stats the files
+  and seeds, without hashing a byte. A missing resume file costs nothing.
+- **Stale resume data is the problem, and it is worse than none.** A resume file
+  holding a _partial_ bitfield cancels the `seed_mode` claim — libtorrent drops
+  it if a single piece is unset. The archive then comes back as a downloader
+  with every byte already on disk, and fetches again what it has. Deleting that
+  one file is the repair; with none, it seeds at once.
+- **A short or missing file is a genuine re-check.** `mismatching_file_size` is
+  the only thing here that really does mean re-hashing, and it is about the data
+  on disk rather than about resume data at all. libtorrent 2.x records no
+  mtimes, so nothing in this depends on a timestamp.
+
+Which archives carry stale resume data is not random, and the pattern is worth
+knowing before it looks like a second bug. An archive from a watched folder was
+read end to end before it was ever registered: it is complete from its first
+moment and no partial bitfield is ever written for it. An archive joined from a
+feed, a magnet or a peer is genuinely partial for hours, with partial resume
+data written for it every `resumeSaveIntervalSeconds` throughout. Only the
+second kind has anything to leave behind, so trouble concentrating there is this
+failure's ordinary shape rather than something the import path introduced.
+
+Two more things the doctor looks at, because they are the ones a snapshot alone
+gets wrong.
+
+**Whether the catalog still points at its own `.torrent` files.** Each entry
+records `torrentPath` as an absolute path, and restore reads that rather than
+recomputing it — so moving `dataDir` out of `/etc`, which this guide tells you
+to do, leaves every recorded path behind. Nothing warns, because a missing
+`.torrent` is not treated as an error: restore quietly falls back to the magnet.
+That fallback is the damage. A magnet carries no metadata, resume data does not
+carry it either, and so the archive waits on BEP 9 for a file list that only a
+peer can supply — from a swarm where this node is the origin. It sits at 0% in
+`downloading_metadata` indefinitely, which no amount of re-checking will fix.
+
+**Whether the periodic save gets through the whole library.** Every torrent is
+asked to save in the same cycle, so a healthy `resumeDir` has all its files
+within one interval of each other. Ages spread wider than that mean some cycles
+are finishing early, and the ages alone cannot say whether the same archives
+lose every time. `--watch` measures it rather than inferring it:
+
+```sh
+sudo -u pmtiles-swarm python3 tools/resume-doctor.py \
+  -c /etc/pmtiles-swarm/swarm.config.json --watch 660
+```
+
+It reports each save cycle as it happens, how many of them covered every
+archive, and which archives were never written at all.
+
+**A poisoned archive may heal itself, so run this twice.** The commonest way a
+whole archive acquires a partial bitfield is a save taken while it was
+re-checking: `write_resume_data` truncates `have_pieces` to the pieces checked
+so far, and the five-minute timer lands inside that window every time a large
+archive checks. That shortened bitfield is what cancels `seed_mode` on the next
+start — so the archive comes back partial, checks again, and writes another one.
+The loop needs no restart to keep itself going.
+
+The doctor tells the two apart by length: a bitfield shorter than the torrent
+was written mid-check, and one at full length is what a finished check
+concluded. A check that completes rewrites its own resume file correctly, so an
+archive poisoned in one run can read `seeds at once` in the next with nothing
+done to it. **Only the archives that stay poisoned across two runs, several
+minutes apart, are worth deleting a file for.**
+
+Two things it deliberately does not treat as faults. An archive whose data sits
+outside the `savePathLayout` shape is normal — one adopted from a file this node
+already holds keeps that file, and a node may keep archives beside whatever
+produced them. What matters is whether the data is where the entry says, which
+is checked directly. And every directory an archive actually lives in has to
+appear in `ReadWritePaths`, not just the `savePath` in the configuration; the
+doctor checks each one in use, because a deliberately-placed archive is exactly
+the case that gets left out of the unit.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "pmtiles-swarm",
-  "version": "0.56.1",
+  "version": "0.59.0",
   "description": "BitTorrent distribution for PMTiles map archives: create torrents, watch folders, publish and subscribe to RSS feeds, and seed through qBittorrent or an embedded client",
   "type": "module",
   "main": "src/index.js",
@@ -46,7 +46,7 @@
     "maplibre-gl": "^6.2.0",
     "parse-torrent": "^11.0.24",
     "pmtiles": "^4.4.1",
-    "pmtiles-torrent": "^0.9.1",
+    "pmtiles-torrent": "^0.10.2",
     "webtorrent": "^3.0.21"
   },
   "engines": {

package/src/config.js

@@ -507,6 +507,76 @@ function clone(value) {
   return value;
 }
 
+/**
+ * Every directory this configuration means to write to.
+ *
+ * Which is exactly what `ReadWritePaths` has to name. Under
+ * `ProtectSystem=strict` a directory missing from that line is refused inside
+ * the unit's namespace, before any permission bit is consulted — so it fails
+ * with ownership and mode both perfect, which is a hard thing to go looking
+ * for. Deriving the list from the config rather than writing it out by hand is
+ * the only way the two cannot drift.
+ *
+ * See docs/running-as-a-service.md — "Where it writes".
+ * @param {object} config - A config whose paths have been resolved.
+ * @param {string} [configPath] - The config file, whose directory is written to.
+ * @returns {string[]} - Absolute directories, deduplicated and shortest-first.
+ */
+export function writablePaths(config, configPath) {
+  const found = [
+    config?.dataDir,
+    config?.savePath,
+    config?.cacheSavePath,
+    config?.libtorrent?.resumeDir,
+    config?.torrentDropDir,
+    // The console rewrites the configuration when a token is minted, so the
+    // directory holding it is written to as surely as any of the above.
+    configPath ? path.dirname(path.resolve(configPath)) : undefined,
+    ...(config?.watch ?? []).map((entry) => entry?.path),
+    ...(config?.locations ?? []).map((entry) => entry?.path),
+    ...(config?.subscriptions ?? []).map((entry) => entry?.savePath),
+  ].filter((value) => typeof value === 'string' && value);
+
+  // Deduplicated but deliberately not collapsed into common ancestors. A
+  // shorter list would grant the same access — `ReadWritePaths` covers a
+  // directory and everything under it — but the two callers want different
+  // things from it, and only one of them would be served. Creating the
+  // directories needs each of them named; and collapsing quietly widens the
+  // grant to whatever ancestor happens to be shared, which for a config beside
+  // its data is the whole tree above both.
+  return [...new Set(found.map((value) => path.resolve(value)))].sort();
+}
+
+/**
+ * Complains about state that has landed in /etc.
+ *
+ * Nobody chooses this. The documented service layout puts the config file in
+ * /etc, every path resolves relative to that file, and the sample read
+ * "./data" — so the catalog and the resume directory ended up on the partition
+ * meant for configuration. Warned rather than corrected: it is a real path
+ * that works, and moving a running node's data would be worse than saying so.
+ *
+ * See docs/running-as-a-service.md — "The paths in the configuration".
+ * @param {object} config - A config whose paths have been resolved.
+ * @returns {string[]} - What to say, empty when there is nothing to say.
+ */
+export function stateUnderEtc(config) {
+  const named = [
+    ['dataDir', config?.dataDir],
+    ['savePath', config?.savePath],
+  ];
+  return named
+    .filter(
+      ([, value]) => typeof value === 'string' && value.startsWith('/etc/'),
+    )
+    .map(
+      ([name, value]) =>
+        `[config] ${name} is ${value}. /etc is for configuration; state ` +
+        'belongs under /var/lib. Give it an absolute path, or let systemd ' +
+        'StateDirectory= make one.',
+    );
+}
+
 /**
  * Loads configuration from a JSON file, applying defaults and environment
  * overrides. A missing file is fine — the defaults run.
@@ -573,6 +643,9 @@ export async function loadConfig(configPath) {
   const { savePath, conflict } = resolveSavePath(config);
   config.savePath = path.resolve(base, savePath);
   config.savePathConflict = conflict;
+
+  for (const message of stateUnderEtc(config)) console.warn(message);
+
   // Kept in step, because older code and older configs both reach for it.
   config.webtorrent = { ...config.webtorrent, savePath: config.savePath };
   if (config.libtorrent) config.libtorrent.savePath = config.savePath;

package/src/engines/composite.js

@@ -433,7 +433,7 @@ export class CompositeEngine {
    * @returns {Promise<{written: number, asked: number}>} - Totals across every
    *   engine that keeps resume data.
    */
-  async saveResume(infoHash) {
+  async saveResume(infoHash, options = {}) {
     // Summed rather than dropped, so a caller can tell the difference between
     // "every torrent wrote" and "half of them will be re-hashed on the next
     // start". An engine that fails outright contributes nothing to either
@@ -444,7 +444,7 @@ export class CompositeEngine {
       // WebTorrent keeps none, and says so by not offering the method.
       if (!engine.saveResume) continue;
       try {
-        const result = (await engine.saveResume(infoHash)) ?? {};
+        const result = (await engine.saveResume(infoHash, options)) ?? {};
         written += Number(result.written) || 0;
         asked += Number(result.asked) || 0;
       } catch (error) {
@@ -653,19 +653,21 @@ export class CompositeEngine {
    * Shuts every engine down.
    * @returns {Promise<void>} - Resolves once all are stopped.
    */
-  async destroy() {
+  async destroy(options = {}) {
     this.#stopping = true;
     if (this.#timer) clearInterval(this.#timer);
     this.#timer = undefined;
 
     for (const engine of this.#secondaries) {
       await engine
-        .destroy()
+        .destroy(options)
         .catch((error) =>
           console.error(`[engine] ${engine.name}: ${error.message}`),
         );
     }
-    await this.#primary.destroy();
+    // Forwarded, because the primary is the one holding resume data and the
+    // budget was worked out from how much of it there is to write.
+    await this.#primary.destroy(options);
   }
 }
 

package/src/engines/libtorrent.js

@@ -763,25 +763,40 @@ export class LibtorrentEngine {
    *   were told to write resume data, and how many actually did before the
    *   deadline.
    */
-  async saveResume(infoHash) {
+  async saveResume(infoHash, options = {}) {
     // Returned rather than discarded. The sidecar reports both numbers, and
     // the gap between them is the thing worth knowing: a torrent that did not
     // write is one that gets re-hashed on the next start, which for a 700 GiB
     // archive is the difference between seeding in seconds and seeding in half
     // an hour. That answer was being thrown away here.
-    return this.#call('save_resume', { infoHash });
+    //
+    // The default #call timeout is 60s and the sidecar's own budget is two
+    // seconds per torrent, so past thirty archives the call gave up first —
+    // and then the counts never came back, which is why the shortfall this
+    // reports could not be seen from outside.
+    return this.#call('save_resume', { infoHash }, options.timeoutMs);
   }
 
   /**
    * Saves resume data and stops the sidecar.
+   *
+   * The timeout is the caller's to set, because only the caller knows how many
+   * torrents are about to be written down and the sidecar spends two seconds
+   * per torrent. Fifteen seconds was the fixed value, which meant every
+   * library past seven archives had its resume save cut off — and each torrent
+   * that missed was re-hashed in full on the way back up.
+   * @param {object} [options] - Overrides.
+   * @param {number} [options.timeoutMs] - How long the sidecar gets to finish.
    * @returns {Promise<void>} - Resolves once stopped.
    */
-  async destroy() {
+  async destroy(options = {}) {
     // Set before anything else, so the exit this is about to cause is
     // recognised as intended by the handler that sees it.
     this.#stopping = true;
     if (!this.#child) return;
-    await this.#call('shutdown', {}, 15000).catch(() => {});
+    await this.#call('shutdown', {}, options.timeoutMs ?? 15000).catch(
+      () => {},
+    );
     this.#child?.kill();
     this.#child = null;
     this.#ready = null;