pmtiles-swarm 0.29.0 → 0.31.0

package/CHANGELOG.md

@@ -7,6 +7,69 @@
 ### 🐞 Bug fixes
 - _...Add new stuff here..._
 
+## 0.31.0
+### ✨ Features and improvements
+- **A Recheck files button, for when an archive's progress and its files disagree.** Every figure a
+  node can give you about how much of an archive is present is derived from something written down
+  earlier: the catalog's `complete` flag, the engine's resume data, the `seedOnly` claim made when
+  the torrent was added. When one of those is wrong there is no path back on its own — an archive
+  built on this node whose entry says `complete: false` is re-added without `seedOnly`, so the
+  engine goes looking for bytes that are already under its nose and sits at 0% beside a finished
+  file. Every restart reaches the same conclusion.
+
+  `POST /api/torrents/<infohash>/recheck` hashes every piece against the torrent and the result
+  wins. libtorrent does it with `force_recheck` through the sidecar (pmtiles-torrent 0.5.1 or
+  newer); qBittorrent has the same operation in its WebUI API. WebTorrent has none — it verifies on
+  add and never again — so it is re-added with the "the data is already here" claim withheld, which
+  is reported as `method: "readd"` rather than dressed up as the same mechanism.
+
+  Answers as soon as the check is under way, because hashing a planet build is tens of minutes and
+  no request should be held open for it. The archive reports state `checking` with progress as the
+  fraction hashed; progress running backwards during that is the operation working. Nothing is
+  deleted, and nothing is written to the catalog — the answer arrives where progress always does,
+  and the completion sweep already acts on it.
+
+  With two engines both are asked, since each keeps its own belief about the same file and a stale
+  one on the secondary is why a browser peer would find nothing while the primary seeds happily.
+
+- _...Add new stuff here..._
+
+### 🐞 Bug fixes
+- _...Add new stuff here..._
+
+## 0.30.0
+### ✨ Features and improvements
+- **The style URL now carries the `.torrent` URL as well as the magnet.** Piece hashes reach a
+  browser only from a peer, over BEP 9 — there is no other route to them — so a magnet alone
+  leaves a page waiting on a tracker connection and a WebRTC handshake before it can read a byte,
+  and never gets there at all on a network that blocks the trackers. One ordinary HTTPS request for
+  the metainfo removes that dependency, and saves a conventional client the same round trip.
+
+  Both handles now ride in the fragment of the `styleUrl` on `/api/categories` and `/latest/`, and
+  of what the console's **Copy TileJSON URL + swarm** button produces. A fragment is still never
+  sent in a request, so an ordinary client fetches the TileJSON and ignores all of it.
+
+  For a category the `.torrent` handle is category-scoped too — `/latest/<category>/archive.torrent`
+  redirects to whatever build is current — so unlike a plain magnet it does not go stale on the next
+  build. That was previously true only where the node publishes a BEP 46 key.
+
+  **This is a format change.** The fragment used to be a bare `#magnet:?…`; it is now
+  `#torrent=…&magnet=…` with both values percent-encoded, because `&` separates them and a magnet
+  is full of them. A reader that took the whole fragment for a magnet must read `magnet=` out of it
+  — `URLSearchParams` does it in one call. The bare form was not kept for the single-handle case: a
+  fragment whose shape depends on what happened to be available means every reader has to handle
+  both anyway.
+
+- **The Added column shows the same date and time on every row.** It used to shorten today's rows
+  to a time and older ones to a date, which reads as two different quantities in one column and
+  makes the eye stop to work out which it is looking at. Seconds are dropped rather than the date,
+  since nothing here is sorted finely enough for them to matter; hovering still gives them.
+
+- _...Add new stuff here..._
+
+### 🐞 Bug fixes
+- _...Add new stuff here..._
+
 ## 0.29.0
 ### ✨ Features and improvements
 - **A connection indicator in the header, for whether the swarm can reach this node.** A node

package/README.md

@@ -60,9 +60,11 @@ day are followed by a dated URL template or by reading a directory listing, on a
 per source.
 
 **Has a console.** Everything above is done from a web UI in the shape of a torrent client:
-archives with progress, ratio and expiry; tabbed detail with trackers, peers, HTTP sources and
-content; and a settings screen covering monitored folders, watched web locations, remote nodes,
-save locations, access tokens and the external-program hooks.
+archives with progress, ratio, expiry and when each was added, searchable and sortable by any of
+them; tabbed detail with trackers, peers, HTTP sources and content; a chart of what the swarm has
+been sending and receiving, kept in SQLite so it survives a restart; an indicator saying whether
+peers can reach this node at all; and a settings screen covering monitored folders, watched web
+locations, remote nodes, save locations, access tokens and the external-program hooks.
 
 **Publishes RSS.** `/feed.xml`, and `/feed/<category>.xml` per category. Plain RSS 2.0 with
 torrent enclosures, so **qBittorrent's built-in RSS auto-downloader can subscribe today** with no
@@ -739,7 +741,7 @@ which the endpoint answers 501.
 
 | Method                | Path                                                        | Purpose                                                                                                                                                                                                                                   |
 | --------------------- | ----------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
-| `GET`                 | `/api/status`                                               | Engine health, counts, watched folders, save locations and free space                                                                                                                                                                     |
+| `GET`                 | `/api/status`                                               | Engine health, counts, watched folders, save locations, free space, and whether the swarm can reach this node                                                                                                                             |
 | `GET`                 | `/api/torrents`                                             | Catalog joined with live swarm state and what is left of each seeding limit                                                                                                                                                               |
 | `POST`                | `/api/torrents`                                             | Add via `{path}`, `{url}`, `{magnet}`, `{torrentUrl}`, or a raw `.torrent` body                                                                                                                                                           |
 | `DELETE`              | `/api/torrents/:infoHash`                                   | Remove (`?deleteData=true` to delete data too)                                                                                                                                                                                            |
@@ -751,6 +753,7 @@ which the endpoint answers 501.
 | `PATCH`               | `/api/torrents/:infoHash/categories`                        | Set, add or remove categories                                                                                                                                                                                                             |
 | `PATCH`               | `/api/torrents/:infoHash/seeding`                           | Per-archive seeding limit, or "use the global one"                                                                                                                                                                                        |
 | `PATCH` `GET`         | `/api/torrents/:infoHash/location`                          | Move the data; poll the move                                                                                                                                                                                                              |
+| `POST`                | `/api/torrents/:infoHash/recheck`                           | Hash what is on disk again and believe the result — for an archive whose progress and whose files disagree                                                                                                                                |
 | `POST`                | `/api/torrents/:infoHash/pause`, `/resume`                  | Stop offering it, without forgetting it                                                                                                                                                                                                   |
 | `POST`                | `/api/torrents/:infoHash/webseeds`                          | Add web seeds — does not change the infohash                                                                                                                                                                                              |
 | `POST`                | `/api/torrents/:infoHash/warm`                              | Pre-fetch a region (`GET` for progress, `DELETE` to cancel)                                                                                                                                                                               |

package/docs/architecture-diagram.md

@@ -255,19 +255,22 @@ remember where it had got to.
 
 ### Bootstrapping without the server
 
-A torrent-aware client still has to _learn_ the magnet from somewhere, and until
-it does, the swarm — the part that depends on no server — is unreachable
-precisely when the server is down. The fix is that the magnet travels in the
+A torrent-aware client still has to _learn_ where to join from somewhere, and
+until it does, the swarm — the part that depends on no server — is unreachable
+precisely when the server is down. The fix is that two handles travel in the
 **fragment** of the TileJSON URL a style already carries:
 
 ```
-https://swarm.example.org/latest/openmaptiles/tiles.json#magnet:?xs=urn:btpk:…&s=openmaptiles&ws=…
+https://…/latest/openmaptiles/tiles.json#torrent=<the .torrent URL>&magnet=<magnet:?xs=urn:btpk:…>
 ```
 
 A fragment is never sent in an HTTP request, so ordinary clients fetch the
-TileJSON and ignore it while a swarm-aware one reads the magnet before making any
-call. With a BEP 46 key in it (`xs=urn:btpk:`) rather than an infohash, that
-string does not go stale on the next build either. See
+TileJSON and ignore it while a swarm-aware one joins before making any call. The
+`.torrent` is the fast handle — a browser can get piece hashes no other way than
+from a peer, so fetching the metainfo over HTTPS is what lets a page start
+without waiting on one — and the magnet is the one that needs no server at all.
+With a BEP 46 key in it (`xs=urn:btpk:`) rather than an infohash, neither goes
+stale on the next build. See
 [serving-tiles.md](serving-tiles.md#a-fragment-that-survives-a-rebuild).
 
 ---

package/docs/configuration.md

@@ -566,6 +566,28 @@ Note what the client address means behind a proxy: without `X-Forwarded-For` it
 the proxy's own address, which still answers whether a request arrived directly or
 through it, but not who sent it.
 
+### `traffic`
+
+What the swarm side has been moving, at `GET /api/traffic` and as the chart in the
+console. Upload and download speed per archive, sampled on a timer.
+
+| setting                 | default |                           |
+| ----------------------- | ------- | ------------------------- |
+| `traffic.sampleSeconds` | `15`    | how finely it looks       |
+| `traffic.keepHours`     | `168`   | how far back it remembers |
+
+Two knobs because they are two questions. Unlike `tileStats` this is kept in
+SQLite, in `stats.db` beside the catalog rather than beside the configuration, and
+the difference is deliberate: tile counters answer a question about now and are
+cheap to rebuild by waiting, while a bandwidth history answers a question about
+the past and cannot be rebuilt at all — restarting to pick up a new version would
+erase exactly the week somebody wanted to look at.
+
+A week of 15-second samples is roughly 40,000 rows per archive, a few megabytes
+for a node carrying twenty. Both settings reload in place. See
+[running-as-a-service.md](running-as-a-service.md#the-statistics-database) for
+sizing and for what a locked-down unit file has to allow.
+
 ## Checking for changed sources
 
 `originCheckIntervalSeconds` (default `0`, disabled) controls how often to
@@ -586,8 +608,8 @@ A changed source does not invalidate the existing torrent. See
 
 | setting                     | default |                                                        |
 | --------------------------- | ------- | ------------------------------------------------------ |
-| `fetchAttempts`             | `10`    | how many times to resume a download that stopped early |
-| `fetchRetrySeconds`         | `5`     | how long to wait before resuming                       |
+| `fetchAttempts`             | `10`    | consecutive failures without progress before giving up |
+| `fetchRetrySeconds`         | `30`    | base delay, multiplied by the consecutive count        |
 | `resumeSaveIntervalSeconds` | `300`   | how often to write resume data. `0` disables it        |
 
 A planet archive is hours of transfer, and a connection that drops partway is
@@ -598,6 +620,28 @@ an ETag or Last-Modified to prove the file has not changed underneath. Where it
 does not, the download restarts, because splicing two builds together produces a
 torrent for bytes that never existed.
 
+**The budget counts consecutive failures that moved nothing, not failures.** An
+attempt that transferred bytes proves the source and the route are alive, so it
+resets the count. Counting every failure made the budget a property of the whole
+download rather than of the trouble it is in, which for a large archive are not
+the same thing: a 700 GiB transfer reached 226 GB across six separate stalls and
+then spent its last four attempts on one bad minute, because a quarter of a
+terabyte of progress counted for nothing. Progress is measured against the
+high-water mark rather than the previous attempt, since an attempt can fail
+having written less than was already on disk.
+
+The delay grows with the consecutive count — at the default, 30s then 60s then
+90s, so ten of them span something over twenty minutes rather than the
+forty-five seconds a flat delay gave. Because progress resets the count, an
+unlucky download can go round more times than `fetchAttempts` names; that is
+intended, and bounded by a ceiling of ten times the budget so a source dribbling
+a few bytes before dropping cannot retry for ever.
+
+**Giving up keeps the partial file.** The staging path is derived from the URL,
+so re-adding the same URL in the console resumes from where it stopped instead of
+starting again, and the error says how many bytes are there to resume from. Only
+cancelling deletes it.
+
 Resume data is what lets a restart skip re-hashing the store — on an 800 GB
 archive, the difference between instant and half an hour. A clean stop always
 writes it; `resumeSaveIntervalSeconds` is for the stops that are not clean, where

package/docs/engines.md

@@ -223,6 +223,35 @@ unforwarded NAT can never connect to each other, so the ones that need you most
 cannot serve. UPnP and NAT-PMP are on by default in both engines and will often open it for you;
 a router with either disabled will not say so.
 
+### The console says whether it worked
+
+Nothing about a node's own traffic reveals that half the swarm cannot reach it. It dials out, its
+transfers work, and it looks healthy — the cost is invisible and permanent. So the console header
+carries an indicator, also in `GET /api/status` as `reachability`:
+
+| Colour | State      | Means                                                  |
+| ------ | ---------- | ------------------------------------------------------ |
+| green  | `open`     | something has connected inward — the port is reachable |
+| amber  | `unproven` | listening, and nothing ever has                        |
+| red    | `offline`  | not listening at all                                   |
+| hidden | `unknown`  | the engine cannot answer                               |
+
+libtorrent answers from `net.has_incoming_connections`; WebTorrent has no such gauge, so it is
+assembled from the wires, each of which carries the direction it was made in. Both latch: the
+question is whether the swarm _can_ reach this node, not whether somebody is connected right now,
+so a reachable node that is merely quiet stays green rather than dropping to amber when its last
+peer leaves.
+
+Reported per engine, not blended. Two engines means two listening ports, forwarded separately, and
+one can be reachable while the other is not; the header shows the primary and names both on hover.
+
+**Amber is not a fault.** On a node no peer has tried, blocked and untried are the same
+observation, and nothing available separates them — which is why it reads "no incoming yet" rather
+than "firewalled". On a busy node it will turn green within minutes; if it does not, the port is
+worth checking. For the same reason an engine that cannot be asked hides the indicator instead of
+showing red: not being able to ask is not the same as being unreachable. The libtorrent side needs
+pmtiles-torrent 0.5.0 or newer.
+
 ### Every libtorrent network setting
 
 ```json
@@ -315,6 +344,43 @@ Trackers live outside the torrent's `info` dictionary, so adding them **does not
 infohash** — but it only applies to torrents created after the change. An existing archive keeps
 announcing where its own torrent says to.
 
+## Rechecking what is on disk
+
+Every figure a node can give you about how much of an archive is present is derived from something
+written down earlier: the catalog's `complete` flag, the engine's resume data, the `seedOnly` claim
+made when the torrent was added. When one of those is wrong there is no path back on its own — the
+archive sits at 0% beside a finished file, downloading bytes it already has, and every restart
+repeats the same conclusion.
+
+**Recheck files** in the console (`POST /api/torrents/<infohash>/recheck`) is the one operation
+that goes and looks. It hashes every piece against the torrent and the result wins.
+
+| Engine      | How                                                             |
+| ----------- | --------------------------------------------------------------- |
+| libtorrent  | `force_recheck`, via the sidecar. Needs pmtiles-torrent ≥ 0.5.1 |
+| qBittorrent | `POST /api/v2/torrents/recheck` — the same library underneath   |
+| WebTorrent  | no such operation; re-added with `seedOnly` withheld instead    |
+
+It answers as soon as the check is under way. Hashing a planet build is tens of minutes, which is
+longer than any request should be held open for, so the archive reports state `checking` with
+progress as the fraction hashed. **Progress running backwards during that is the operation
+working**, not a fault. Nothing is deleted at any point.
+
+WebTorrent's route is cruder and is reported as `method: "readd"` rather than dressed up as the
+same thing: it verifies on add and never again, so the only way to make it look is to add the
+torrent a second time with the "the data is already here" claim withheld. It also refuses on a
+paused archive, since a re-add would start it.
+
+Nothing is written to the catalog when the check begins. The answer arrives where progress always
+does — the engine's own figures, which the completion sweep already reads. Note the one thing this
+does not repair on its own: the sweep promotes and never demotes, so a recheck finding _less_ than
+the record claims shows the truth in the console but leaves `complete: true` in place. Demoting on
+a progress figure would strip the finished name off any archive that happened to be mid-check.
+
+With two engines both are asked. Each keeps its own belief about the same file, so a stale one on
+the secondary is why a browser peer finds nothing while the primary seeds happily. A secondary that
+cannot check is reported and does not fail the operation; the primary failing does.
+
 ## Marking incomplete files
 
 An archive that is not whole yet is written under a marked name and renamed when
@@ -346,7 +412,10 @@ checked and the unreadable ones are joined by magnet instead.
 
 ## Writing another engine
 
-Implement `connect`, `add`, `remove`, `list`, `get`, `destroy`, and optionally `peers`.
+Implement `connect`, `add`, `remove`, `list`, `get`, `destroy`, and optionally `peers`,
+`reachability` and `recheck`. Anything optional that is missing is simply not offered — an engine
+without `reachability` hides the indicator rather than reporting a node unreachable, and one
+without `recheck` is rechecked by re-adding instead.
 The interface is deliberately small; see
 [`src/engines/types.js`](../src/engines/types.js) for the contract and
 [`src/engines/webtorrent.js`](../src/engines/webtorrent.js) for the shortest example.

package/docs/publishing.md

@@ -231,10 +231,15 @@ rather than exceptional. Each attempt continues from the bytes already on disk w
 HTTP range request, so a drop costs the retry delay instead of everything transferred so
 far.
 
+The budget counts _consecutive_ failures that moved nothing: an attempt that transferred
+bytes proves the source and the route are alive, so it resets the count. The delay grows
+with that count — 30s, 60s, 90s — so ten of them span an outage rather than a moment. When
+it does give up the partial file is kept, and re-adding the same URL resumes from it.
+
 ```json
 {
   "fetchAttempts": 10,
-  "fetchRetrySeconds": 5
+  "fetchRetrySeconds": 30
 }
 ```
 

package/docs/running-as-a-service.md

@@ -566,6 +566,15 @@ own, and not `torrentPort`. `6883` sits clear of both.
 Note that each engine runs its own DHT as well, so a node with both engines has
 three UDP participants. Only this one is yours to place.
 
+### Checking the forward actually happened
+
+A forward that did not take is invisible from here: the node still dials out, its
+transfers still work, and nothing in its own numbers says half the swarm can
+never open a connection to it. The console header answers it — green once
+something has connected inward, amber while nothing has, red if the engine is not
+listening at all. Give a new node a few minutes of seeding before reading it;
+amber on a node no peer has tried yet means untried, not blocked.
+
 ## Updating
 
 ```sh

package/docs/serving-tiles.md

@@ -192,20 +192,20 @@ Both peer-to-peer engines reuse the client already seeding the archive rather
 than starting a second one. One peer pool, one port, one DHT node — and for
 libtorrent, one sidecar process.
 
-## Carrying the magnet in the URL fragment
+## Carrying the swarm in the URL fragment
 
 The `torrent` block below solves the problem _after_ the TileJSON has been
 fetched. It does not solve the one before it: a torrent-aware client that cannot
 reach this server has nothing to work with, so the swarm — the part that does
 not depend on any server — is unreachable precisely when the server is down.
 
-The fix is to put the magnet in the URL **fragment**:
+The fix is to put the ways into the swarm in the URL **fragment**:
 
 ```json
 "sources": {
   "openmaptiles": {
     "type": "vector",
-    "url": "https://swarm.example.org/latest/openmaptiles/tiles.json#magnet:?xt=urn:btih:4813a0e6…&dn=…&tr=…&ws=…"
+    "url": "https://swarm.example.org/latest/openmaptiles/tiles.json#torrent=https%3A%2F%2Fswarm.example.org%2Flatest%2Fopenmaptiles%2Farchive.torrent&magnet=magnet%3A%3Fxt%3Durn%3Abtih%3A4813a0e6…"
   }
 }
 ```
@@ -213,34 +213,72 @@ The fix is to put the magnet in the URL **fragment**:
 A fragment is never sent in an HTTP request, so the same string works
 everywhere:
 
-| Client                            | What happens                                                                      |
-| --------------------------------- | --------------------------------------------------------------------------------- |
-| maplibre-gl-js, Leaflet, anything | fetches the TileJSON, ignores the fragment                                        |
-| maplibre-native without a plugin  | the same                                                                          |
-| torrent-aware                     | reads the magnet **before any network call**, and still has it if the fetch fails |
+| Client                            | What happens                                                        |
+| --------------------------------- | ------------------------------------------------------------------- |
+| maplibre-gl-js, Leaflet, anything | fetches the TileJSON, ignores the fragment                          |
+| maplibre-native without a plugin  | the same                                                            |
+| torrent-aware                     | joins **before any network call**, and still can if the fetch fails |
 
-The console's **Copy TileJSON URL + magnet** button produces exactly this.
+The console's **Copy TileJSON URL + swarm** button produces exactly this, and so
+does the `styleUrl` on every row of `/api/categories` and `/latest/`.
 
-A magnet needs no encoding in a fragment — RFC 3986 allows `?`, `&`, `=` and `:`
-there — and leaving it readable matters for something people paste into a style
-file by hand.
+### Two handles, and why both
+
+They are not a ladder from worse to better. They fail in different directions.
+
+| Handle     | Needs           | Gets you                                            |
+| ---------- | --------------- | --------------------------------------------------- |
+| `torrent=` | this host       | the metainfo itself, over one ordinary HTTP request |
+| `magnet=`  | a peer, no host | everything, eventually, from the swarm alone        |
+
+`torrent=` is not redundant with the URL it is attached to. **Piece hashes reach
+a browser only from a peer, over BEP 9** — there is no other route to them — so
+a magnet alone leaves a page waiting on a tracker connection and a WebRTC
+handshake before it can read a byte, and never gets there at all on a network
+that blocks the trackers. Fetching the metainfo over HTTPS removes that
+dependency entirely. It also saves a conventional client the same round trip.
+
+`magnet=` is the handle that needs nothing of this node, which is the case the
+fragment exists for in the first place. A client with a DHT should prefer it if
+this host is unreachable; a browser, which has neither DHT nor UDP, should reach
+for the `.torrent` first.
+
+For a `/latest/<category>/` URL the `.torrent` handle points at the category too,
+so it redirects to whatever build is current and does not go stale — see the
+caveat below, which it answers for the half of the fragment that a BEP 46 magnet
+otherwise has to.
+
+Both are percent-encoded, since `&` separates them and a magnet is full of them.
+`URLSearchParams` reads the fragment; `get('magnet')` gives back the magnet
+exactly.
+
+> **This changed in 0.30.0.** The fragment used to be a bare `#magnet:?…` with
+> nothing else in it. A reader that took the whole fragment for a magnet needs to
+> read `magnet=` out of it now. The bare form was not kept for the single-handle
+> case, because a fragment whose shape depends on what happened to be available
+> means every reader has to handle both anyway.
 
 ### What a client should do with it
 
-Three paths, each a strict fallback of the one above:
+Four paths, each a fallback of the one above:
 
 1. **The TileJSON URL.** One request, the full document including
    `vector_layers`. Fastest, and what an ordinary client does anyway.
-2. **The `ws=` web seed.** Two HTTP range requests — the header and root
-   directory near the start of the archive, the JSON metadata at the far end —
-   and the TileJSON can be derived from them. Works when this API is down but
-   the file is still on a web server, and it is the same order of cost as (1).
-3. **The swarm.** No HTTP at all. Slowest from cold, because BEP 9 has to
+2. **The `torrent=` metainfo.** One request, and it yields piece hashes, the
+   file name and the web seeds — enough to join and to range-read without any
+   peer having spoken yet. Works when the TileJSON route is down but this host
+   is up, and it is the only one of these a browser can use to start quickly.
+3. **The `ws=` web seed** inside the magnet. Two HTTP range requests — the header
+   and root directory near the start of the archive, the JSON metadata at the far
+   end — and the TileJSON can be derived from them. Works when this API is down
+   but the file is still on a web server.
+4. **The swarm.** No HTTP at all. Slowest from cold, because BEP 9 has to
    deliver the metainfo first, and the only one that survives the server
    disappearing entirely.
 
-Everything those need is in the magnet: `xt` identifies the archive, `dn` names
-the file, `tr` finds peers, `ws` gives the HTTP fallback.
+Everything those need is in the two handles: `xt` identifies the archive, `dn`
+names the file, `tr` finds peers, `ws` gives the HTTP fallback, and `torrent=`
+gives the metainfo without asking the swarm for it.
 
 ### One caveat on `/latest/` URLs
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "pmtiles-swarm",
-  "version": "0.29.0",
+  "version": "0.31.0",
   "description": "BitTorrent distribution for PMTiles map archives: create torrents, watch folders, publish and subscribe to RSS feeds, and seed through qBittorrent or an embedded client",
   "type": "module",
   "main": "src/index.js",

package/src/api.js

@@ -66,7 +66,43 @@ function route(handler) {
  * @returns {import('express').Express} - The configured app.
  */
 /**
- * A TileJSON URL carrying a magnet in its fragment.
+ * Attaches whatever swarm handles exist to a URL, in its fragment.
+ *
+ * Two handles, and they are not a ladder from worse to better — they fail in
+ * different directions:
+ *
+ * - `torrent=` is the metainfo, and needs this host. Not redundant with the URL
+ *   it is attached to: piece hashes reach a browser only from a peer over
+ *   BEP 9, so this is the one handle that works from a page on a network that
+ *   blocks the trackers, and the one that saves every client the metadata round
+ *   trip that a magnet costs.
+ * - `magnet=` needs no host at all, and needs a peer. The better handle for a
+ *   client with a DHT, the weaker one for a browser, which has neither DHT nor
+ *   UDP.
+ *
+ * Ordered by what a client should reach for first when it can fetch.
+ *
+ * Both are percent-encoded, which is a change from the bare `#magnet:?…` this
+ * used to emit: `&` separates the handles now, and a magnet is full of them.
+ * That does break a reader that took the whole fragment for a magnet, and the
+ * alternative -- keeping the bare form when there is only one handle -- is
+ * worse, because it makes the shape of the fragment depend on what happened to
+ * be available, so every reader has to handle both anyway. One shape,
+ * `URLSearchParams` reads it, and a magnet survives the round trip exactly.
+ *
+ * @param {string} url - The URL a style points at.
+ * @param {object} handles - `{torrent, magnet}`, either of which may be absent.
+ * @returns {string} - The URL, with a fragment if there is anything to put in one.
+ */
+function withSwarmHandles(url, { torrent, magnet }) {
+  const parts = [];
+  if (torrent) parts.push(`torrent=${encodeURIComponent(torrent)}`);
+  if (magnet) parts.push(`magnet=${encodeURIComponent(magnet)}`);
+  return parts.length > 0 ? `${url}#${parts.join('&')}` : url;
+}
+
+/**
+ * A TileJSON URL carrying the ways into the swarm in its fragment.
  * @param {string} category - Which category.
  * @param {object} newest - Its newest entry.
  * @param {string} base - Public base URL.
@@ -90,7 +126,14 @@ function styleUrlFor(category, newest, base) {
         webSeeds: newest.webSeeds,
       })
     : newest?.magnet;
-  return magnet ? `${url}#${magnet}` : url;
+  // Category-scoped rather than the newest build's immutable URL, for the same
+  // reason the TileJSON in front of it is: it redirects to whatever is current,
+  // so a style holding this keeps working across rebuilds. The magnet beside it
+  // only manages that when the node publishes a BEP 46 key; this handle manages
+  // it always, which makes it the more durable half of the pair as well as the
+  // faster one.
+  const torrent = `${base}/latest/${category}/archive.torrent`;
+  return withSwarmHandles(url, { torrent, magnet });
 }
 
 export function createApp({
@@ -997,6 +1040,21 @@ export function createApp({
     }),
   );
 
+  // When the record and the disk disagree, this is the only thing that goes and
+  // looks. Answers as soon as the check is under way -- hashing a planet build
+  // is tens of minutes, and no request should be held open for it.
+  app.post(
+    '/api/torrents/:infoHash/recheck',
+    route(async (req, res) => {
+      try {
+        const result = await library.recheck(req.params.infoHash);
+        res.status(202).json(result);
+      } catch (error) {
+        res.status(error.status ?? 500).json({ error: error.message });
+      }
+    }),
+  );
+
   // Joining defaults to cache, deliberately. This is how that is changed
   // afterwards, without re-adding the archive by hand.
   app.patch(
@@ -1879,11 +1937,12 @@ export function createApp({
           servable,
           endpoints: {
             tileJson: servable ? `${base}/latest/${category}/tiles.json` : null,
-            // The same URL with a magnet in the fragment, which is what a
-            // style should carry. A fragment is never sent in a request, so
-            // an ordinary client fetches the TileJSON and ignores it, while
-            // a swarm-aware one has the magnet before it makes any call --
-            // and can therefore still start when this server cannot answer.
+            // The same URL with the ways into the swarm in the fragment,
+            // which is what a style should carry. A fragment is never sent in
+            // a request, so an ordinary client fetches the TileJSON and
+            // ignores it, while a swarm-aware one has somewhere to join before
+            // it makes any call -- and can therefore still start when this
+            // server cannot answer.
             //
             // The mutable magnet where there is one, because a category is
             // precisely where an infohash goes stale: it names the category

package/src/engines/composite.js

@@ -549,6 +549,50 @@ export class CompositeEngine {
     return this.#primary.resume ? this.#primary.resume(infoHash) : false;
   }
 
+  /**
+   * Hashes what is on disk again, on every engine that can.
+   *
+   * Each engine keeps its own belief about the same file, so a stale one on the
+   * secondary is the same fault as a stale one on the primary -- it is why a
+   * browser peer would find nothing while the primary seeds happily. Both are
+   * asked.
+   *
+   * They hash concurrently, which sounds worse than it is: the second pass over
+   * a file the first just read is mostly page cache, and a recheck is a rare
+   * thing somebody asked for rather than something that runs on a timer.
+   *
+   * An engine without the operation is skipped rather than worked around. The
+   * re-add that WebTorrent needs instead is the library's to do, because it
+   * takes the catalog entry -- see Library.recheck.
+   * @param {string} infoHash - The archive to verify.
+   * @returns {Promise<object>} - The primary's answer, plus one row per engine.
+   */
+  async recheck(infoHash) {
+    const engines = [];
+    for (const engine of [this.#primary, ...this.#secondaries]) {
+      if (!engine.recheck) {
+        engines.push({ engine: engine.name, rechecking: false, skipped: true });
+        continue;
+      }
+      try {
+        const result = await engine.recheck(infoHash);
+        engines.push({ engine: engine.name, ...result });
+      } catch (error) {
+        // A secondary that cannot verify must not fail the primary's check.
+        // The primary holds the data the tiles are served from.
+        engines.push({
+          engine: engine.name,
+          rechecking: false,
+          error: error.message,
+        });
+      }
+    }
+
+    const primary = engines[0];
+    if (primary.error) throw new Error(primary.error);
+    return { ...primary, engines };
+  }
+
   /**
    * Tells every engine about a web seed.
    * @param {string} infoHash - The archive.

package/src/engines/libtorrent.js

@@ -303,6 +303,33 @@ export class LibtorrentEngine {
     }
   }
 
+  /**
+   * Hashes what is on disk again and believes the result over the record.
+   *
+   * Returns as soon as the check is under way, not when it finishes: a planet
+   * archive is tens of minutes of disk. The torrent reports state `checking`
+   * while it runs, with progress as the fraction hashed.
+   * @param {string} infoHash - The archive to verify.
+   * @returns {Promise<object>} - `{rechecking, wasPaused}`.
+   */
+  async recheck(infoHash) {
+    try {
+      return await this.#call('recheck', { infoHash });
+    } catch (error) {
+      // An older sidecar answers "unknown op", which is true and useless: it
+      // reads as a bug in the request rather than as a package that needs
+      // updating. Said plainly instead, because this is a button somebody just
+      // pressed and the next thing they do depends on which it is.
+      if (/unknown op/i.test(error.message)) {
+        throw new Error(
+          'this sidecar cannot recheck; pmtiles-torrent 0.5.1 or newer is needed',
+          { cause: error },
+        );
+      }
+      throw error;
+    }
+  }
+
   async list() {
     // A node that is shutting down still has a console polling it and a sweep
     // or two in flight. Answering "the sidecar exited" to each of them fills

package/src/engines/qbittorrent.js

@@ -211,6 +211,20 @@ export class QBittorrentEngine {
     await this.#request('/api/v2/torrents/delete', { method: 'POST', body });
   }
 
+  /**
+   * Hashes what is on disk again and believes the result over the record.
+   *
+   * qBittorrent starts the check and answers immediately, the same as the
+   * libtorrent engine does -- which is the same library underneath.
+   * @param {string} infoHash - The archive to verify.
+   * @returns {Promise<object>} - `{rechecking: true}`.
+   */
+  async recheck(infoHash) {
+    const body = new URLSearchParams({ hashes: infoHash.toLowerCase() });
+    await this.#request('/api/v2/torrents/recheck', { method: 'POST', body });
+    return { rechecking: true };
+  }
+
   /**
    * Lists every torrent qBittorrent holds.
    * @returns {Promise<import('./types.js').TorrentStatus[]>} - Normalised statuses.

package/src/engines/types.js

@@ -60,6 +60,7 @@
  * @property {(filePath: string, options?: object) => Promise<object>} [createTorrent] - Builds a torrent from a local file, where the engine can do it better than the default — libtorrent produces hybrid v1+v2, which create-torrent cannot.
  * @property {(infoHash: string) => Promise<object[]>} [trackerStatus] - Per-tracker announce results, where the engine keeps them.
  * @property {(infoHash: string) => Promise<Uint8Array | null>} [metadata] - The torrent's metainfo once known, so an archive joined by magnet can be written down rather than re-fetched over BEP 9 on every start.
+ * @property {(infoHash: string) => Promise<object>} [recheck] - Hash what is on disk again and believe the result over the stored state -- resume data, or the `seedOnly` claim made when the torrent was added. Returns once the check is under way, not when it finishes: a planet archive is tens of minutes of disk. An engine without this is rechecked by re-adding it instead, which the library does.
  * @property {() => Promise<object | null>} [reachability] - Whether peers can open a connection to this node, or only the reverse. Three states: `open` (something has connected inward), `unproven` (listening, but nothing ever has) and `offline` (not listening at all). The middle is deliberately not called firewalled -- on a node with no peers, blocked and untried are the same observation.
  * @property {() => Promise<void>} destroy - Releases resources.
  */

package/src/library.js

@@ -1657,6 +1657,68 @@ export class Library {
     return this.#catalog.put({ infoHash, paused: true });
   }
 
+  /**
+   * Hashes what is on disk again and believes the result over the record.
+   *
+   * Every other answer about how much of an archive is here comes from
+   * something written down earlier -- the catalog's `complete` flag, resume
+   * data, the `seedOnly` claim made when it was added. When one of those is
+   * wrong there is no path back on its own: an archive built here whose entry
+   * says `complete: false` is re-added without `seedOnly`, so the engine goes
+   * looking for bytes that are already under its nose and sits at 0% beside a
+   * finished file. This is the way out.
+   *
+   * Nothing is written to the catalog here. The check runs for as long as it
+   * takes to hash the archive -- tens of minutes for a planet build -- so the
+   * answer arrives long after this returns, and it arrives where it always
+   * does: the engine's own progress, which the completion sweep already reads
+   * and acts on. Recording a guess now would only have to be corrected later.
+   *
+   * Note what this cannot fix on its own. The sweep promotes, it never demotes,
+   * so a recheck that finds *less* than the record claims shows the truth in
+   * the console but leaves `complete: true` in place. That is deliberate: a
+   * torrent reports progress below 1 for perfectly ordinary reasons while it is
+   * checking, and demoting on that would strip the finished name off an archive
+   * that is merely being verified.
+   * @param {string} infoHash - The archive to verify.
+   * @returns {Promise<object>} - `{rechecking, method}`.
+   */
+  async recheck(infoHash) {
+    const entry = this.#catalog.get(infoHash);
+    if (!entry) {
+      const error = new Error('unknown archive');
+      error.status = 404;
+      throw error;
+    }
+
+    if (this.#engine.recheck) {
+      const result = await this.#engine.recheck(infoHash);
+      return { ...result, rechecking: true, method: 'recheck' };
+    }
+
+    // WebTorrent has no recheck at all: it verifies on add and never again. So
+    // the only way to make it look is to make it add the torrent again, with
+    // the claim that the data is already there withheld -- `seedOnly` is
+    // precisely "do not verify this", and it is computed from `complete`, so a
+    // copy of the entry saying otherwise is what turns the check on.
+    //
+    // Nothing is deleted and the catalog is not touched; this re-adds the same
+    // torrent against the same files. It is a slower and cruder mechanism than
+    // force_recheck, and it is reported as a different one rather than dressed
+    // up as the same thing.
+    if (entry.paused) {
+      const error = new Error(
+        'this engine rechecks by re-adding the archive, which a paused ' +
+          'archive cannot do. Resume it first.',
+      );
+      error.status = 409;
+      throw error;
+    }
+    await this.#engine.remove(infoHash, { deleteData: false }).catch(() => {});
+    await this.#readd({ ...entry, complete: false });
+    return { rechecking: true, method: 'readd' };
+  }
+
   /**
    * Starts offering a paused archive again.
    * @param {string} infoHash - The archive.

package/src/web/index.html

@@ -749,11 +749,14 @@
       const pct = (n) => `${Math.round((n ?? 0) * 100)}%`;
 
       /**
-       * When an archive was added, as something short enough for a column.
+       * When an archive was added: the same date and time on every row.
        *
-       * A date on its own for anything older than today, and a time for
-       * today: the question a list answers is "which of these is recent", and
-       * a full timestamp on every row is harder to scan than either.
+       * It used to shorten today's rows to a time and older ones to a date,
+       * on the theory that a column is easier to scan that way. It is not:
+       * a column of `8/13/2026` with an `08:14 AM` in the middle of it reads
+       * as two different quantities, and the eye has to stop and work out
+       * which. Seconds are dropped rather than the date, since nothing here
+       * is sorted finely enough for them to matter; the title keeps them.
        * @param {string} value - ISO timestamp.
        * @returns {string} - Markup for the cell.
        */
@@ -761,12 +764,15 @@
         if (!value) return '<span class="sub">—</span>';
         const when = new Date(value);
         if (Number.isNaN(when.getTime())) return '<span class="sub">—</span>';
-        const today = new Date().toDateString() === when.toDateString();
-        return `<span title="${escapeHtml(when.toLocaleString())}">${
-          today
-            ? when.toLocaleTimeString([], { hour: '2-digit', minute: '2-digit' })
-            : when.toLocaleDateString()
-        }</span>`;
+        return `<span title="${escapeHtml(when.toLocaleString())}">${escapeHtml(
+          when.toLocaleString([], {
+            year: 'numeric',
+            month: 'numeric',
+            day: 'numeric',
+            hour: '2-digit',
+            minute: '2-digit',
+          }),
+        )}</span>`;
       };
 
       // The same three questions the public page asks, against an admin row.
@@ -1563,7 +1569,7 @@
             <button id="copy-magnet">Copy magnet</button>
             <button id="copy-hash">Copy infohash</button>
             ${servable ? `<button id="copy-tilejson">Copy TileJSON URL</button>
-            <button id="copy-tilejson-swarm" title="The same URL with the magnet in the fragment. A fragment is never sent to the server, so ordinary clients fetch the TileJSON exactly as before; a torrent-aware client reads the magnet from it and can still join the swarm when this server is unreachable.">Copy TileJSON URL + magnet</button>
+            <button id="copy-tilejson-swarm" title="The same URL with the .torrent and the magnet in the fragment. A fragment is never sent to the server, so ordinary clients fetch the TileJSON exactly as before; a torrent-aware client reads them from it, and a browser can fetch the metainfo over HTTP instead of waiting on a peer for it.">Copy TileJSON URL + swarm</button>
             <a href="${tileJson}" target="_blank" rel="noreferrer"><button type="button">Open TileJSON</button></a>
             <a href="${base}/archives/${entry.infoHash}/preview" target="_blank" rel="noreferrer"><button type="button">${
               summary.format === 'pbf' ? 'Inspect' : 'Preview'
@@ -1635,6 +1641,7 @@
             <button id="set-location">Set location…</button>
             <button id="add-seed">Add web seed</button>
             <button id="clear-cache" ${mode === 'cache' ? '' : 'disabled title="only cache-mode archives have a cache to clear"'}>Clear cache</button>
+            <button id="recheck" title="Hash the files on disk again and believe the result. For an archive that reads 0% next to a file that is plainly there, or one that claims to be complete and is not — every other figure comes from something written down earlier, and this is the only thing that goes and looks.">Recheck files</button>
             <button id="pause">${entry.paused ? 'Resume' : 'Pause'}</button>
             <button id="remove" class="danger">Remove</button>
           </div>
@@ -1835,20 +1842,30 @@
         $('copy-magnet').onclick = () => copy(entry.magnet ?? '', 'Magnet link');
         if (servable) {
           $('copy-tilejson').onclick = () => copy(tileJson, 'TileJSON URL');
-          // The magnet rides in the fragment, which is client-side only and
+          // The handles ride in the fragment, which is client-side only and
           // never sent in the request. So the same string serves both: an
           // ordinary client fetches the TileJSON and ignores the fragment, and
-          // a torrent-aware one reads the magnet before making any call at all
-          // — which is what lets it start when this server is down.
+          // a torrent-aware one has somewhere to join before making any call at
+          // all — which is what lets it start when this server is down.
           //
-          // A magnet is legal in a fragment unencoded (RFC 3986 allows ?, &, =
-          // and : there), and leaving it readable matters for something people
-          // paste into a style file by hand.
-          $('copy-tilejson-swarm').onclick = () =>
+          // The .torrent URL is there because a browser cannot obtain piece
+          // hashes any other way: they come only from a peer over BEP 9, so a
+          // magnet alone leaves a page waiting on a WebRTC handshake before it
+          // can read a byte. Same fragment, same string, both handles.
+          //
+          // A lone magnet keeps the bare unencoded form — legal in a fragment
+          // (RFC 3986 allows ?, &, = and : there) and readable, which matters
+          // for something people paste into a style file by hand.
+          $('copy-tilejson-swarm').onclick = () => {
+            const parts = [`torrent=${encodeURIComponent(torrentUrl)}`];
+            if (entry.magnet) parts.push(`magnet=${encodeURIComponent(entry.magnet)}`);
             copy(
-              entry.magnet ? `${tileJson}#${entry.magnet}` : tileJson,
-              entry.magnet ? 'TileJSON URL with magnet' : 'TileJSON URL (no magnet on this archive)',
+              `${tileJson}#${parts.join('&')}`,
+              entry.magnet
+                ? 'TileJSON URL with torrent and magnet'
+                : 'TileJSON URL with torrent (no magnet on this archive)',
             );
+          };
         }
         $('copy-hash').onclick = () => copy(entry.infoHash, 'Infohash');
 
@@ -1933,6 +1950,38 @@
           }
         };
 
+        $('recheck').onclick = async () => {
+          if (
+            !window.confirm(
+              `Recheck ${entry.name}.
+
+Every piece is hashed against the ` +
+                'torrent, which for a large archive is minutes to tens of ' +
+                'minutes of disk. Nothing is deleted, and the archive keeps ' +
+                'serving whatever it can while the check runs.',
+            )
+          ) {
+            return;
+          }
+          try {
+            const result = await api(`/api/torrents/${infoHash}/recheck`, {
+              method: 'POST',
+            });
+            // Said rather than left to be inferred: this returns as soon as the
+            // check is under way, and the progress bar going backwards for the
+            // next twenty minutes is the operation working, not a fault.
+            toast(
+              result.method === 'readd'
+                ? 'rechecking — this engine verifies by re-adding, so it will start from 0%'
+                : 'rechecking — progress shows the fraction hashed',
+            );
+            refresh();
+            renderDetail(infoHash);
+          } catch (error) {
+            toast(error.message);
+          }
+        };
+
         $('pause').onclick = async () => {
           const action = entry.paused ? 'resume' : 'pause';
           try {
@@ -2572,12 +2621,15 @@
                                     url.length > 96 ? `${url.slice(0, 96)}…` : url,
                                   )}</code>${
                                     label === 'For a style'
-                                      ? `<div class="sub">the magnet rides in the fragment, which is
-                                         never sent to the server — an ordinary client fetches the
-                                         TileJSON and ignores it, a swarm-aware one reads it before
-                                         making any call${
-                                           url.includes('xs=urn:btpk')
-                                             ? ' and follows the category rather than this build'
+                                      ? `<div class="sub">the .torrent URL and the magnet ride in the
+                                         fragment, which is never sent to the server — an ordinary
+                                         client fetches the TileJSON and ignores them, a swarm-aware
+                                         one reads them before making any call${
+                                           // Matched unanchored because the magnet is
+                                           // percent-encoded in the fragment now that it
+                                           // shares one with the .torrent URL.
+                                           url.includes('btpk')
+                                             ? ', and both follow the category rather than this build'
                                              : ''
                                          }</div>`
                                       : ''

package/src/web/public.html

@@ -401,9 +401,9 @@
           card.append(links);
 
           const urls = el('div', 'urls');
-          // The one worth copying: the TileJSON URL with the magnet in its
-          // fragment, which a plain client fetches over HTTP and a swarm-aware
-          // one joins directly.
+          // The one worth copying: the TileJSON URL with the .torrent URL and
+          // the magnet in its fragment, which a plain client fetches over HTTP
+          // and a swarm-aware one joins directly.
           if (ends.styleUrl) {
             urls.append(
               urlRow('style URL', new URL(ends.styleUrl, location.href).href),