pmtiles-swarm 0.2.0 β†’ 0.3.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -7,6 +7,732 @@
7
7
  ### 🐞 Bug fixes
8
8
  - _...Add new stuff here..._
9
9
 
10
+ ## 0.3.1
11
+
12
+ Depends on pmtiles-torrent 0.3.0, which is what carries the resume-data fix below to an installed
13
+ copy β€” 0.3.0 of this package shipped against a sidecar that could not find its own resume data.
14
+
15
+ ### ✨ Features and improvements
16
+ - **The sample configuration ships with the package**, so an installed copy has one to copy from
17
+ rather than only the repository β€” which is the one place someone installing from npm has not got.
18
+ - **The `allowScripts` warning from npm 11.17 is explained.** npm is moving dependency install
19
+ scripts behind an allowlist; today it warns and still runs them, so an install that prints it is
20
+ fine. One of those scripts matters β€” `node-datachannel` downloads the WebRTC binary WebTorrent
21
+ needs, which is not in the published tarball β€” so the documentation gives a one-line check that
22
+ it landed, and says to leave `--strict-allow-scripts` alone, since in testing it blocked approved
23
+ scripts as well as unapproved ones.
24
+ - **The service documentation installs into the account's own directory, not globally.**
25
+ `sudo npm install -g` fails on some machines: a WebTorrent dependency runs `npx only-allow pnpm`
26
+ as a preinstall step, and under `sudo` that npx cannot write root's cache. Installing as the
27
+ service account avoids root's cache entirely, keeps the version pinned per service, and makes
28
+ upgrading one command. `--ignore-scripts` is explicitly not the answer β€” `node-datachannel`
29
+ fetches its prebuilt binary in an install script, and without it WebTorrent cannot do WebRTC,
30
+ which is the only reason to run it alongside libtorrent.
31
+ - **Running as a systemd service is documented**, with a unit file, and now with the account setup:
32
+ creating a dedicated `pmtiles-swarm` system user and group, the two directories, and where Node
33
+ and the package go. Both directories have to be writable by the service, including the one under
34
+ `/etc` β€” minting a token or pressing Save rewrites the configuration, so a root-owned file the
35
+ service can only read loses tokens on restart. Two things in it are not
36
+ preferences: `Restart=always` is required rather than optional, because the console's *Save &
37
+ Restart* detects the supervisor and exits 0 expecting to be brought back β€” under
38
+ `Restart=on-failure` the first use of it stops the node and leaves a unit reporting success. And
39
+ the `ExecStop=/bin/kill -15 $MAINPID` line commonly copied between units should be omitted, since
40
+ systemd already sends SIGTERM and the node installs its handlers before it begins work.
41
+
42
+ ### 🐞 Bug fixes
43
+ - **A restart no longer re-hashes every archive.** Resume data was being written and never found β€”
44
+ the lookup used an infohash no caller supplied, and `add_torrent_params.info_hashes` reads as
45
+ forty zeros for a torrent added from a `.torrent` file, so keying it off the torrent alone would
46
+ not have helped either. Fixed in pmtiles-torrent 0.3.0; on a 512 MiB archive the difference
47
+ measured 1.21s with a full re-hash against 0.02s with none, and it scales with the archive.
48
+ Resume data is also written every `resumeSaveIntervalSeconds` (five minutes by default) rather
49
+ than only at shutdown, so a kill or a power cut costs the last few minutes instead of everything.
50
+ - **Every path in the configuration resolves against the configuration file.** `dataDir`,
51
+ `savePath`, `cacheSavePath` and watched folders already did; `locations[].path` and
52
+ `libtorrent.resumeDir` were left relative, which means relative to the working directory. Started
53
+ by hand from the repository the two agree, so it never showed β€” but a service does not run from
54
+ the directory its config lives in, and under systemd the working directory defaults to `/`, so
55
+ `./data/resume` became `/data/resume`: somewhere the unit almost certainly cannot write, for a
56
+ reason nothing in the config hints at.
57
+
58
+ ## 0.3.0
59
+ ### ✨ Features and improvements
60
+ - **The peers column distinguishes who is connected from what the swarm holds.** `0 / 2` on a
61
+ complete, seeding archive is correct β€” the counts are remote clients only, and a client is never
62
+ its own peer β€” but it reads like a fault. The tracker's own totals now follow in parentheses, in
63
+ qBittorrent's notation, and the cell explains itself on hover. Nothing is shown until a tracker
64
+ has actually answered, since claiming an empty swarm on no information is worse than saying
65
+ nothing.
66
+ - **An archive fetched from a URL is filed under its infohash like every other.** It could not be
67
+ before: the infohash is computed from the bytes, which are the thing still arriving, so a
68
+ scheduled download landed in the root of the save path while everything else sat under its own
69
+ directory β€” reintroducing exactly the collision that layout exists to prevent, since two sources
70
+ publishing `planet.pmtiles` would write into one file. It now downloads into a randomly named
71
+ directory under `<savePath>/.incoming/` and is moved into place once the torrent has been hashed.
72
+ The move is a rename within one filesystem, so it is instant whatever the archive weighs, and the
73
+ random name keeps two in-flight downloads of the same filename apart. A download interrupted by a
74
+ kill leaves its directory behind and the next start clears it.
75
+ - **A watched location can keep only the newest few builds.** `keep` on a source, and **Builds to
76
+ keep** in the console. Each build is a whole archive, so a daily 137 GB planet build kept for
77
+ ever fills any disk within the week. It deletes the data of what it retires, so it is off unless
78
+ set, and it only ever touches archives that same named source imported β€” anything added by hand,
79
+ adopted from a client, or taken from a peer is never considered. Sources can also carry their own
80
+ `seeding` limit, since a daily build that has done its share deserves different treatment from
81
+ the only copy of something.
82
+ - **"Newest" now means the newest build, not the most recently added archive.** The two disagree
83
+ and can be opposite: a poll takes candidates newest first, so importing several at once gives the
84
+ newest build the *earliest* arrival time. `/latest/<category>` and the category feeds ordered by
85
+ arrival, which would have served the oldest of a batch, and a retention policy ordered the same
86
+ way would have deleted the newest. Entries record the date of the build they are, and one
87
+ comparison in the catalog answers it for both.
88
+ - **The global seeding limit has real fields.** It was editable only as a raw JSON textarea among
89
+ every other object setting, which is not a way to ask someone for a ratio. It now has the same
90
+ shape as the per-archive dialog that already existed.
91
+ - **A download that stops is resumed, not restarted.** A planet archive is hours of transfer and a
92
+ connection that drops partway is ordinary; until now that threw away everything transferred and
93
+ began again, repeatedly. Each attempt now continues from the bytes already on disk with an HTTP
94
+ range request β€” `fetchAttempts` and `fetchRetrySeconds` β€” so a drop costs the retry delay rather
95
+ than 49 GB. Appending only happens when it is provably safe: the response must be a 206 (a server
96
+ that ignores `Range` answers 200 with the whole file, and appending that gives a file that is
97
+ part duplicate), the ETag or Last-Modified must be unchanged (resuming across a new build splices
98
+ the head of one onto the tail of another), and `Content-Range` must begin where it was asked to.
99
+ Any of those failing restarts the download, as does a server offering no validator at all β€”
100
+ fetching an archive twice is expensive, but publishing a torrent for bytes that never existed
101
+ anywhere hashes perfectly well here and fails for every peer that tries it.
102
+ - **Downloads that have no torrent yet are visible.** An archive added from a URL is fetched whole
103
+ before there is anything to hash a torrent out of, so until that finishes there is no catalog
104
+ entry and nothing in the table β€” for a planet build, hours in which a watched location looks like
105
+ it silently did nothing. They now appear under the archive list with progress and a cancel
106
+ button. `/api/adds` reports bytes and totals rather than bare URLs.
107
+ - **A watched web location can say whether its URL is published as a web seed.** `webSeed` on a
108
+ source, and **Use URL as web seed** on each row in the console. The behaviour was already the
109
+ right default β€” the origin is a valid web seed for exactly those bytes, and publishing it is the
110
+ single biggest difference to a cold start β€” but it was not settable per source, and there are two
111
+ reasons to change it. An upstream that deletes old builds leaves a URL that outlives the file it
112
+ points at, so every peer that tries it fails; and where the archive also sits on public storage
113
+ under another address, `webSeeds` names that instead and keeps the fetch URL private. A URL that
114
+ appears to carry credentials is still never published unless `webSeed: true` says so explicitly,
115
+ because a torrent goes to the swarm and cannot be recalled.
116
+ - **Piece maps.** A **Pieces** tab showing what this node holds, how rare each piece is across the
117
+ swarm, and what each connected peer has β€” plus `firstPiece` and `pieceCount` per file on
118
+ `/content`, which need no engine at all, since a torrent is one byte stream cut into equal pieces
119
+ and a file's offset already says which it occupies. Worth more here than in an ordinary client: a
120
+ cache-mode archive holds a scatter of pieces on purpose, so the bar is a picture of what has been
121
+ *viewed* rather than a progress indicator. Maps arrive bucketed to the width they will be drawn
122
+ at, each reduced for the question its bar answers β€” held counts only when every piece in a column
123
+ is (or a 60%-complete archive paints as almost solid), availability takes the *rarest* (one piece
124
+ nobody has is the answer to "can this be completed"), and a peer's map takes *any* (a peer
125
+ holding part of a column can still serve it). Supported by libtorrent **and WebTorrent**, whose
126
+ `torrent.bitfield` and per-wire `peerPieces` carry the same information; qBittorrent's API has
127
+ piece states but neither availability nor per-peer maps, so it is refused rather than half-drawn.
128
+ - **Speed limits, with a schedule.** Two sets of global limits and a window that swaps them,
129
+ modelled on qBittorrent: `speed.uploadLimit` / `downloadLimit`, `speed.alternative`, and
130
+ `speed.schedule` taking `from`, `to` and `days` (`everyday`, `weekdays`, `weekends`, or weekday
131
+ numbers). A window whose end is before its start wraps past midnight, so `22:00`–`06:00` is one
132
+ overnight window rather than an empty one, and `days` picks the night it opens. The console has
133
+ the settings and a header switch that forces either set, handing control back to the schedule the
134
+ next time the window itself changes β€” so forcing "slow" at lunchtime does not leave the node
135
+ throttled tomorrow. Applied live, enforced by whichever engines can throttle, and applied whole
136
+ to each rather than divided between them, since they share one uplink.
137
+ - **A listen failure is reported rather than thrown.** The two `server.on('error')` registrations
138
+ had been spliced into the middle of the watch-folder reloader, so nothing was listening for the
139
+ event at startup: a port taken between the pre-flight check and the actual bind produced a raw
140
+ stack trace instead of the sentence explaining it, and every settings reload added two more
141
+ listeners.
142
+ - **libtorrent's network settings are configurable.** The sidecar has always accepted `upnp`,
143
+ `natpmp`, `dht`, `lsd`, `uploadLimit` and `downloadLimit`, and nothing passed them β€” so a node
144
+ could not decline UPnP however the config was written. That is the wrong default on a network
145
+ where port forwards are made by hand: the router has UPnP off deliberately and the client fails
146
+ at it quietly on every start. Unset keys still take libtorrent's own defaults.
147
+ - **An archive that is not whole yet is named so.** It downloads as
148
+ `planet.pmtiles.incomplete` and is renamed the instant it finishes. These files get published:
149
+ a web seed URL is predictable and goes out before the file exists, so an unmarked partial in a
150
+ served directory is a URL that answers with half an archive, and every peer that tries it fails
151
+ hash verification. Now it 404s until the file is real. The rename is inside one directory, so it
152
+ is atomic and instant at any size β€” where moving between directories is instant only when they
153
+ share a filesystem, and otherwise copies the whole archive. Remote downloads are marked the same
154
+ way, qBittorrent's own `.!qB` preference is turned on rather than overridden, and
155
+ `incompleteSuffix: ""` switches the whole thing off.
156
+ - **`cacheSavePath` is now off by default.** It existed to tell whole archives from partial ones on
157
+ disk, which the name above does better; it stays as a placement choice for putting cache pieces
158
+ on faster disk. Archives already in a catalog keep the save path they were added with.
159
+ - **A Categories screen in the console**, listing every tag with the endpoints that resolve to its
160
+ newest build β€” TileJSON, `.torrent`, magnet, feed and latest-only feed β€” each copyable. Backed by
161
+ a new `GET /api/categories`. A category whose newest archive is not PMTiles gets everything
162
+ except the tile endpoint.
163
+ - **Categories can be changed after an archive is added**, from its detail panel or at
164
+ `PATCH /api/torrents/{infohash}/categories` (whole list, or `add`/`remove` one at a time). They
165
+ could only be set at the moment of adding, which is the wrong time to have to know: a build
166
+ becomes `weekly` once there is a second one, and an archive is marked for sharing long after it
167
+ arrives.
168
+ - **Monitored folders and watched web locations are editable in Settings**, as tables rather than
169
+ a textarea full of JSON β€” the shape a torrent client gives a grid for. Folders take categories, a
170
+ save location, a publish directory and a web seed base; web locations take a URL template or a
171
+ directory to list.
172
+ - **The date in a watched URL is built by clicking, not by remembering.** Paste the URL of a
173
+ recent build, select the date in it and click a token β€” the token replaces what is selected.
174
+ A `{...}` group is now read as a date *pattern* rather than matched against a fixed list of
175
+ spellings, so it can say whatever the upstream says: `{M}-{D}-{YY}` gives `8-7-26`,
176
+ `{DD.MM.YYYY}` gives `07.08.2026`, `{YY}` gives `26`. Run length decides padding β€” `MM` is
177
+ padded, `M` is not β€” and case is ignored, since using case for padding as well would make `{m}`
178
+ and `{M}` differ with nothing to see. A group that is not a date is left exactly as found, so a
179
+ URL containing `{id}` is not quietly rewritten. Every spelling that worked before still does. Day offset and look-back are columns of their own
180
+ (protomaps publishes yesterday's build, so it wants `-1`), and Preview refuses to run on a URL
181
+ that still has a fixed date in it, since that would ask for the same build forever.
182
+ - **`onAdded`, beside the existing `onComplete`** β€” the same pair a torrent client offers, and
183
+ different moments: an archive joined in cache mode is added and will never be complete, while one
184
+ built here is both at once. Both are now shown in Settings under **Run external program**, with
185
+ the full placeholder list, laid out the way a client lays it out.
186
+ - **`allowHooksFromApi`.** The hooks stayed config-file-only because a token that manages torrents
187
+ becoming one that runs arbitrary commands as the service user is a large step to take by
188
+ accident β€” but a setting nobody can find is not much safer than one anybody can change, it is
189
+ just harder to use. The panel is read-only until this is set in the config file, where a token
190
+ cannot reach, and says so.
191
+ - **Adopting is a dialog now**, like adding. It lists what an engine holds that this node does not
192
+ yet know about β€” name, size, progress, format β€” and lets you pick, rather than importing
193
+ everything and reading afterwards what it did. Categories can be applied to the lot. It can also
194
+ adopt from **a qBittorrent instance other than the configured engine**, which is what "adopt
195
+ existing" sounded like it did.
196
+ - **Startup refuses to run two nodes over one data directory, and checks its ports first.** The
197
+ port is the symptom people notice; the data directory is the one that costs something, since the
198
+ catalog is rewritten whole by each node and the last writer silently wins. Both are checked
199
+ before an engine is connected or a library restored, and both explain what to change. A lock left
200
+ by a node that was killed rather than stopped is taken over rather than needing to be deleted.
201
+ - **The console and the API can have a port of their own.** `adminPort`, with an optional
202
+ `adminHost`, leaves tiles, TileJSON, `.torrent` files, the feeds, the `latest` endpoints and
203
+ `/api/catalog` on the public port and moves everything else. The public port can then face the
204
+ internet while the admin one is bound to loopback β€” so the thing that can rewrite the
205
+ configuration is unreachable rather than merely guarded, which is a statement a firewall can
206
+ enforce. On the public listener the admin surface answers 404 rather than 403, because a refusal
207
+ confirms there is something behind it. Routing is by the port a request arrived on, never by a
208
+ header, since a header is something the caller controls. The refusal to start unauthenticated now
209
+ reads the admin interface rather than the public one, because tiles on `0.0.0.0` is the point of
210
+ the tiles.
211
+ - **Torrents are created hybrid v1+v2 wherever libtorrent is present** β€” as the primary or
212
+ merely as a secondary, since what matters is that it is there at all. A hybrid is not a
213
+ trade-off: v2 clients gain per-file merkle trees over 16 KiB leaves, which is exactly the shape
214
+ of a tile read, and v1 clients see an ordinary torrent. `torrentFormat` takes `hybrid`, `v1` or
215
+ `v2`, and a node with no libtorrent falls back to v1 rather than failing. Previously every
216
+ torrent was v1 whatever the engine, and the docs said otherwise.
217
+ - **Two engines can run at once.** `secondaryEngines: ["webtorrent"]` beside a libtorrent or
218
+ qBittorrent primary β€” the arrangement the docs had been recommending without any code to do it,
219
+ which until now meant two processes and two catalogues. libtorrent handles the bulk and speaks
220
+ BitTorrent v2; WebTorrent is the only one that can talk to a browser. One rule keeps it safe:
221
+ only the primary writes, so a secondary is handed an archive only once it is complete and never
222
+ in cache mode β€” two clients writing one incomplete file produce a file neither one's bitfield
223
+ describes. Progress and state come from the primary; peers, seeds and speeds are added together.
224
+ A secondary that will not start is a warning, not a failure.
225
+ - **A map preview for every archive**, at `/archives/{infohash}/preview` and behind an
226
+ **Inspect** or **Preview** button in the detail panel. Vector archives get an inspector: each
227
+ declared layer drawn in a colour derived from its name, toggleable, with click-to-see-properties
228
+ on the features under the cursor, using MapLibre's own `@maplibre/maplibre-gl-inspect` β€” it is
229
+ maintained alongside the renderer, so it keeps working across major versions without this having
230
+ to notice. Raster archives get the raster. It is built from the archive's
231
+ own TileJSON, which is already a complete source description β€” nothing is reconstructed. No
232
+ symbol layers and no glyphs, since an archive carries tiles and not fonts. Both libraries are
233
+ ordinary dependencies served out of `node_modules`, the way tileserver-gl does it, so a node on
234
+ an internal network can render its own previews.
235
+ - **Ratio and Expires columns**, so a seeding limit can be seen coming rather than noticed
236
+ afterwards. Expires counts down a time limit β€” `42d 1h` β€” and says `∞` where nothing applies,
237
+ with the reason on hover: a cache-mode archive, or one told to seed forever. A ratio target is
238
+ reported as progress towards a number rather than as a duration, because how long it takes
239
+ depends on how fast peers happen to be downloading, and the ratio is coloured as it approaches
240
+ the point where it would remove the archive. The detail panel carries the same countdown beside
241
+ the limit in effect.
242
+ - **A move checks there is room first**, before the engine is disturbed β€” running out of disk
243
+ halfway through several hundred gigabytes means an hour spent, a partial file to clean up and an
244
+ archive to put back. Only when it will actually be a copy: a move within one filesystem is a
245
+ rename and needs no free space at all, so checking unconditionally would refuse moves that would
246
+ have worked. A filesystem that will not report its free space is gone ahead with rather than
247
+ refused. Free space is shown beside each save location in the picker, including for a directory
248
+ that has not been created yet.
249
+ - **An archive's data can be moved after the fact** β€” **Set location…** in its detail panel, or
250
+ `PATCH /api/torrents/{infohash}/location`. The engine is told to let go, the file is moved, and
251
+ the torrent handed back pointed at the new path. Within one filesystem that is a rename and
252
+ finishes at once; across two it is a real copy, so it runs in the background and reports
253
+ progress rather than holding a request open for an hour, and the original is removed only after
254
+ the copy has been checked. An unfinished archive moves under the name it actually has, marker
255
+ and all.
256
+ - **`savePathLayout: "infohash"`**, giving each joined archive `<savePath>/<infohash>/` to itself.
257
+ Filenames are not unique β€” two builds of the same map are both `planet.pmtiles` β€” and this is the
258
+ only arrangement in which that can never matter. Flat stays the default, because the collision is
259
+ now refused outright when the second archive is added, and flat is what makes dropping a finished
260
+ archive in before adding its torrent work. Works from a bare magnet, since the infohash is the
261
+ one thing a magnet always carries. Archives created here are unaffected, and web seed URLs are
262
+ built from the published location rather than the save path, so they keep their shape.
263
+ - **Named save locations.** Everything used to land in one place. Name the others under
264
+ `locations` in Settings and they are offered wherever something is added β€” the add dialog, the
265
+ adopt dialog, each monitored folder and each watched web location β€” alongside the default and a
266
+ path given outright. qBittorrent hangs the save path off the category, which cannot work here:
267
+ an archive can carry several categories on purpose, and two of them naming two disks is a
268
+ question with no right answer. So the location is chosen rather than derived. The directory is
269
+ created and checked when it is chosen rather than when the first byte arrives, and a name this
270
+ node does not know is refused with the ones it does, since falling back quietly would put
271
+ several hundred gigabytes somewhere other than where it was asked for.
272
+ - **Most settings no longer need a restart, and there is a button for the ones that do.** Changing
273
+ the watched folders means restarting the watchers, not the node; the same goes for hooks, web
274
+ locations, remote nodes, seeding limits and the completion watcher. Those are applied on Save and
275
+ the console says which subsystem was restarted. What is left genuinely belongs to the process β€”
276
+ the listening socket, the data directory, the torrent client β€” and **Save & restart** appears
277
+ only for those. How the node comes back is detected rather than assumed: under systemd, Docker,
278
+ pm2 or Kubernetes it stops, because exiting is the restart there and a replacement would fight
279
+ over the port; started by hand it starts a replacement itself.
280
+ - **Named access tokens, with roles.** `auth.apiKey` was one credential and one power, so letting
281
+ another node follow this one meant handing over the key that can also delete the library. There
282
+ are now as many tokens as you like, each named, each `peer` (reads the catalogue, feeds, tiles
283
+ and torrent files β€” what a node needs to follow this one) or `admin` (everything). A peer token
284
+ can be narrowed to categories and then sees exactly those and nothing else. Minted in Settings or
285
+ at `POST /api/tokens`, shown once, revoked individually, and each records when it was last used
286
+ so retiring an old one is an informed decision. Only a SHA-256 is stored. The existing `apiKey`
287
+ keeps working and keeps meaning admin.
288
+ - **Adopt can pull from another pmtiles-swarm node**, reading its `/api/catalog` once and letting
289
+ you pick β€” which is not the same as following it, and is the right shape for "give me that one
290
+ build" rather than "take everything it ever publishes". What the peer already knew comes across
291
+ with it: the archive summary, categories, web seeds and checksum. That is what makes it better
292
+ than pasting the magnet, since a joined magnet has no summary until something reads its header
293
+ out of a swarm it has only just joined, and no web seeds at all.
294
+ - **Adopting across machines joins the swarm instead.** An archive whose data this node cannot read
295
+ β€” a client on another host, or a path that is not mounted here β€” used to be unusable, since a
296
+ catalog entry pointing at a file that is not there can never serve a tile. But its infohash is
297
+ right here, and an infohash is all it takes to join the swarm that client is already seeding
298
+ into, so those are joined by magnet as cache or mirror, your choice. Anything readable is still
299
+ adopted where it lies, and neither re-hashed nor re-downloaded.
300
+ - **Remote nodes are editable in Settings**, alongside folders and web locations: feed or catalog
301
+ URL, protocol, whether to take archives as a cache or a mirror, a tag to apply, a name filter, a
302
+ token and the pruning policy. A **Test** button β€” `POST /api/subscriptions/preview` β€” reports
303
+ whether the peer is reachable, which protocol it speaks and how many archives it is offering that
304
+ this node could actually take. A feed that 404s and a token the peer rejects both fail silently
305
+ otherwise: nothing arrives, which looks exactly like a peer with nothing new.
306
+ - **Polling can be finer than an hour.** `everyMinutes` on a watched web location, for somewhere a
307
+ build pipeline writes into rather than a daily planet build. And `pollSeconds` on a monitored
308
+ folder, for network shares: SMB and NFS do not deliver the change notifications a local
309
+ filesystem does, so a watch on one can sit silent forever while files arrive. Off by default,
310
+ because on a local folder it is pure waste.
311
+ - **Each watched location says when to check.** `at: "03:30"` β€” a time of day in UTC, or a list of
312
+ them β€” for an upstream that publishes on a schedule, or `everyHours` for one that publishes
313
+ whenever it is ready. Polling every six hours from whenever the process started found a daily
314
+ build up to six hours late, and those are hours during which nobody could be seeding it. Sources
315
+ naming neither fall back to `sourceCheckIntervalHours` as before. A source that has never run is
316
+ always due, so a daemon that was down over a scheduled time catches up on start.
317
+ - **A source can watch a directory instead of guessing filenames.** `sources[].index` reads a
318
+ listing β€” an HTML autoindex or an S3 `ListBucketResult` β€” filters it and takes the newest match,
319
+ for upstreams whose naming is not predictable enough to write as a template. Only links
320
+ underneath the index URL are followed: a listing is a document from somewhere else, and this node
321
+ is about to download gigabytes from whatever it names and republish the result under its own
322
+ name. `newest` bounds how many are considered and defaults to one.
323
+ - **`POST /api/sources/preview`**, and a Preview button beside each web location, reporting what a
324
+ source would take without taking any of it. A directory URL typed slightly wrong is otherwise
325
+ discovered by watching several hundred gigabytes arrive.
326
+ - Adding a scheduled source no longer needs a restart. The poll timer only started when the list
327
+ was already non-empty, and every pass reads the list fresh.
328
+ - Settings now presents the download options the way a torrent client does: a checkbox for the
329
+ marker, and the separate cache directory as an option that ships off.
330
+ - New `sparse` setting, global with a per-archive override, matching tileserver-gl.
331
+ - Watch folders can move each archive into the directory a web server serves (`publishDir`) and
332
+ advertise that URL as a web seed, rather than assuming the watched folder is already the web
333
+ root.
334
+ - Cache-mode archives can be kept under `cacheSavePath`, separate from mirrors. This began as the
335
+ way to tell whole archives from partial ones on disk; the marker above does that job now, and
336
+ this is a placement choice.
337
+ - **An archive can carry several categories.** A planet build can be both `basemaps` and `weekly`
338
+ without choosing. Feeds match on *any* tag, so it appears in both. Catalogues holding the older
339
+ single `category` string are read as a list of one and normalised on the next write.
340
+ - **Asking for the TileJSON reads the header.** A joined torrent arrives with no summary, because
341
+ at that moment there is nothing to read one from β€” and it used to stay that way, so the archive
342
+ was permanently unusable as a tile endpoint. The header is now read on demand, which for a
343
+ cache-mode archive means pulling the one piece it lives in, and kept once read. A swarm that has
344
+ not found peers yet says so and suggests trying again, rather than refusing outright.
345
+ - **Pause and resume**, in the console and at `POST /api/torrents/{infohash}/pause`. "Not right
346
+ now" is a different intention from "not any more", and remove was the only way to say either.
347
+ - **Mirror or cache is now a choice you can make, and change.** The add dialog offers it when
348
+ joining a magnet or a `.torrent`, and `PATCH /api/torrents/{infohash}/mode` switches an archive
349
+ afterwards β€” with buttons in the detail panel. Nothing already downloaded is discarded in
350
+ either direction: going to mirror keeps what the cache accumulated and fills in the rest.
351
+ - **Tabbed detail per archive**, as a torrent client has: General, Trackers, Peers, HTTP sources
352
+ and Content. Trackers are shown in their tiers, files with piece geometry, comment and creator.
353
+ Panes load when first opened. New `trackers` and `content` endpoints back them.
354
+ - **The console shows where each archive came from** β€” built here, adopted, added by hand, or the
355
+ host of the peer that sent it. Worth showing rather than inferring: an archive taken from a peer
356
+ is one this node seeds and serves under its own name.
357
+ - `prune` gained a `"report"` mode that logs what it would remove and removes nothing, so a new
358
+ peer can be watched before it is trusted. Pruning stays off unless asked for, only ever
359
+ considers archives that peer sent, and never acts on a filtered or partial view.
360
+ - **Optional MD5**, `md5: true` globally or per add, published as `<pmtiles:md5>` in the feed and
361
+ exposed in the API. Not for integrity β€” the torrent already verifies per piece, which is
362
+ stronger β€” but for the quick manual check and for tooling that expects a checksum. Off by
363
+ default because on a local file it costs a second read of the whole archive; where the bytes
364
+ are already streaming past it is free.
365
+ - **Run a command when a download finishes.** `onComplete` closes the loop for a build pipeline:
366
+ subscribe to a feed of source data, let the swarm fetch it, start the job that turns it into
367
+ something worth publishing. Placeholders match a torrent client's, so an existing
368
+ `torrent_finished.sh` keeps working. Command and arguments are separate rather than one shell
369
+ string, so a name with spaces stays one argument. Configurable from the config file only β€”
370
+ never through the API, since a token that manages archives should not also choose what code
371
+ runs as the service user.
372
+ - **A stable URL for the current build.** `/latest/{category}/tiles.json`, plus `archive.torrent`,
373
+ `magnet` and an `.xml` feed of just the newest. A style can point at one and survive every
374
+ rebuild. The tiles it names stay infohash URLs, so they remain immutable and cacheable for a
375
+ year β€” this document is the only mutable thing, and is cached for five minutes.
376
+ - **Seeding limits**, in the shape a torrent client uses: stop at a ratio, or after so long
377
+ seeding a complete copy, then stop, remove, or remove and delete the files. Global by default
378
+ with a per-archive override β€” including "seed forever", which a change to the global rule must
379
+ not undo. Never applies to a cache-mode archive, which holds a few pieces on purpose and has
380
+ not been seeding in the sense a ratio measures.
381
+ - **Trackers are settable wherever a torrent is created** β€” per watch folder, per scheduled
382
+ source, per request β€” with `trackers` replacing the global list and `addTrackers` appending to
383
+ it. Watch folders could not set them at all before. The add dialog shows the defaults and
384
+ offers a field to announce to more.
385
+ - **Adding is a dialog now**, with everything the API could already do: multiple categories picked
386
+ from those in use or typed fresh, keep-or-discard for URL fetches, and whether the source URL is
387
+ published as a web seed β€” plus a list of your own to publish instead of it.
388
+ - **`feedCategories` decides what leaves the node.** Category feeds let a subscriber narrow what
389
+ it takes; they never narrowed what was published, since `/feed.xml` carried the whole catalogue.
390
+ With an allow-list set, only those categories appear in any feed and other category feeds
391
+ answer 404. Untagged archives are excluded, because untagged means unmarked for sharing.
392
+ - A subscription can carry a `token`, and a credential lifts `feedCategories`. One feed then
393
+ serves two audiences: an internal node holding the token syncs the whole catalogue, untagged
394
+ archives included, while the outside world sees only the categories marked for sharing.
395
+ - **Access control.** Tiles, TileJSON and the feed stay public; everything under `/api/` and the
396
+ console are guarded whenever `auth.apiKey`, `auth.password` or `auth.passwordHash` is set. A
397
+ bearer token for scripts, a sign-in form and session cookie for people. Passwords set through
398
+ the settings screen are stored as a scrypt hash, and credentials are redacted from every
399
+ response. Configuring nothing keeps the previous behaviour.
400
+ - The startup line prints an address a browser can open. It previously printed the bind address,
401
+ and `http://0.0.0.0:8090` is rejected outright with `ERR_ADDRESS_INVALID`.
402
+ - The console's own page is public, so its sign-in form can load; only `/api/` is guarded.
403
+ - A node configured with only `auth.apiKey` can still use the console: the token is accepted at
404
+ sign-in and the form asks for a token rather than a password that does not exist. Previously
405
+ the console showed a sign-in form that could never succeed.
406
+ - **A node with no credential now refuses to start on a reachable address**, rather than warning.
407
+ The refusal prints the JSON to paste, into the config file it names β€” or how to create one when
408
+ there is none β€” along with a generated key and the `curl` that uses it. It prints without a
409
+ stack trace, since a configuration refusal is not a crash. Bind to loopback, configure `auth`,
410
+ or set `allowUnauthenticated: true`. See [docs/security.md](docs/security.md).
411
+ - **The web UI is now a real console.** Live-refreshing archive table with progress, peers and
412
+ speeds; a detail panel per archive with disk usage, web seeds and a tile preview; per-archive
413
+ actions for warming, clearing a cache, adding a web seed and removing; export by downloading
414
+ the `.torrent` or copying the magnet, TileJSON URL or infohash; and a settings screen.
415
+ - `GET`/`PATCH /api/config` read and write settings. Anything read per request applies
416
+ immediately; anything bound at startup is written to the file and reported back as needing a
417
+ restart, rather than being accepted and quietly ignored. Credentials are redacted on the way
418
+ out and never overwritten by their own placeholder.
419
+ - `POST /api/torrents/{infohash}/webseeds` adds web seeds to a torrent already in circulation.
420
+ This does not change the infohash β€” `url-list` sits outside the `info` dictionary β€” so magnets
421
+ and peers stay valid, and anything published without a web seed can be given one.
422
+ - `DELETE /api/torrents/{infohash}/cache` reclaims what on-demand reading has accumulated for one
423
+ archive without forgetting the archive, and `GET /api/torrents/{infohash}` reports `diskBytes`.
424
+ Nothing else bounded that disk usage.
425
+
426
+ ### 🐞 Bug fixes
427
+ - **A second request for a URL already being fetched joins the first.** The catalog cannot answer
428
+ that question β€” an entry exists only once the download has finished and the torrent has been
429
+ hashed, so for the hours in between `findBySource` says no and every caller starts its own copy.
430
+ The scheduler was safe by accident, since a poll holds a flag for its whole run, but nothing
431
+ protected `POST /api/torrents {url}` for something a schedule was already fetching: two
432
+ downloads of the same hundred gigabytes, both producing the same infohash, both trying to move
433
+ into the same directory. A failed download is not retained, so one network error does not become
434
+ permanent.
435
+ - **One poll takes one build.** A date-based source imported *every* candidate in its lookback
436
+ window, where a directory-listing source has always capped at `newest` (default 1) for the stated
437
+ reason that each candidate is a whole archive. With a daily 137 GB planet build, `lookbackDays: 3`
438
+ therefore meant 411 GB from a single poll. The same cap now applies, and a candidate that is
439
+ already held stops the scan β€” candidates run newest first, so anything past one on disk is older
440
+ than it, and without stopping lookback walks backwards through history one archive per poll.
441
+ - **`latestLink` works without elevation.** Windows refuses symlinks with EPERM unless the process
442
+ is elevated or the machine is in developer mode, so `latest` was left pointing at nothing. It
443
+ falls back to a hard link, which needs neither and costs no extra space β€” another name for the
444
+ same bytes rather than a copy, which for a 137 GB archive is the point.
445
+ - **A poll that takes nothing says why.** A source asking only for today's date against an upstream
446
+ that publishes at 09:00 does nothing at all between midnight and then β€” and silence there is
447
+ indistinguishable from a broken template, a dead server, or a daemon that is not running. It now
448
+ names how many candidate URLs were not published yet and the first of them, and points at
449
+ `lookbackDays: 0` where that is the reason only one date is ever asked for. Nothing is logged
450
+ when the candidates are simply already held, since that is the normal state of every poll after
451
+ the first.
452
+ - **A watched location no longer restarts its download the moment one finishes.** The last-run time
453
+ was recorded when a poll *began* and never again, so by the time a planet build had been fetched
454
+ the stamp was hours old, `now - lastRun` was far past any interval, and the next tick started the
455
+ whole thing again β€” for ever, on a 72 GB archive. A failed fetch behaved the same way: one that
456
+ died at 35% was retried from zero immediately. The time is now recorded on the way in *and* on
457
+ the way out, so overlap is still prevented and the interval is measured from when the work
458
+ actually ended. The comment there had described this exact behaviour as the thing it was
459
+ avoiding.
460
+ - **`libtorrent` and `feedTitle` are settings the API knows about.** `DEFAULTS` doubles as the
461
+ allow-list, and neither key was in it β€” so a libtorrent node saving anything at all was answered
462
+ `unknown setting: libtorrent`, because the console posts back every key it was given.
463
+ `libtorrent` was even named in `RESTART_REQUIRED`: known everywhere except where it was checked.
464
+ - **A refused save now changes nothing.** Validation happened inside the loop that assigned, so a
465
+ save containing one bad key applied every key before it and then threw, leaving the running node
466
+ changed and the file on disk not. A watched location added that way started polling immediately
467
+ and vanished from the console. Every key is checked before any is applied.
468
+ - **One loaded config no longer leaks into the next.** `merge()` spread the defaults, so a nested
469
+ object in a loaded config *was* the one in `DEFAULTS` whenever the file did not mention it β€” and
470
+ load writes the resolved save path back into it. One process loads one config, so this only
471
+ surfaced in tests, but it made the defaults mutable at runtime.
472
+ - **Settings save again.** A setting that may only be set in the config file was refused on the
473
+ key's *presence* rather than on a change, and the console renders every setting it knows about
474
+ and posts the lot β€” so `allowHooksFromApi` rode along with every save and failed all of them,
475
+ including saves that touched nothing but a watch folder. The error even named a way out that
476
+ could not work: setting `allowHooksFromApi: true` unlocks the hooks, but the flag itself stays
477
+ guarded for ever, so the console kept echoing it and kept being refused. Echoing back the value
478
+ already in force is now a no-op; only a real change is refused, which is what the guard was
479
+ always for.
480
+ - **A preallocated file is no longer mistaken for a finished one.** A torrent client allocates the
481
+ whole file up front β€” libtorrent creates a 77 GB sparse file the moment a download starts β€” so an
482
+ archive 0% downloaded already measures exactly its final size. The completion sweep checked the
483
+ disk *first*, called that complete, and recorded it. On the next restart the composite then
484
+ handed a 10%-downloaded archive to a secondary as a finished seed, which is the one thing "only
485
+ the primary writes" exists to prevent. The engine's own progress now wins whenever it has an
486
+ opinion; the size check remains for the case it was written for, an archive the engine is not
487
+ holding at all.
488
+ - **A secondary is given long enough to hash what it was handed.** It is not waiting for metadata β€”
489
+ the `.torrent` carries that β€” it is verifying every byte against it, which is minutes for tens of
490
+ gigabytes. Against a default measured in seconds this appeared as `timed out after 60000ms
491
+ waiting for torrent metadata`, blaming the one thing that was never missing. Now
492
+ `secondaryShareTimeoutSeconds`, an hour by default.
493
+ - **The composite asks the primary before handing anything over**, rather than trusting the
494
+ caller's `seedOnly`. That flag is read from the catalog on restore, and a wrong `complete` there
495
+ was all that stood between one incomplete file and two clients writing to it.
496
+ - **A partial vector archive now gets its `vector_layers`, so the preview is not black.** A
497
+ PMTiles header is the first 127 bytes, but the JSON metadata carrying the layer list goes
498
+ wherever the writer put it β€” and planetiler puts it at the *end*, after every tile: byte
499
+ 77,139,967,368 of a 77 GB archive. Probing a file that is 10% downloaded therefore reads a
500
+ perfectly good header and 1528 zero bytes where the metadata should be, and every field except
501
+ the one vector rendering needs looks right. The header records that offset, so the range is
502
+ known and fetchable: `tiles.json` now reads it out of the swarm in the background, with a
503
+ timeout of its own (`tiles.metadataTimeoutMs`, 120s) rather than the interactive header budget,
504
+ which was far too short for a piece at the far end of an archive that nobody has asked for. The
505
+ reply is not held up, and the next request has the layers.
506
+ - **The vector preview draws.** `showInspectMap: true` sets a flag on maplibre-gl-inspect and
507
+ nothing else β€” the control renders from exactly two places, a source-change handler it
508
+ subscribes to only when `sources` was *not* passed, and the toggle button's click. This page
509
+ passed `sources` and hid the button, closing both, so nothing ever called `render()` and the map
510
+ stayed on a style that was a background colour and nothing else: correct TileJSON, correct tiles,
511
+ no console error, black map. Now rendered explicitly once the map has loaded.
512
+ - **The preview says why a vector map is blank** instead of showing a black rectangle. Related:
513
+ `sources` is no longer passed to maplibre-gl-inspect when there are no layers, since passing it
514
+ disables the control's own lookup β€” though that lookup only re-reads the TileJSON, so it is the
515
+ metadata fix above that actually makes the map draw.
516
+ - **The Pieces tab had no pane to render into**, so it appeared, highlighted when clicked, and did
517
+ nothing. Tabs and panes are now checked against each other in both directions.
518
+ - **The peers tab is no longer silently empty on libtorrent.** `peer_info.utp_socket` is absent
519
+ from libtorrent's 2.x Python bindings, so the sidecar raised on the first peer and returned
520
+ nothing β€” an archive downloading at 10 MiB/s from a connected seed reported having no peers at
521
+ all. Fixed in pmtiles-torrent; a node has to be restarted to pick up a sidecar change. Three
522
+ layers here had each turned that exception into an empty list, so a broken engine and an empty
523
+ swarm produced identical output: the route now answers `{ peers, error }` and the console shows
524
+ the reason, and the composite engine logs which engine failed instead of swallowing it. Peer
525
+ rows also now carry the engine that found them and whether each is an ordinary peer, a web seed
526
+ or an HTTP seed β€” an archive pulling at full speed from one web seed looks exactly like one
527
+ pulling from a swarm until that single server goes away.
528
+ - **Restoring skipped the tracker repair**, which is the one moment somebody expects a fix to take
529
+ effect. It built its own add rather than going through the shared one, so an archive stored
530
+ without trackers stayed unable to find a peer across every restart. It now takes the same path as
531
+ every other re-add.
532
+ - **A save path that has gone is reported.** An unmounted share or a tidied-away directory left the
533
+ engine unable to open anything and the archive sitting at nothing, with no error of its own.
534
+ Restore now says which archive, which path and what to do about it.
535
+ - **The Trackers tab explains an empty list.** An archive with no trackers and no `.torrent` can
536
+ only find peers through the DHT, which on a private or quiet swarm means it may never start β€”
537
+ and "downloading, 0 peers, indefinitely" is otherwise a mystery. It now says so, and shows what
538
+ the magnet itself carries while the metainfo has not arrived.
539
+ - **An archive joined from a bare infohash never started.** It was given no trackers, so there was
540
+ nowhere to look for a peer, and it sat reporting "downloading" indefinitely. Two causes, both
541
+ now fixed: `parse-torrent` gives a bare magnet an `announce` of `[]` rather than leaving it
542
+ undefined, so the nullish fallback to this node's own trackers kept the empty array and never
543
+ fired; and a magnet supplied by hand was stored verbatim rather than rebuilt, so it kept whatever
544
+ it lacked. The magnet is rebuilt from what was parsed β€” nothing is lost, since a supplied
545
+ magnet's trackers and web seeds are in there β€” and an archive already stored without any is
546
+ repaired whenever it is handed back to the engine, which is how the ones added before this get
547
+ fixed.
548
+ - **Stopping a node logged a page of engine errors.** The console keeps polling and a sweep or two
549
+ is still in flight while the engine is being torn down, and each of them was told the sidecar had
550
+ exited. An engine on its way out now reports an empty library instead, which is what it has.
551
+ - **A second engine was handed archives that were still downloading, and wrote its own copy.**
552
+ `restore` and every re-add claimed `seedOnly` for any mirror-mode archive, which means "the data
553
+ is already here, do not fetch it" β€” and for a half-downloaded archive that is untrue. A composite
554
+ engine took it at its word and passed the archive to the secondary, which honoured the incomplete
555
+ marker and opened `name.incomplete` while the primary wrote `name`: two clients, two files, one
556
+ archive, in one directory. `seedOnly` is now claimed only for archives that are actually
557
+ complete, and a secondary is never given a marker at all, since it only ever receives whole
558
+ archives.
559
+ - **An archive left with both filenames retried for ever.** Finalising refused to rename over an
560
+ existing file β€” correctly β€” and then tried again every fifteen seconds, logging the same
561
+ paragraph each time and telling nobody anything they could act on. When the file under the
562
+ archive's own name is the right size the archive is finished, so that is now recorded and the
563
+ leftover named once as something that can be deleted. Nothing is deleted automatically.
564
+ - **Two archives could be pointed at one file.** Filenames are not unique β€” two builds of the same
565
+ map are both `planet.pmtiles`, and a rebuild keeps the name while minting a new infohash β€” so
566
+ adding the second one now fails with a 409 naming the first, instead of letting them take turns
567
+ writing into the same file.
568
+ - **"marked incomplete" appeared beside a progress bar reading 100%.** It now sits with the state,
569
+ and only while the file on disk actually carries the marker.
570
+ - **Running two engines silently disabled on-demand tile reading.** The tile reader chose how to
571
+ fetch pieces by switching on the engine's name, and a composite calls itself
572
+ `libtorrent+webtorrent` β€” which matched neither case, so it fell through to "cannot read pieces
573
+ on demand". A half-downloaded archive that pmtiles-torrent could have served a header and tiles
574
+ from answered a 501 instead, and the preview showed an empty map. The reader now asks the
575
+ primary, which is the only engine that downloads and therefore the only one that holds a partial
576
+ archive at all. Verified end to end: header, metadata and a tile read out of a swarm from an
577
+ archive this node held none of, with 16 KiB on disk afterwards β€” one piece.
578
+ - **The map preview showed nothing but "Loading…".** It imported MapLibre as a default export, and
579
+ MapLibre's ESM build has only named ones β€” which is a `SyntaxError` raised before a line of the
580
+ module runs, so there was no failed request and no clue in the page, only a line in the browser
581
+ console. It is a namespace import now, and a test reads both bundles and asserts the import form
582
+ matches what each actually exports, and that every `maplibregl.X` the page uses is a name the
583
+ bundle provides.
584
+ - **Stopping a node running the libtorrent engine printed a Python stack trace.** Windows delivers
585
+ a console Ctrl-C to every process in the group, so the sidecar received it too and reported a
586
+ `KeyboardInterrupt` on the way out. Nothing was wrong, but a traceback at the end of a clean stop
587
+ reads as a crash and buries the lines that say what actually happened. Fixed properly in the
588
+ sidecar, which ships with `pmtiles-torrent`, and suppressed here as well so an older sidecar is
589
+ quiet too. Separately, the engine no longer reports an exit it asked for as a failure β€” that
590
+ rejected a promise nobody was waiting on, which is how Node announces a crash.
591
+ - **An archive adopted from a client on another machine never started.** The magnet built for it
592
+ carried the infohash and nothing else β€” no trackers β€” so there was nowhere to look for peers but
593
+ the DHT, and it sat at 0% reporting "downloading" and meaning nothing of the kind. The client
594
+ being adopted from is seeding the archive and therefore *has* the metainfo, so that is fetched
595
+ and used instead: trackers, web seeds and piece geometry included, and kept on disk so a restart
596
+ does not need the swarm. Where a client cannot export one, the magnet at least carries this
597
+ node's own trackers now.
598
+ - **Adopting from this node's own engine restarted the download.** Whether the data could be read
599
+ from this process was being used to decide whether the engine held it, which are different
600
+ questions β€” so an archive under a path this process could not open was re-added as a magnet,
601
+ pointed at a different directory, and downloaded again from nothing. Adopting from the configured
602
+ engine is a catalog operation now; readability only decides whether tiles can be served straight
603
+ off the file.
604
+ - **The archives table lost a column.** Adding *Ratio* replaced the *Up* cell instead of following
605
+ it, so every value from there rightwards sat under the wrong heading β€” the upload speed appeared
606
+ as the ratio, and *State* was blank. A test now asserts the row builds exactly as many cells as
607
+ the table has headings.
608
+ - **"Add archive…" threw `locationPicker is not defined`.** The save-location helpers were declared
609
+ inside the detail panel's renderer, so the add and adopt dialogs β€” which are not β€” could not see
610
+ them. `node --check` accepts that happily: it is a syntax-clean script and a `ReferenceError` at
611
+ click time. The console script is now checked for it, by counting brace depth over a source with
612
+ strings, template literals, comments and regular expressions blanked out, and asserting that
613
+ every helper called from more than one place is declared at the top level. Verified against the
614
+ commit that broke it.
615
+ - **Magnets dropped the web seeds their torrents advertised.** Torrents created here have always
616
+ put them in the magnet; a torrent that was *joined* did not, and a web seed added after
617
+ publication reached everyone holding the `.torrent` and nobody holding the magnet β€” which is the
618
+ link that actually gets shared. Both now carry `ws=` for every seed the torrent advertises. This
619
+ does not weaken anything: whether a URL may be published is decided once, when the torrent is
620
+ created, and once it is in the `url-list` anyone holding the `.torrent` already has it.
621
+ - **An archive joined by magnet forgot everything the swarm told it.** A magnet carries an
622
+ infohash and, if you are lucky, a display name; the real name, the exact size and the piece
623
+ geometry arrive afterwards over BEP 9 β€” and arrived into nothing. Every restart asked the swarm
624
+ again for what the node had already been told, which needs a peer, so a restart while the swarm
625
+ was quiet left the archive stuck. The `.torrent` endpoint had nothing to serve and the feed
626
+ advertised a URL that answered 404, the Content tab was empty, and the size stayed at whatever
627
+ the magnet claimed β€” usually zero, which made the disk-space check before a move meaningless.
628
+ The metainfo is now written to the torrent directory as soon as the engine has it, which for a
629
+ magnet is the moment the add resolves. Anything joined before this is picked up by the sweep.
630
+ Only gaps are filled: a name chosen here is a decision about this node's copy and is not
631
+ overruled.
632
+ - **Every radio and checkbox in the console sat centred on a line of its own**, with its label
633
+ above it. `.field label` makes a label `display: block` and `.field input` stretches a control to
634
+ the full width of its dialog, and both applied to these too. They share a `choice` class now,
635
+ defined last in the stylesheet because the rules it has to beat match just as tightly β€” position,
636
+ not specificity, is what settles it. A test asserts both halves, since moving the block up the
637
+ sheet would silently revert the layout.
638
+ - **Shutting down could leave the port held, so the next run could not start.** Three faults in one
639
+ loop. The signal handlers were installed at the *end* of startup, so a Ctrl-C while the catalogue
640
+ was being handed back to the engine reached nothing at all and killed the process outright β€” port
641
+ still held, trackers still believing it was seeding. They are installed before any of what they
642
+ stop exists now. Closing the HTTP server only dropped *idle* connections, so one stuck request β€”
643
+ a tile read waiting on the swarm, say β€” kept it open past its own timeout; anything still
644
+ in-flight is now forced shortly after. And a WebTorrent client that cannot open its port reports
645
+ it asynchronously, long after construction: that was logged and ignored, after which every add
646
+ waited out a five-minute metadata timeout against a client that could never talk to anyone. It is
647
+ fatal now, reported with what to do about it, and restore stops at the first one rather than
648
+ repeating it per archive. A `.torrent` also no longer waits on the magnet timeout, since it
649
+ carries its own metadata.
650
+ - **Peer tokens were returned in plain text by `GET /api/config`.** A token is what persuades a
651
+ peer to publish more than it publishes to the world β€” the same class of thing as the qBittorrent
652
+ password, which was already redacted. Now redacted too, and a save that echoes the placeholder
653
+ back keeps the stored token rather than overwriting it with asterisks.
654
+ - **Adding the first peer did nothing until a restart.** The refresh timer only started when the
655
+ subscription list was already non-empty, so a peer added through the console was never polled.
656
+ The same bug as scheduled sources had; every refresh reads the list fresh.
657
+ - **Categories set when adding an archive never appeared.** The console read `entry.category`,
658
+ singular β€” the field the catalog folds into the list and deletes on write β€” so every archive
659
+ showed a blank tag line. The tags were stored correctly the whole time.
660
+ - **Pausing, resuming and switching mode silently did nothing on the WebTorrent engine.**
661
+ `client.get()` is async β€” it parses whatever identifier it is handed before matching β€” so the
662
+ promise it returned read as a perfectly good torrent whose every property was `undefined`. Every
663
+ guard therefore saw "no such torrent" and returned false, and `setMode` fell back to removing and
664
+ re-adding the torrent, which is why it appeared to work at all. Looked up directly by infohash
665
+ now, which needs no parsing.
666
+ - **The console offered a TileJSON URL for archives that can never have one.** Identification only
667
+ ran when an archive was created here, so a *joined* MBTiles torrent had no recorded format and
668
+ was treated as PMTiles: a tile endpoint was offered, and asking for it read pieces out of the
669
+ swarm until the reader hit the magic-number check. A joined torrent now takes an initial format
670
+ from its filename, the first read records what the content actually is, and both `tiles.json`
671
+ and the tile route answer 415 rather than retrying forever. The console hides the TileJSON,
672
+ preview and warm controls for anything that is not PMTiles and says why.
673
+ - **An archive opened in cache mode was read through the swarm forever.** Which source to use was
674
+ decided once, at open, so switching to mirror β€” or the download simply finishing β€” changed
675
+ nothing, and tiles kept being pulled a piece at a time while a complete copy sat on disk. The
676
+ reader is now told to forget an archive whenever its mode changes, it is paused or resumed, or
677
+ its cache is cleared.
678
+ - A TileJSON request for an archive nobody is seeding waited a full minute before saying so, which
679
+ reads as a hang. It is bounded at twelve seconds now (`tiles.headerTimeoutMs`) and says what is
680
+ actually wrong: no peers yet, and no web seed to fall back on.
681
+ - **Ctrl-C could hang.** Once archives were restored to the engine at startup, stopping meant
682
+ telling every tracker so β€” and an unreachable one waits for a timeout each. Every shutdown step
683
+ is now bounded, a watchdog exits regardless after fifteen seconds, in-flight downloads are
684
+ cancelled first, and a second Ctrl-C forces the issue instead of stacking another shutdown.
685
+ - Opening a detail tab and waiting sent you back to General. The three-second poll rebuilt the
686
+ whole panel; it now updates the table only, and an action that does re-render the panel returns
687
+ to the tab you were on.
688
+ - **A restart silently stopped seeding everything.** Nothing handed the catalogue back to the
689
+ engine, so the catalog still listed every archive and the console still showed them while the
690
+ engine held none. They are restored at startup now, each in the mode it was left in.
691
+ - **Switching mode after a restart crashed the process.** WebTorrent throws for an unknown
692
+ infohash, and because its `remove()` is async the rejection escaped from inside the executor
693
+ where a caller's `catch` could not see it. Removing something the engine does not hold is now
694
+ treated as already done, which is what was wanted.
695
+ - `webtorrent` is a plain dependency rather than an optional one. It is the *default* engine, so
696
+ calling it optional was wrong, and npm repeatedly dropped it from the lockfile while leaving the
697
+ declaration β€” after which `npm install webtorrent` reported "up to date" and changed nothing,
698
+ and the default engine failed to start.
699
+ - A torrent's comment and piece length were accepted by the API but never passed on.
700
+ - A custom `webSeeds` list was discarded when `webSeed: false` β€” exactly the case where the source
701
+ must not be published and a public URL was supplied in its place.
702
+ - **A pre-signed source URL was published as a web seed, credentials and all.** Adding an archive
703
+ from an S3 or Azure signed link baked that link β€” a bearer credential β€” into the `.torrent` and
704
+ broadcast it to the swarm, where it cannot be recalled. Such URLs are now detected and not
705
+ published; `webSeed: false` suppresses any source URL, and `webSeeds` supplies a public one
706
+ instead.
707
+ - **Creating a torrent from a local path published any readable file to a public swarm.** The
708
+ PMTiles probe failure was caught and discarded, so `{"path": "/etc/shadow"}` produced a
709
+ seeded torrent and returned its infohash. Archives are now identified by content β€” PMTiles
710
+ and MBTiles are recognised, anything else is a 400 unless `allowUnknown` is passed. Only
711
+ PMTiles can have its tiles served; MBTiles is SQLite, whose pages are scattered rather than
712
+ spatially clustered, so it is distributable but not servable.
713
+ - **A missing tile answered 204 for every archive, which breaks sparse raster.** MapLibre only
714
+ overzooms a parent tile when the child 404s, so a sparse raster-dem β€” Mapterhorn, or any
715
+ terrain built only where there is land β€” rendered as holes wherever data was never built.
716
+ Raster now answers 404 and vector keeps 204.
717
+
718
+ ### πŸ“š Documentation
719
+ - **Ports and reachability**, which nothing covered before: which of the four listeners wants a
720
+ forwarding rule (the peer port, exactly as in qBittorrent), why WebRTC wants none of them β€” it
721
+ is signalled over a `wss://` tracker and carried over ICE with STUN, so it needs outbound UDP
722
+ rather than an inbound rule β€” and why peer traffic never touches the load balancer. Also that
723
+ two engines need two peer ports, since WebTorrent picks a random one unless told otherwise, and
724
+ that **browser peers need a `wss://` tracker in the announce list**: the defaults are UDP-only,
725
+ a browser has no UDP socket and no DHT, and WebTorrent's own WebSocket trackers ship only in its
726
+ browser bundle. Without one, the browser half of the swarm cannot find a peer however many nodes
727
+ are seeding.
728
+ - The topology diagram shows the **browser bridge**: browsers speak WebRTC and conventional
729
+ clients speak TCP and uTP, so a browser peer is only ever reached by a node running WebTorrent.
730
+ The deployment notes cover the **two-port split**, which decides what a load balancer may be
731
+ pointed at. The API table gained the four routes it was missing (`/api/adds`, `/api/session`,
732
+ `/archives/{hash}/archive.torrent`, `/archives/{hash}/preview`), and there are now tests that a
733
+ diagram's `linkStyle` indices are in range, that every relative link and anchor resolves, and
734
+ that no route is missing a row.
735
+
10
736
  ## 0.2.0
11
737
  ### ✨ Features and improvements
12
738
  - **Serve tiles.** Every archive now has a TileJSON endpoint and a `{z}/{x}/{y}` tile endpoint