pmtiles-swarm 0.2.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md ADDED
@@ -0,0 +1,53 @@
1
+ # pmtiles-swarm changelog
2
+
3
+ ## master
4
+ ### ✨ Features and improvements
5
+ - _...Add new stuff here..._
6
+
7
+ ### šŸž Bug fixes
8
+ - _...Add new stuff here..._
9
+
10
+ ## 0.2.0
11
+ ### ✨ Features and improvements
12
+ - **Serve tiles.** Every archive now has a TileJSON endpoint and a `{z}/{x}/{y}` tile endpoint
13
+ under `/archives/{infohash}/`. A node holding a complete copy reads its local file; a node in
14
+ cache mode reads through the swarm via `pmtiles-torrent`, fetching only the pieces a requested
15
+ tile lives in and seeding them back.
16
+ - The TileJSON carries a non-standard `torrent` block — infohash, magnet, `.torrent` URL, web
17
+ seeds and any BEP 46 publisher key. Ordinary clients ignore it and fetch over HTTP;
18
+ torrent-aware clients use it to join the swarm directly. One URL serves both.
19
+ - Tiles are served `immutable` with a year-long max-age. An infohash pins content, so a tile
20
+ under one can never change, and an updated archive gets new URLs rather than needing a purge.
21
+ - **Warm a region before serving it.** `POST /api/torrents/{infohash}/warm` pre-fetches the
22
+ tiles covering a bounding box, so a cache-mode node is useful the moment it enters a
23
+ load-balanced pool rather than paying for the first request to every area. Progress and
24
+ cancellation via `GET` and `DELETE` on the same path. The zoom range is clamped to what the
25
+ archive actually holds.
26
+ - New `trustProxy` config option. With it set, absolute URLs in TileJSON and the RSS feed are
27
+ derived per request from `X-Forwarded-Proto` and `X-Forwarded-Host`, so one node can answer
28
+ correctly on both `https://public` and `http://internal`.
29
+ - Depend on `pmtiles-torrent` from npm, and drop the local copy of the libtorrent sidecar in
30
+ favour of the one it ships. The two copies had drifted: the read side had grown `info` and
31
+ `set_priority` ops this project never got, which are exactly what on-demand tile reads need.
32
+ - Upgrade WebTorrent to 3.x, dropping the `uint8-util` override that the 2.x line needed to add
33
+ magnets at all. **Node 20 is no longer supported** — WebTorrent 3 requires Node 22+, and Node
34
+ 20 reached end of life in April 2026.
35
+
36
+ ### šŸž Bug fixes
37
+ - Absolute URLs used the raw `Host` header, which behind a reverse proxy is the internal address
38
+ the proxy dialled. They now follow `X-Forwarded-Host` when a proxy is trusted, so published
39
+ tile and feed URLs are reachable.
40
+ - Stop tracking a compiled `.pyc` that predated the `__pycache__` ignore rule.
41
+
42
+ ## 0.1.0
43
+ ### ✨ Features and improvements
44
+ - Initial release: BitTorrent distribution for PMTiles map archives.
45
+ - Pluggable seeding engines: libtorrent (via sidecar), qBittorrent (WebUI API), and embedded WebTorrent.
46
+ - Four ways to add an archive: local file, remote URL, existing torrent or magnet, and adoption of what the engine already seeds.
47
+ - Web seeds (BEP 19) registered automatically, so a new archive is usable before it has any peers.
48
+ - Mirror and cache modes; joining defaults to cache so a large archive cannot silently claim the disk.
49
+ - Scheduled sources for upstreams publishing a new dated URL per build.
50
+ - Origin change detection, with optional guarded auto-rebuild.
51
+ - RSS publish and subscribe, with map metadata (format, zoom range, bounds) in each item.
52
+ - Hybrid v1+v2 torrent creation through libtorrent.
53
+ - BEP 46 mutable-torrent helpers.
package/LICENSE ADDED
@@ -0,0 +1,28 @@
1
+ BSD 3-Clause License
2
+
3
+ Copyright (c) 2026, Andrew Calcutt and contributors
4
+
5
+ Redistribution and use in source and binary forms, with or without
6
+ modification, are permitted provided that the following conditions are met:
7
+
8
+ 1. Redistributions of source code must retain the above copyright notice, this
9
+ list of conditions and the following disclaimer.
10
+
11
+ 2. Redistributions in binary form must reproduce the above copyright notice,
12
+ this list of conditions and the following disclaimer in the documentation
13
+ and/or other materials provided with the distribution.
14
+
15
+ 3. Neither the name of the copyright holder nor the names of its contributors
16
+ may be used to endorse or promote products derived from this software
17
+ without specific prior written permission.
18
+
19
+ THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
20
+ AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
21
+ IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
22
+ DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
23
+ FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
24
+ DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
25
+ SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
26
+ CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
27
+ OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
28
+ OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
package/NOTICE.md ADDED
@@ -0,0 +1,47 @@
1
+ # Third-party notices
2
+
3
+ `pmtiles-swarm` is licensed BSD-3-Clause (see [LICENSE](LICENSE)). This file records the
4
+ third-party work it builds on.
5
+
6
+ ## PMTiles — BSD-3-Clause (reference implementation), CC0-1.0 (specification)
7
+
8
+ > Copyright 2021 and later, Protomaps LLC and contributors
9
+ > https://github.com/protomaps/PMTiles
10
+
11
+ The `pmtiles` npm package is used at runtime to read archive headers and metadata. No
12
+ implementation code is copied.
13
+
14
+ ## WebTorrent — MIT
15
+
16
+ > Copyright (c) Feross Aboukhadijeh and WebTorrent, LLC
17
+ > https://github.com/webtorrent/webtorrent
18
+
19
+ `src/engines/webtorrent.js` is an adapter written against WebTorrent's public API.
20
+ `src/mutable.js` builds on `bittorrent-dht`'s BEP 44 put/get. `create-torrent` and `parse-torrent`,
21
+ from the same ecosystem and also MIT, are used for torrent creation and parsing. No implementation
22
+ code is copied.
23
+
24
+ Note the `uint8-util` override pinned in `package.json`: version 2.3.0 rewrote `arr2hex` in a way
25
+ that throws when handed the hex-string infohash that webtorrent's `Torrent._onTorrentId` passes it,
26
+ breaking every magnet add. webtorrent declares `^2.2.5`, so the pin holds it at a working version.
27
+ Remove it once webtorrent fixes the call site.
28
+
29
+ ## qBittorrent — GPL-2.0-or-later
30
+
31
+ > https://github.com/qbittorrent/qBittorrent
32
+
33
+ **No qBittorrent code is used in this project.** `src/engines/qbittorrent.js` is a client for its
34
+ documented WebUI HTTP API, written from the public API documentation. qBittorrent is GPL-2.0+,
35
+ which is not compatible with redistribution under this project's BSD-3-Clause license, so code
36
+ must not be copied from it. Speaking to a program over its network API does not create a derived
37
+ work; copying its source would.
38
+
39
+ The same caution applies to libtorrent-rasterbar if a libtorrent-backed engine is added later:
40
+ libtorrent is BSD-3-Clause and therefore fine to link and derive from, but it must be pulled in
41
+ directly rather than by way of any GPL client.
42
+
43
+ ## BitTorrent Enhancement Proposals
44
+
45
+ The BEPs implemented here — BEP 19 (web seeds), BEP 44 (DHT storage) and BEP 46 (updating torrents
46
+ via DHT mutable items) — are open specifications published by the BitTorrent community at
47
+ https://www.bittorrent.org/beps/bep_0000.html.
package/README.md ADDED
@@ -0,0 +1,172 @@
1
+ # pmtiles-swarm
2
+
3
+ BitTorrent distribution built for PMTiles map archives.
4
+
5
+ A torrent client and server that knows what a map is: it creates torrents from PMTiles archives,
6
+ watches folders for new builds, publishes an RSS feed carrying each archive's coverage and zoom
7
+ range, and follows other nodes' feeds to mirror or cache what they publish.
8
+
9
+ It does not reimplement BitTorrent. Seeding is delegated to an engine — your existing qBittorrent,
10
+ or an embedded WebTorrent client — so libtorrent keeps doing what it is good at.
11
+
12
+ ```sh
13
+ npm install
14
+ node src/index.js --config swarm.config.json
15
+ # then open http://localhost:8090
16
+ ```
17
+
18
+ ## Documentation
19
+
20
+ - **[docs/engines.md](docs/engines.md)** — libtorrent, qBittorrent and WebTorrent: what each
21
+ can do, what to install, and how cache mode differs between them.
22
+ - **[docs/publishing.md](docs/publishing.md)** — creating torrents, web seeds, piece size,
23
+ hybrid v1+v2, watch folders, and the keep-or-discard choice when adding from a URL.
24
+ - **[docs/subscribing.md](docs/subscribing.md)** — a worked two-node setup, mirror vs cache,
25
+ feed contents, and updatable torrents.
26
+ - **[docs/serving-tiles.md](docs/serving-tiles.md)** — the TileJSON and z/x/y endpoints, the
27
+ `torrent` block that torrent-aware clients use, caching, and running behind a proxy.
28
+ - **[docs/architecture-diagram.md](docs/architecture-diagram.md)** — how a publishing node, a
29
+ serving tier, the swarm and both kinds of client fit together.
30
+
31
+ ## What it does
32
+
33
+ **Adds archives four ways.** A local `.pmtiles` file (hashed into a new torrent, data left where
34
+ it is); a remote URL (see below); an existing `.torrent` or magnet, which it simply joins; or
35
+ adoption of everything your torrent client already seeds — the migration path for an existing
36
+ library, which re-hashes nothing.
37
+
38
+ **Watches folders.** A new `.pmtiles` appearing in a watched folder is imported automatically.
39
+ Imports wait for the file to stop changing first, because hashing a half-written archive produces
40
+ a torrent for bytes that no longer exist.
41
+
42
+ **Publishes RSS.** `/feed.xml`, and `/feed/<category>.xml` per category. Plain RSS 2.0 with
43
+ torrent enclosures, so **qBittorrent's built-in RSS auto-downloader can subscribe today** with no
44
+ new software. Items also carry a namespaced description of the map — format, zoom range, bounds,
45
+ tile count — so a subscriber can decide whether it wants a 72 GiB download before starting one.
46
+
47
+ **Follows feeds.** Subscribed feeds are polled and new archives added in one of two modes.
48
+
49
+ **Serves tiles.** Every archive is also a TileJSON endpoint and a `{z}/{x}/{y}` tile endpoint, so
50
+ a map can point straight at it. A node holding a complete copy reads its local file; a node in
51
+ cache mode reads through the swarm, pulling only the pieces a requested tile lives in. The
52
+ TileJSON carries a `torrent` block that ordinary clients ignore and torrent-aware ones use to
53
+ join the swarm directly — one URL serves both. See
54
+ [docs/serving-tiles.md](docs/serving-tiles.md).
55
+
56
+ ## Mirror and cache
57
+
58
+ The distinction is the point of the project.
59
+
60
+ | Mode | Disk cost | What it is for |
61
+ | --- | --- | --- |
62
+ | `mirror` | the whole archive | Becoming a full seeder and adding redundancy to the swarm |
63
+ | `cache` | only what is read | Serving tiles from a 700 GiB archive on a small disk |
64
+
65
+ Cache mode joins the swarm without downloading anything up front. A tile server reads byte ranges
66
+ on demand through [`pmtiles-torrent`](https://github.com/TechIdiots-LLC/pmtiles-torrent), and the node still
67
+ seeds whatever pieces it has picked up along the way. Disk use tracks what people actually look
68
+ at rather than what exists.
69
+
70
+ Joining a torrent **defaults to cache**, because committing a disk to a copy of something that may
71
+ be hundreds of gigabytes should be a decision, not a side effect.
72
+
73
+ Cache mode needs piece-level control. WebTorrent has it; qBittorrent's WebUI exposes only per-file
74
+ priorities, which for a single-file archive is all or nothing — so under the qBittorrent engine,
75
+ cache mode adds the torrent stopped and leaves on-demand reads to a client that can do them.
76
+
77
+ ## Adding from a URL
78
+
79
+ Piece hashes are computed over content, so there is no way to create a torrent from a remote
80
+ archive without reading every byte of it. What you choose is whether those bytes are kept:
81
+
82
+ - **retain** (default) — written to disk as they arrive, so the node is a real seeder the moment
83
+ the torrent is published.
84
+ - **discard** (`"retain": false`) — streamed past the hasher and dropped. No disk cost, but the
85
+ node cannot seed what it just published; peers depend on the web seed until someone mirrors it.
86
+
87
+ Either way the origin URL is registered as a **web seed** (BEP 19), which is what makes a
88
+ brand-new archive usable before it has any peers at all. If your archives are already on a web
89
+ server or S3, always pass the URL — it turns cold start from a dead end into an HTTP fallback that
90
+ gets cheaper as peers appear.
91
+
92
+ ## Updatable torrents
93
+
94
+ A rebuilt archive is a different torrent: the infohash is a hash of the content. Two ways to carry
95
+ subscribers forward, and they fail differently, so publishing both is cheap insurance.
96
+
97
+ - **RSS** — easy to consume, understood by existing clients, needs a server that stays up.
98
+ - **BEP 46** — an ed25519-signed DHT record naming the current infohash, addressed by public key
99
+ rather than infohash (`magnet:?xs=urn:btpk:…`). No server needed, but the record expires and must
100
+ be republished. See [src/mutable.js](src/mutable.js).
101
+
102
+ ## Configuration
103
+
104
+ ```json
105
+ {
106
+ "port": 8090,
107
+ "dataDir": "./data",
108
+ "engine": "qbittorrent",
109
+ "qbittorrent": { "url": "http://127.0.0.1:8080", "username": "admin", "password": "…" },
110
+ "webtorrent": { "savePath": "./data/torrents-data" },
111
+ "pieceLength": 4194304,
112
+ "maxConnections": 100,
113
+ "feedMaxItems": 50,
114
+ "publicUrl": "https://maps.example.org",
115
+ "watch": [
116
+ { "path": "/mnt/maps/generated", "category": "basemaps", "webSeedBase": "https://maps.example.org/files" }
117
+ ],
118
+ "subscriptions": [
119
+ { "url": "https://other.example.org/feed.xml", "mode": "cache", "filter": "terrain" }
120
+ ]
121
+ }
122
+ ```
123
+
124
+ Environment overrides: `PMTILES_SWARM_PORT`, `PMTILES_SWARM_DATA_DIR`, `PMTILES_SWARM_ENGINE`,
125
+ `PMTILES_SWARM_QBT_URL`, `PMTILES_SWARM_QBT_USERNAME`, `PMTILES_SWARM_QBT_PASSWORD`,
126
+ `PMTILES_SWARM_PUBLIC_URL`.
127
+
128
+ ### Piece length
129
+
130
+ Creation tools size pieces for whole-file downloads — 16 MiB or more for a large archive. That is
131
+ a poor fit for a tile server, where a cold tile costs a whole piece regardless of how few bytes it
132
+ needs. The default here is 4 MiB, trading a larger hash list for a quarter of the read
133
+ amplification. Measured against a 72 GiB archive at 16 MiB pieces, a cold tile cost roughly 30
134
+ seconds against a single peer.
135
+
136
+ Overridable globally, per scheduled source, and per request — an archive nobody will read
137
+ randomly can keep larger pieces. Note that piece size has little bearing on load imposed on
138
+ network equipment: peers request 16 KiB blocks whatever the piece size. The setting that
139
+ matters there is `maxConnections`, since every peer holds a NAT table entry. See
140
+ [docs/publishing.md](docs/publishing.md).
141
+
142
+ ## API
143
+
144
+ | Method | Path | Purpose |
145
+ | --- | --- | --- |
146
+ | `GET` | `/api/status` | Engine health, counts, watched folders, subscriptions |
147
+ | `GET` | `/api/torrents` | Catalog joined with live swarm state |
148
+ | `POST` | `/api/torrents` | Add via `{path}`, `{url}`, `{magnet}`, `{torrentUrl}`, or a raw `.torrent` body |
149
+ | `DELETE` | `/api/torrents/:infoHash` | Remove (`?deleteData=true` to delete data too) |
150
+ | `GET` | `/api/torrents/:infoHash/file` | Download the `.torrent` |
151
+ | `GET` | `/api/torrents/:infoHash/magnet` | Magnet URI |
152
+ | `GET` | `/api/torrents/:infoHash/peers` | Per-peer detail |
153
+ | `POST` | `/api/torrents/:infoHash/check` | Has the source changed since the torrent was made? |
154
+ | `POST` | `/api/torrents/:infoHash/rebuild` | Rebuild from the current source (mints a new infohash) |
155
+ | `POST` | `/api/check-origins` | Check every archive with a watchable source |
156
+ | `POST` | `/api/adopt` | Import what the engine already holds |
157
+ | `POST` | `/api/subscriptions/refresh` | Poll subscribed feeds now |
158
+ | `GET` | `/feed.xml`, `/feed/:category.xml` | RSS |
159
+
160
+ ## Status
161
+
162
+ Working and verified end to end against a live 71.93 GiB OpenMapTiles archive: torrent creation,
163
+ joining existing torrents, catalog persistence, map metadata extraction, category feeds, live
164
+ swarm stats, and cache mode holding at zero bytes on disk.
165
+
166
+ Not yet exercised: the qBittorrent engine against a real instance, watch-folder imports, feed
167
+ subscription round-trips between two nodes, and BEP 46 publish/resolve against a live DHT (the
168
+ crypto and magnet handling are tested; interop with libtorrent's encoding is not).
169
+
170
+ ## License and attribution
171
+
172
+ BSD-3-Clause. Third-party notices in [NOTICE.md](NOTICE.md).
@@ -0,0 +1,287 @@
1
+ # Architecture Diagram — pmtiles-swarm
2
+
3
+ How a map library is ingested once, distributed over BitTorrent, and served to
4
+ both ordinary and torrent-aware clients from the same URLs.
5
+
6
+ Four views: **the whole topology**, **how a tile request resolves**, **what a
7
+ torrent-aware client does differently**, and **what happens when an archive is
8
+ updated**.
9
+ (Renders in VS Code's Markdown preview and on GitHub.)
10
+
11
+ ---
12
+
13
+ ## 1. Topology
14
+
15
+ One node publishes, any number serve, everything meets in the swarm.
16
+
17
+ **"Primary" and "secondary" are roles, not node types.** The only thing that makes
18
+ a node primary is that it creates torrents and owns the catalog others subscribe
19
+ to. It can sit in the load-balanced pool alongside everything else — it holds
20
+ complete copies, so it is the *best* tile server in the fleet, not an exception
21
+ to it. The diagram separates them only to keep the arrows legible.
22
+
23
+ Likewise **mirror and cache are per-archive choices, not node identities.** A
24
+ serving node can mirror the archives it cares about and cache the rest. Mirroring
25
+ everything is a perfectly good deployment: every node seeds fully, every node
26
+ reads locally, and cache mode is there for the nodes and archives where a full
27
+ copy is not worth the disk.
28
+
29
+ ```mermaid
30
+ %%{init: {"flowchart": {"rankSpacing": 80, "nodeSpacing": 50}}}%%
31
+ graph LR
32
+ subgraph IN["Ingest — any of four ways"]
33
+ F1["watch folder<br/><small>new .pmtiles appears</small>"]
34
+ F2["scheduled source<br/><small>protomaps, mapterhorn…</small>"]
35
+ F3["torrent / magnet URI<br/><small>join an existing swarm</small>"]
36
+ F4["local file or HTTP URL<br/><small>hashed into a new torrent</small>"]
37
+ end
38
+
39
+ F1 & F2 & F3 & F4 --> PRI
40
+
41
+ PRI["<b>pmtiles-swarm Ā· publisher</b><br/>creates torrents Ā· owns the catalog<br/>publishes /feed.xml<br/><small>mirrors — and can serve tiles too</small>"]
42
+
43
+ PRI -->|"RSS<br/>torrents + magnets"| S1 & S2 & S3
44
+
45
+ subgraph SEC["Serving tier — subscribed to the publisher's feed"]
46
+ S1["pmtiles-swarm<br/><small>mirror — reads locally, seeds fully</small>"]
47
+ S2["pmtiles-swarm<br/><small>mirror — reads locally, seeds fully</small>"]
48
+ S3["pmtiles-swarm<br/><small>cache — reads pieces on demand</small>"]
49
+ end
50
+
51
+ BT{{"BitTorrent swarm<br/><small>DHT Ā· trackers Ā· web seeds</small>"}}
52
+
53
+ PRI <==>|"seeds"| BT
54
+ S1 & S2 <==>|"seed complete copies"| BT
55
+ S3 <==>|"pull pieces on demand<br/>seed back what it holds"| BT
56
+
57
+ PRI --> LB
58
+ S1 & S2 & S3 --> LB["Load balancer / CDN<br/>HAProxy Ā· Cloudflare<br/><small>caches tiles — they are immutable</small>"]
59
+
60
+ LB --> C1 & C2
61
+
62
+ subgraph CL["Clients"]
63
+ C1["<b>Torrent-aware client</b><br/>maplibre-maui-ac<br/><small>reads the torrent block,<br/>joins the swarm directly</small>"]
64
+ C2["<b>Ordinary client</b><br/>maplibre-gl-js Ā· Leaflet<br/><small>ignores the torrent block,<br/>fetches tiles over HTTP</small>"]
65
+ end
66
+
67
+ C1 <==>|"tiles from pieces"| BT
68
+
69
+ %% 4-6 = RSS distribution, 7-10 and 17 = BitTorrent, rest = HTTP
70
+ linkStyle 4,5,6 stroke:#3F8F4F,stroke-width:2.5px;
71
+ linkStyle 7,8,9,10,17 stroke:#F5A623,stroke-width:3.5px;
72
+ ```
73
+
74
+ **Key points:** **orange = BitTorrent, green = RSS distribution, plain = HTTP.**
75
+ The publisher is the only node that *creates* torrents; the rest learn about
76
+ archives from its feed and join the swarm, each choosing per archive whether to
77
+ mirror it or cache it.
78
+
79
+ Every node is both a reader and a seeder. A mirror seeds the whole archive; a
80
+ cache-mode node seeds whatever pieces it has pulled to answer requests. Either
81
+ way serving load turns into swarm capacity rather than consuming it, which is the
82
+ inversion that makes this worth building.
83
+
84
+ The publisher is a single point of failure for **publishing new archives only**.
85
+ Once a torrent exists, the swarm and the serving tier keep working without it —
86
+ including the feed's existing items, which subscribers have already acted on.
87
+
88
+ ---
89
+
90
+ ## 2. How a tile request resolves
91
+
92
+ The same URL takes very different paths depending on who asks and what the
93
+ answering node holds.
94
+
95
+ ```mermaid
96
+ %%{init: {"flowchart": {"rankSpacing": 55, "nodeSpacing": 40}}}%%
97
+ graph TD
98
+ REQ["GET /archives/{infohash}/{z}/{x}/{y}.pbf"] --> CDN{"CDN cache?"}
99
+ CDN -->|"hit"| DONE(["tile bytes"])
100
+ CDN -->|"miss"| NODE["a secondary, via the load balancer"]
101
+
102
+ NODE --> HOLD{"does this node hold<br/>a complete copy?"}
103
+ HOLD -->|"yes Ā· mirror"| LOCAL["read the local file<br/><small>NodeFileSource</small>"]
104
+ HOLD -->|"no Ā· cache mode"| ENG{"can the engine<br/>read pieces?"}
105
+
106
+ ENG -->|"libtorrent Ā· webtorrent"| SWARM["map the byte range onto pieces<br/><small>TorrentSource</small>"]
107
+ ENG -->|"qBittorrent"| ERR["501 — no piece-level read"]
108
+
109
+ SWARM --> PC{"piece cached?"}
110
+ PC -->|"yes"| LOCAL
111
+ PC -->|"no"| FETCH["fetch the piece<br/><small>web seed, or peers</small>"]
112
+ FETCH --> SEED["keep it Ā· seed it back"]
113
+ SEED --> LOCAL
114
+
115
+ LOCAL --> DONE
116
+
117
+ %% 10-12 = the piece fetch and seed-back path
118
+ linkStyle 10,11,12 stroke:#F5A623,stroke-width:3px;
119
+ ```
120
+
121
+ **Key points:** a cold tile on a cache-mode node costs one piece fetch — and a
122
+ **web seed answers that in well under a second**, against 30+ seconds for a cold
123
+ swarm-only fetch. If your archives are also on plain HTTP storage, put that URL
124
+ in the torrent's `url-list`; it is the single biggest lever on this whole design.
125
+
126
+ A node holding a complete copy skips all of this and reads its local file.
127
+
128
+ ---
129
+
130
+ ## 3. What a torrent-aware client does
131
+
132
+ The server-side path above is what an *ordinary* client triggers. A torrent-aware
133
+ client does something different: it takes over the archive reading itself, and
134
+ stops needing the tile endpoint at all.
135
+
136
+ The important part is that it does both at once — HTTP for the first paint, swarm
137
+ in the background — so there is never a blank map waiting for metadata.
138
+
139
+ ```mermaid
140
+ sequenceDiagram
141
+ autonumber
142
+ participant App as Map (maplibre)
143
+ participant P as Plugin
144
+ participant HTTP as pmtiles-swarm<br/>(via CDN)
145
+ participant BT as BitTorrent swarm
146
+
147
+ App->>P: load style → tiles.json
148
+ P->>HTTP: GET /archives/{hash}/tiles.json
149
+ HTTP-->>P: TileJSON + torrent block
150
+
151
+ Note over P: claims the /archives/{hash}/ prefix,<br/>so only these URLs come to it
152
+
153
+ par Map is usable immediately
154
+ App->>P: tile 12/2145/1436
155
+ P->>HTTP: GET …/12/2145/1436.pbf
156
+ HTTP-->>App: tile bytes
157
+ and Swarm warms up in the background
158
+ P->>BT: join (.torrent — metadata already in hand)
159
+ BT-->>P: connected
160
+ P->>BT: fetch PMTiles header + root directory
161
+ BT-->>P: those pieces
162
+ Note over P: now able to resolve any tile<br/>to a byte range locally
163
+ end
164
+
165
+ App->>P: tile 12/2146/1436
166
+ Note over P: tile → byte range (PMTiles directory)<br/>→ piece index
167
+ P->>BT: fetch that piece
168
+ BT-->>P: piece
169
+ P-->>App: tile bytes, no HTTP involved
170
+
171
+ Note over P,BT: the client is now a peer —<br/>it seeds those pieces back
172
+
173
+ App->>P: tile in an unfetched region
174
+ P->>BT: fetch piece
175
+ BT--)P: too slow / unavailable
176
+ P->>HTTP: fall back for this tile
177
+ HTTP-->>App: tile bytes
178
+ ```
179
+
180
+ **Key points:** the plugin resolves tiles the same way the server does — PMTiles
181
+ directory lookup, byte range, piece index — it just does it on the device. That
182
+ is why the `torrent` block carries the archive's `.torrent` rather than per-tile
183
+ URLs: **there is nothing tile-specific in the swarm.** The swarm holds one file,
184
+ and both ends know how to read tiles out of it.
185
+
186
+ Three consequences worth being clear about:
187
+
188
+ - **HTTP is never fully abandoned.** It is the fallback for anything the swarm
189
+ cannot answer quickly, and the only path until the swarm is connected.
190
+ - **The client becomes a seeder.** Every piece it pulls, it serves — so a popular
191
+ region gets *faster* as more clients view it, which is the opposite of how a
192
+ tile server behaves under load.
193
+ - **Prefer the `.torrent` over the magnet.** A magnet carries only an infohash, so
194
+ the client must find peers and complete a metadata exchange before it knows
195
+ anything about the archive — measured at 90 to 240 seconds against a 72 GiB
196
+ archive. The `.torrent` served alongside the TileJSON already contains the
197
+ metadata and is ready immediately.
198
+
199
+ ---
200
+
201
+ ## 4. Updating an archive
202
+
203
+ New data means a new infohash, which is what makes cache invalidation free.
204
+
205
+ ```mermaid
206
+ %%{init: {"flowchart": {"rankSpacing": 60}}}%%
207
+ graph LR
208
+ NEW["new planet.pmtiles<br/><small>weekly build</small>"] --> HASH["primary re-hashes<br/>→ new infohash"]
209
+ HASH --> BEP["BEP 46 mutable entry<br/><small>same public key, seq+1</small>"]
210
+ HASH --> FEED["new RSS item"]
211
+
212
+ BEP --> FOLLOW["clients following the key<br/>see the new version"]
213
+ FEED --> SUBS["secondaries add the new torrent"]
214
+
215
+ SUBS --> URLS["tiles.json now points at<br/>/archives/{new infohash}/…"]
216
+ URLS --> CACHE["old CDN entries simply<br/>stop being referenced<br/><small>no purge needed</small>"]
217
+
218
+ linkStyle 2,4 stroke:#3F8F4F,stroke-width:2.5px;
219
+ ```
220
+
221
+ **Key points:** tile URLs are content-addressed, so they never need invalidating.
222
+ The old infohash stays valid and servable for as long as anyone still holds it —
223
+ useful for clients pinned to a known-good build — while new requests move to the
224
+ new one as soon as they re-read `tiles.json`. The only mutable thing in the whole
225
+ system is that one document.
226
+
227
+ ---
228
+
229
+ ## Deployment notes
230
+
231
+ **Decide how absolute URLs get built.** TileJSON and the RSS feed both contain
232
+ absolute URLs, and there are three ways to arrive at them:
233
+
234
+ | Config | Behaviour | Use when |
235
+ | --- | --- | --- |
236
+ | `publicUrl` set | One canonical URL, whatever the request said | There is a single public name |
237
+ | `trustProxy` set | Per request, from `X-Forwarded-Proto` and `X-Forwarded-Host` | One node answers on several names or schemes |
238
+ | neither | From the connection itself | Direct access, no proxy |
239
+
240
+ ```json
241
+ {
242
+ "trustProxy": "loopback, 10.0.0.0/8"
243
+ }
244
+ ```
245
+
246
+ `trustProxy` is what lets a single node serve **both** `https://maps.example.org`
247
+ to the internet and `http://maps.internal` to the LAN, rewriting every published
248
+ URL to match how the request arrived. It takes anything Express accepts: `true`,
249
+ a hop count, or a subnet list. Prefer the subnet list — trusting these headers
250
+ from an untrusted client lets it claim any host it likes, and that host ends up
251
+ in documents you publish.
252
+
253
+ Behind a TLS-terminating proxy with neither option set, the node advertises
254
+ `http://` URLs, and a browser that loaded the map over `https` blocks every one
255
+ as mixed content — which looks like an empty map rather than a misconfiguration.
256
+
257
+ **Load balancing needs no session affinity.** Any node serving a given infohash
258
+ returns byte-identical tiles, because the infohash pins the content. Round-robin
259
+ is fine, and a node can be added or removed mid-request-stream without a client
260
+ noticing.
261
+
262
+ **Cold start only applies to cache-mode nodes.** A node holding a complete copy
263
+ reads its local file and is fast from the first request. If every serving node
264
+ mirrors, this section does not apply to you at all.
265
+
266
+ Where nodes *do* run in cache mode, each warms its own piece cache, so scattering
267
+ requests for one region across N nodes costs N first-fetches rather than one.
268
+ Three things reduce that, in order of effect:
269
+
270
+ 1. **The CDN absorbs repeats.** Tiles are immutable, so the second request for a
271
+ tile never reaches any node.
272
+ 2. **The nodes are peers in the same swarm.** A node fetching a piece a sibling
273
+ already holds gets it *from that sibling*, usually over the LAN. Local service
274
+ discovery is on by default, so same-subnet nodes find each other without
275
+ configuration. The cost is one external fetch plus Nāˆ’1 local ones, not N
276
+ external ones.
277
+ 3. **Warm before rotating in.** `POST /api/torrents/{infohash}/warm` pre-fetches
278
+ a region so the first real request is never the slow one. See
279
+ [serving tiles](serving-tiles.md#warming-a-region).
280
+
281
+ Path-based affinity (HAProxy's `balance uri`) helps too, but it is the smallest
282
+ of the four levers and it costs you even load distribution.
283
+
284
+ **Size cache-mode disks for what gets viewed, not for the archive.** Cache usage
285
+ grows with what people actually look at and is not bounded on its own. Watch it,
286
+ and mirror instead where a full copy is affordable — a mirror is predictable,
287
+ faster, and a better peer.