pmtiles-swarm 0.60.0 → 0.62.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +222 -0
- package/NOTICE.md +13 -0
- package/README.md +72 -54
- package/docs/configuration.md +22 -2
- package/docs/internals.md +9 -2
- package/docs/serving-tiles.md +2 -2
- package/docs/tile-stacks.md +811 -0
- package/docs/tilejson.md +2 -2
- package/package.json +7 -1
- package/src/api.js +744 -3
- package/src/auth.js +2 -1
- package/src/codec.js +164 -0
- package/src/config.js +47 -2
- package/src/elevation.js +597 -0
- package/src/index.js +15 -0
- package/src/library.js +47 -6
- package/src/rgba.js +312 -0
- package/src/savepath.js +71 -0
- package/src/stack-cache.js +223 -0
- package/src/stacks.js +592 -0
- package/src/web/index.html +2416 -172
- package/src/web/public.html +53 -4
- package/stacks.json.sample +12 -0
- package/tools/resume-doctor.py +1628 -0
- package/tools/resume-experiment.py +401 -0
- package/tools/tile-bench.mjs +566 -0
package/CHANGELOG.md
CHANGED
|
@@ -7,6 +7,228 @@
|
|
|
7
7
|
### 🐞 Bug fixes
|
|
8
8
|
- _...Add new stuff here..._
|
|
9
9
|
|
|
10
|
+
## 0.62.0
|
|
11
|
+
### ✨ Features and improvements
|
|
12
|
+
- **Tile stacks: several archives served as one tile endpoint.** A stack is a
|
|
13
|
+
recipe rather than a file — an ordered list of sources, bottom first, with the
|
|
14
|
+
last painting over the ones before it. Sources are named by category, so a
|
|
15
|
+
stack follows a rebuild the way `/latest/<category>/` does, or by infohash
|
|
16
|
+
where it must not move.
|
|
17
|
+
|
|
18
|
+
What ships is the part that needs no image handling: `/stacks/<id>/tiles.json`
|
|
19
|
+
and `/stacks/<id>/{z}/{x}/{y}.<ext>`, answered by the topmost source holding
|
|
20
|
+
the tile. That is enough for the common shape — a regional archive over a
|
|
21
|
+
global one — and it costs nothing per tile beyond the read it would have done
|
|
22
|
+
anyway. `X-Stack-Sources` names which sources were asked and what each said,
|
|
23
|
+
because a stack missing a layer still renders, and flat ocean looks like a
|
|
24
|
+
plausible map rather than like a failure.
|
|
25
|
+
|
|
26
|
+
A recipe asking for masking, height shifts, opacity, blending or a different
|
|
27
|
+
output encoding answers 501 and names the field. Those need a pixel codec,
|
|
28
|
+
which this node does not have yet; approximating them would be worse than
|
|
29
|
+
refusing. See docs/tile-stacks.md.
|
|
30
|
+
|
|
31
|
+
- **A pixel codec, for the parts of a stack that are not passthrough.** `sharp`,
|
|
32
|
+
as an optional dependency probed at first use — the same library tileserver-gl
|
|
33
|
+
uses, so one image stack covers both ends of the pipeline. A node that only
|
|
34
|
+
distributes archives never needs it, and a node without it answers 501 naming
|
|
35
|
+
what to install rather than failing at the first tile.
|
|
36
|
+
|
|
37
|
+
Encoding is **lossless by default and has to be made lossy by name**. A
|
|
38
|
+
terrain-RGB pixel is not a colour: the three channels are the three bytes of
|
|
39
|
+
one height, so a lossy codec that shifts red by one moves the ground by 65
|
|
40
|
+
kilometres. Over an ordinary gradient, lossy WebP is wrong by about 125 km at
|
|
41
|
+
worst where lossless is byte-exact.
|
|
42
|
+
|
|
43
|
+
- **Elevation stacks merge for real.** A stack whose sources mask, shift or
|
|
44
|
+
re-encode is now served rather than refused: each source is decoded to metres,
|
|
45
|
+
masked, adjusted, resampled in float space and painted in the recipe's order,
|
|
46
|
+
then encoded once. A source with no tile at the requested zoom is taken from
|
|
47
|
+
its parent and cropped to the right sub-square, which is what lets a z8 global
|
|
48
|
+
source keep contributing at z14 — the passthrough path cannot do that, because
|
|
49
|
+
a parent's *bytes* are the wrong tile.
|
|
50
|
+
|
|
51
|
+
Two ways to say "no data here": `maskValues` names decoded heights, and
|
|
52
|
+
`maskColors` names pixel colours as `"#rrggbb"` or `[r, g, b]`. The colour form
|
|
53
|
+
is exact, comparing the bytes that were stored; the height form rounds, because
|
|
54
|
+
decoding produces `base + n * interval` in floating point and a mask of `-0.1`
|
|
55
|
+
meets a decoded `-0.09999999999763531`.
|
|
56
|
+
|
|
57
|
+
A tile no source covered answers 404 rather than a slab of nodata, so a client
|
|
58
|
+
overzooms a lower one. That decision is made on the coverage before nodata is
|
|
59
|
+
substituted in — afterwards every pixel holds a real value and there is nothing
|
|
60
|
+
left to test.
|
|
61
|
+
|
|
62
|
+
- **Merged tiles are cached on disk.** A merged tile costs a read of every
|
|
63
|
+
source, a decode each and an encode, and against a cache-mode source those
|
|
64
|
+
reads may go to the swarm — doing that again for a tile somebody already asked
|
|
65
|
+
for is the difference between a map that pans and one that does not. Bounded
|
|
66
|
+
by `stacks.cacheBytes` (2 GiB by default, zero to turn it off), evicted
|
|
67
|
+
least-recently-used, and indexed from disk at startup so a restart does not
|
|
68
|
+
throw the work away.
|
|
69
|
+
|
|
70
|
+
Keyed by the tile's ETag, which already covers the recipe's revision and what
|
|
71
|
+
its sources resolved to — so editing a stack or rebuilding a source produces a
|
|
72
|
+
different key rather than needing anything to remember to invalidate the old
|
|
73
|
+
one. Only the merging path is cached: passthrough already costs one read, and
|
|
74
|
+
keeping its answer would put a second copy of the archive's own bytes beside
|
|
75
|
+
the first.
|
|
76
|
+
|
|
77
|
+
Several requests for the same tile at once run one merge between them, which
|
|
78
|
+
matters because each duplicate would otherwise issue its own reads to every
|
|
79
|
+
source underneath it.
|
|
80
|
+
|
|
81
|
+
- **A stack's TileJSON declares `sparse`.** True by default, which for a stack
|
|
82
|
+
is not a guess: `maxzoom` is the deepest any source reaches, so most of the
|
|
83
|
+
pyramid below it is covered by only some of them. A tile no source covered
|
|
84
|
+
answers 404, which is what makes maplibre-gl-js and maplibre-native overzoom
|
|
85
|
+
the parent rather than draw nothing — the same flag, the same name and the
|
|
86
|
+
same rule tileserver-gl reads. A stack can set `sparse: false` to answer 204
|
|
87
|
+
instead.
|
|
88
|
+
|
|
89
|
+
- **Image stacks composite, with opacity and blend modes.** `space: "rgba"`
|
|
90
|
+
treats a tile as what it looks like rather than as packed numbers: each source
|
|
91
|
+
carries an `opacity` and a `blend` (`normal`, `multiply`, `screen`, `overlay`,
|
|
92
|
+
`darken`, `lighten`), and `maskColors` clears coverage so what is underneath
|
|
93
|
+
shows through. Hillshade over satellite is the case it exists for.
|
|
94
|
+
|
|
95
|
+
The W3C compositing model in full, not the source-over shortcut — the shortcut
|
|
96
|
+
is only correct when the backdrop is opaque, and a hillshade over a satellite
|
|
97
|
+
tile with transparent edges is exactly where that shows. Resampling from a
|
|
98
|
+
parent interpolates with alpha premultiplied, which is what stops a
|
|
99
|
+
transparent pixel dragging its colour into its neighbours and ringing
|
|
100
|
+
everything with a dark halo.
|
|
101
|
+
|
|
102
|
+
Terrain stays lossless; imagery may be compressed as a picture, which is the
|
|
103
|
+
only place the two spaces disagree about encoding.
|
|
104
|
+
|
|
105
|
+
- **A Stacks view in the console.** Every stack, what each source resolved to,
|
|
106
|
+
the zooms each covers, and — kept apart, because they call for different
|
|
107
|
+
things — what is invalid in a recipe, what cannot be served without a codec,
|
|
108
|
+
and whose sources are missing. Sources are listed in the file's order with
|
|
109
|
+
their indices, so the screen and `data/stacks.json` never disagree.
|
|
110
|
+
|
|
111
|
+
The tab is always present, including on a node with no stacks — that is where
|
|
112
|
+
a stack gets made, so hiding it until one exists would make the first one
|
|
113
|
+
unreachable. An empty state says what a stack is and how to add one.
|
|
114
|
+
|
|
115
|
+
- **`savePathLayout: "name"`**, giving each joined archive `<savePath>/<archive name>/`. The
|
|
116
|
+
infohash layout already separated two builds of the same map, but nothing in
|
|
117
|
+
`<savePath>/7fae2931a9269684a7d4ed6e5fdd7d0014e6bcd1/` tells you which map is in it. This is
|
|
118
|
+
the same separation in a directory you can find.
|
|
119
|
+
|
|
120
|
+
The name comes from the metainfo, or from a magnet's `dn=` — every magnet this node hands out
|
|
121
|
+
carries one — and there are three tiers so a placement can never fail to produce a directory.
|
|
122
|
+
A name already held by another archive takes the first eight characters of the infohash as a
|
|
123
|
+
suffix, which is the rebuild case: same name, new infohash. An archive with no usable name
|
|
124
|
+
takes its infohash, which is a bare magnet from somewhere else.
|
|
125
|
+
|
|
126
|
+
The name is written by whoever built the torrent, so it is sanitised down to one path segment
|
|
127
|
+
before it is joined to anything: separators, control characters and the characters Windows
|
|
128
|
+
refuses become `-`, leading and trailing dots go, the Windows device names get out of the way,
|
|
129
|
+
and it is cut to 120 characters. A name that survives none of that falls back to the infohash.
|
|
130
|
+
|
|
131
|
+
The directory is settled when the archive is added and never revised. A name learned later
|
|
132
|
+
over BEP 9 leaves the data where it is rather than moving hundreds of gigabytes to match a
|
|
133
|
+
tidier directory. Changing the setting places new arrivals only — everything already held
|
|
134
|
+
keeps the save path recorded for it, so nothing moves and nothing is re-checked.
|
|
135
|
+
|
|
136
|
+
- **The settings pane has tabs, and the settings in them have names and reasons.** Seven
|
|
137
|
+
groups — Serving tiles, Network, Publishing, Feeds, Engine, Transfers, Security — describing
|
|
138
|
+
67 settings as labelled fields with help text, beside one tab holding what is not described
|
|
139
|
+
yet. Nested keys are described one at a time, so `libtorrent.resumeDir` is a field with a
|
|
140
|
+
reason rather than a line inside a JSON blob.
|
|
141
|
+
|
|
142
|
+
The tables moved with them. Monitored folders, watched web locations, RSS feeds and remote
|
|
143
|
+
nodes are four ways of answering one question — where archives arrive from — and they now
|
|
144
|
+
sit together under **Feeds** rather than stacked underneath everything else. Tokens went to
|
|
145
|
+
Security, hooks and the speed and seeding limits to Transfers. Save locations went with the
|
|
146
|
+
feeds: a location is where an arriving archive lands, and the four tables above it are what
|
|
147
|
+
choose one. A
|
|
148
|
+
tab is therefore not only schema fields: `Feeds` is four tables and nothing else, which is
|
|
149
|
+
why the tabs are declared rather than derived from the schema.
|
|
150
|
+
|
|
151
|
+
What the node's own RSS says moved the other way, into Publishing. One tab called Feeds
|
|
152
|
+
holding both what this node emits and what it subscribes to was the muddle that made the
|
|
153
|
+
grouping worth doing.
|
|
154
|
+
|
|
155
|
+
Every setting the configuration declares is now described. The six that were left — the
|
|
156
|
+
WebTorrent client, the console login, DHT publishing, tile statistics, the traffic chart and
|
|
157
|
+
automatic rebuilds — were objects rendered as unlabelled JSON, so changing `traffic.keepHours`
|
|
158
|
+
meant editing JSON in a browser. The last tab now removes itself when it holds nothing, and
|
|
159
|
+
comes back on its own the moment a setting exists that the schema does not describe.
|
|
160
|
+
|
|
161
|
+
They come from a schema rather than from hand-written markup, which is what makes the rest
|
|
162
|
+
of it possible. Adding a setting used to be three edits in three places: the control, a
|
|
163
|
+
branch in the save handler, and a line in the skip-list of the generic renderer — where
|
|
164
|
+
forgetting the third showed the field twice and forgetting the second saved nothing. It is
|
|
165
|
+
now one row.
|
|
166
|
+
|
|
167
|
+
Two things fall out of that. A list-valued setting like `trackers` is edited one per line
|
|
168
|
+
instead of as JSON in a textarea. And the restart badge is per setting rather than per
|
|
169
|
+
top-level key, which is the thing the server cannot express: `tiles.maxOpenArchives` is read
|
|
170
|
+
live while `tiles.directoryCacheEntries` is not, and `RESTART_REQUIRED` has to mark the
|
|
171
|
+
whole object because the console used to edit it as one blob.
|
|
172
|
+
|
|
173
|
+
A save sends only what changed. Sending a whole group back would have reported that a
|
|
174
|
+
restart was needed on every press of Save — `JSON.stringify` keeps `null` and drops
|
|
175
|
+
`undefined`, so a group carrying either where the config has the other compares unequal
|
|
176
|
+
against a configuration nobody touched.
|
|
177
|
+
|
|
178
|
+
### 🐞 Bug fixes
|
|
179
|
+
- **A settings control that could only ever fail.** Describing every setting as a field gave
|
|
180
|
+
`allowHooksFromApi` a checkbox, and that one is refused by `saveConfig` whether it is on or
|
|
181
|
+
off — it decides whether an API token may choose what code the service runs, so it is the
|
|
182
|
+
config file's alone. Toggling it returned an error, and because everything in a save is
|
|
183
|
+
checked before anything is applied, it took the rest of that tab's settings down with it.
|
|
184
|
+
|
|
185
|
+
Fields the config file owns now render disabled with a `config file` badge, and the reader
|
|
186
|
+
skips disabled controls outright, so a stray enabled attribute cannot turn into a rejected
|
|
187
|
+
save. The hooks themselves were never affected: their editor has always disabled itself and
|
|
188
|
+
said why.
|
|
189
|
+
|
|
190
|
+
- **Two settings reported success and changed nothing.** `tiles` and `resumeSaveIntervalSeconds`
|
|
191
|
+
are read while the process starts — the tile reader's directory cache when the store is
|
|
192
|
+
built, the resume timer when it is created — and neither is consulted again. Both were
|
|
193
|
+
absent from `RESTART_REQUIRED`, so the console applied them, said so, and the node went on
|
|
194
|
+
behaving exactly as before. Being told to restart when you need not have is a small cost;
|
|
195
|
+
a setting that lies about taking effect is not.
|
|
196
|
+
|
|
197
|
+
`tiles` is blunter than it could be, deliberately: `maxOpenArchives` beside
|
|
198
|
+
`directoryCacheEntries` really is read live, but the console edits the object as a whole so
|
|
199
|
+
a badge on half of it is not expressible. That resolves when the settings pane grows real
|
|
200
|
+
fields for it.
|
|
201
|
+
|
|
202
|
+
- **`seedingCheckIntervalSeconds` is reloadable, and was neither.** The sweep reads its own
|
|
203
|
+
interval when it starts and there is already a reloader that restarts the sweep, so this
|
|
204
|
+
needed no restart and no new machinery — only to be listed beside the `seeding` object it
|
|
205
|
+
belongs to.
|
|
206
|
+
|
|
207
|
+
## 0.61.0
|
|
208
|
+
### ✨ Features and improvements
|
|
209
|
+
- **"Style URL" is now "source URL", because that is what it is.** It goes in a style's
|
|
210
|
+
`sources` block and is not itself a style, so the old name told a reader to put it in the
|
|
211
|
+
wrong place. The field on `/api/categories` and `/latest/` is `sourceUrl`; `styleUrl` is
|
|
212
|
+
still sent alongside it and is deprecated, so nothing reading the old name breaks on the
|
|
213
|
+
correction.
|
|
214
|
+
|
|
215
|
+
- **A copy button copies, rather than opening a box to copy from.** `navigator.clipboard`
|
|
216
|
+
needs a secure context and a console reached by IP over plain HTTP on a LAN is not one —
|
|
217
|
+
which is how most of them are reached — so the console fell back to `window.prompt` every
|
|
218
|
+
time. It now falls back to the selection API, which predates the clipboard API and carries
|
|
219
|
+
no such requirement, and confirms on the button itself the way the public page does. The
|
|
220
|
+
public page gained the same fallback: it was failing outright wherever the console was
|
|
221
|
+
prompting, which is the same nodes.
|
|
222
|
+
|
|
223
|
+
- **The XYZ template is offered on the public catalogue page too**, beside the TileJSON it
|
|
224
|
+
already had. 0.60.0 added it to the console only, which is the wrong way round: the console
|
|
225
|
+
is for the operator, and the person who needs a tile URL to paste into a Leaflet layer or a
|
|
226
|
+
GIS client is usually looking at the public page. Both draw from the same builder, so the
|
|
227
|
+
field was already in the public payload and only the button was missing.
|
|
228
|
+
|
|
229
|
+
### 🐞 Bug fixes
|
|
230
|
+
- _...Add new stuff here..._
|
|
231
|
+
|
|
10
232
|
## 0.60.0
|
|
11
233
|
### ✨ Features and improvements
|
|
12
234
|
- **A tile URL that survives a rebuild.** Every archive is addressed by infohash, which is
|
package/NOTICE.md
CHANGED
|
@@ -44,6 +44,19 @@ library is used unmodified through its public API; `src/web/preview.html` is our
|
|
|
44
44
|
served the same way from its own `dist`. Used through its documented options; no implementation
|
|
45
45
|
code is copied.
|
|
46
46
|
|
|
47
|
+
## sharp — Apache-2.0
|
|
48
|
+
|
|
49
|
+
> Copyright 2013 Lovell Fuller and others
|
|
50
|
+
> https://github.com/lovell/sharp
|
|
51
|
+
|
|
52
|
+
The pixel codec behind tile stacks: decoding a tile to samples and encoding the result. An
|
|
53
|
+
optional dependency, probed at first use, so a node that only distributes archives never
|
|
54
|
+
installs it — see `src/codec.js`. Used unmodified through its public API.
|
|
55
|
+
|
|
56
|
+
The prebuilt `@img/sharp-*` packages it resolves carry libvips (LGPL-3.0-or-later) and its own
|
|
57
|
+
dependencies, each under their own terms; sharp links to libvips dynamically and ships it
|
|
58
|
+
unmodified.
|
|
59
|
+
|
|
47
60
|
## qBittorrent — GPL-2.0-or-later
|
|
48
61
|
|
|
49
62
|
> https://github.com/qbittorrent/qBittorrent
|
package/README.md
CHANGED
|
@@ -237,7 +237,7 @@ each monitored folder and each watched web location:
|
|
|
237
237
|
|
|
238
238
|
```json
|
|
239
239
|
"locations": [
|
|
240
|
-
{ "name": "bulk storage", "path": "M
|
|
240
|
+
{ "name": "bulk storage", "path": "M:\\archives\\finished-unsorted" },
|
|
241
241
|
{ "name": "fast", "path": "/mnt/nvme/tiles" }
|
|
242
242
|
]
|
|
243
243
|
```
|
|
@@ -262,7 +262,18 @@ one at the same path is refused with a 409 naming the first. Where that comes up
|
|
|
262
262
|
data/torrents-data/7fae2931a9269684a7d4ed6e5fdd7d0014e6bcd1/planet.pmtiles
|
|
263
263
|
```
|
|
264
264
|
|
|
265
|
-
It works from a bare magnet, since the infohash is the one thing a magnet always carries.
|
|
265
|
+
It works from a bare magnet, since the infohash is the one thing a magnet always carries.
|
|
266
|
+
`"savePathLayout": "name"` gives the same separation in a directory you can find without knowing
|
|
267
|
+
the infohash:
|
|
268
|
+
|
|
269
|
+
```
|
|
270
|
+
data/torrents-data/planet.pmtiles/planet.pmtiles
|
|
271
|
+
```
|
|
272
|
+
|
|
273
|
+
The name comes from the metainfo, or from a magnet's `dn=` — every magnet this node hands out
|
|
274
|
+
carries one — and is sanitised down to a single path segment before it is used. A second archive
|
|
275
|
+
of the same name takes `planet.pmtiles-7fae2931`, and one joined by a bare magnet, which carries
|
|
276
|
+
no name at all, takes its infohash. Flat
|
|
266
277
|
stays the default: it is what makes dropping a finished archive into the save path before adding
|
|
267
278
|
its torrent work, and it keeps a served filename readable. Archives _created_ from a local file
|
|
268
279
|
are unaffected either way — they keep the file they were made from — and web seed URLs are built
|
|
@@ -739,58 +750,65 @@ which the endpoint answers 501.
|
|
|
739
750
|
|
|
740
751
|
## API
|
|
741
752
|
|
|
742
|
-
| Method
|
|
743
|
-
|
|
|
744
|
-
| `GET`
|
|
745
|
-
| `GET`
|
|
746
|
-
| `POST`
|
|
747
|
-
| `DELETE`
|
|
748
|
-
| `GET`
|
|
749
|
-
| `GET`
|
|
750
|
-
| `GET`
|
|
751
|
-
| `GET`
|
|
752
|
-
| `PATCH`
|
|
753
|
-
| `PATCH`
|
|
754
|
-
| `PATCH`
|
|
755
|
-
| `PATCH` `GET`
|
|
756
|
-
| `POST`
|
|
757
|
-
| `POST`
|
|
758
|
-
| `POST`
|
|
759
|
-
| `DELETE`
|
|
760
|
-
| `POST`
|
|
761
|
-
| `POST`
|
|
762
|
-
| `DELETE`
|
|
763
|
-
| `POST`
|
|
764
|
-
| `POST`
|
|
765
|
-
| `POST`
|
|
766
|
-
| `GET` `DELETE`
|
|
767
|
-
| `GET` `POST`
|
|
768
|
-
| `GET`
|
|
769
|
-
| `POST`
|
|
770
|
-
| `POST`
|
|
771
|
-
| `POST`
|
|
772
|
-
| `POST`
|
|
773
|
-
| `POST`
|
|
774
|
-
| `POST`
|
|
775
|
-
| `GET` `POST` `DELETE`
|
|
776
|
-
| `GET` `POST`
|
|
777
|
-
| `GET` `DELETE`
|
|
778
|
-
| `GET`
|
|
779
|
-
| `GET` `PATCH`
|
|
780
|
-
| `POST`
|
|
781
|
-
| `GET`
|
|
782
|
-
| `GET`
|
|
783
|
-
| `GET`
|
|
784
|
-
| `GET`
|
|
785
|
-
| `GET`
|
|
786
|
-
| `GET`
|
|
787
|
-
| `GET`
|
|
788
|
-
| `GET`
|
|
789
|
-
| `GET`
|
|
790
|
-
| `GET`
|
|
791
|
-
| `GET`
|
|
792
|
-
| `GET`
|
|
793
|
-
| `GET`
|
|
753
|
+
| Method | Path | Purpose |
|
|
754
|
+
| ---------------------- | ----------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
|
|
755
|
+
| `GET` | `/api/status` | Engine health, counts, watched folders, save locations, free space, and whether the swarm can reach this node |
|
|
756
|
+
| `GET` | `/api/torrents` | Catalog joined with live swarm state and what is left of each seeding limit |
|
|
757
|
+
| `POST` | `/api/torrents` | Add via `{path}`, `{url}`, `{magnet}`, `{torrentUrl}`, or a raw `.torrent` body |
|
|
758
|
+
| `DELETE` | `/api/torrents/:infoHash` | Remove (`?deleteData=true` to delete data too) |
|
|
759
|
+
| `GET` | `/api/torrents/:infoHash` | One archive, with disk usage and how it is being read |
|
|
760
|
+
| `GET` | `/api/torrents/:infoHash/file`, `/magnet` | Download the `.torrent`, or its magnet URI |
|
|
761
|
+
| `GET` | `/api/torrents/:infoHash/peers`, `/trackers`, `/content` | Per-peer, per-tracker and per-file detail |
|
|
762
|
+
| `GET` | `/api/torrents/:infoHash/pieces` | Which pieces are held, how rare each is, and what peers hold |
|
|
763
|
+
| `PATCH` | `/api/torrents/:infoHash/mode` | Switch between mirror and cache |
|
|
764
|
+
| `PATCH` | `/api/torrents/:infoHash/categories` | Set, add or remove categories |
|
|
765
|
+
| `PATCH` | `/api/torrents/:infoHash/seeding` | Per-archive seeding limit, or "use the global one" |
|
|
766
|
+
| `PATCH` `GET` | `/api/torrents/:infoHash/location` | Move the data; poll the move |
|
|
767
|
+
| `POST` | `/api/torrents/:infoHash/recheck` | Hash what is on disk again and believe the result — for an archive whose progress and whose files disagree |
|
|
768
|
+
| `POST` | `/api/torrents/:infoHash/pause`, `/resume` | Stop offering it, without forgetting it |
|
|
769
|
+
| `POST` | `/api/torrents/:infoHash/webseeds` | Add web seeds — does not change the infohash |
|
|
770
|
+
| `DELETE` | `/api/torrents/:infoHash/webseeds` | Drop web seeds — likewise |
|
|
771
|
+
| `POST` | `/api/torrents/:infoHash/publish` | What this node offers of the archive itself: `serveArchive`, `selfWebSeed`, `publicDownload` |
|
|
772
|
+
| `POST` | `/api/torrents/:infoHash/warm` | Pre-fetch a region (`GET` for progress, `DELETE` to cancel) |
|
|
773
|
+
| `DELETE` | `/api/torrents/:infoHash/cache` | Reclaim cached pieces, keep the archive |
|
|
774
|
+
| `POST` | `/api/torrents/:infoHash/check` | Has the source changed since the torrent was made? |
|
|
775
|
+
| `POST` | `/api/torrents/:infoHash/rebuild` | Rebuild from the current source (mints a new infohash) |
|
|
776
|
+
| `POST` | `/api/check-origins` | Check every archive with a watchable source |
|
|
777
|
+
| `GET` `DELETE` | `/api/adds` | Adds still in flight — downloads, and local files being hashed — and cancelling a download by URL. A hash cannot be stopped part-way, so it is listed but not cancellable |
|
|
778
|
+
| `GET` `POST` | `/api/speed` | Which speed limits are in force, and the manual switch between the two sets |
|
|
779
|
+
| `GET` | `/api/categories` | Every category, with the endpoints resolving to its newest build |
|
|
780
|
+
| `POST` | `/api/adopt`, `/api/adopt/candidates` | Take over what an engine or another node holds |
|
|
781
|
+
| `POST` | `/api/sources/preview` | What a watched web location would take, without taking it |
|
|
782
|
+
| `POST` | `/api/subscriptions/preview` | Whether a peer is reachable and what it offers |
|
|
783
|
+
| `POST` | `/api/subscriptions/refresh` | Poll subscribed feeds now |
|
|
784
|
+
| `POST` | `/api/sources/check` | Check scheduled sources now, rather than at the next due time |
|
|
785
|
+
| `POST` | `/api/torrents/:infoHash/hooks/complete` | Run the completion hook again for one archive |
|
|
786
|
+
| `GET` `POST` `DELETE` | `/api/tokens`, `/api/tokens/:id` | Mint, list and revoke access tokens |
|
|
787
|
+
| `GET` `POST` | `/api/restart` | What a restart would do, and doing it |
|
|
788
|
+
| `GET` `DELETE` | `/api/stats` | What this node has served — per-archive counters and the last N requests; `DELETE` clears them |
|
|
789
|
+
| `GET` | `/api/traffic` | Upload and download speed per archive over time, from `stats.db`; `infoHash`, `hours` and `buckets` narrow it, and `totals` ranks archives by bytes moved |
|
|
790
|
+
| `GET` `PATCH` | `/api/config` | Read and change settings |
|
|
791
|
+
| `POST` | `/api/login`, `/api/logout` | Console sign-in |
|
|
792
|
+
| `GET` | `/api/session` | Who this request is, and whether a credential is needed at all |
|
|
793
|
+
| `GET` | `/api/catalog` | The whole catalogue, for a peer keeping itself in step |
|
|
794
|
+
| `GET` | `/archives/:infoHash/tiles.json` | TileJSON — **public** |
|
|
795
|
+
| `GET` | `/archives/:infoHash/:z/:x/:y.:ext` | One tile — **public** |
|
|
796
|
+
| `GET` | `/health` | 200 when this node can serve, 503 when its engine cannot — **public**, no credential, for a load balancer |
|
|
797
|
+
| `GET` | `/archives/:infoHash/ready` | Whether this node can serve _this_ archive yet: 200 ready, 503 not yet, 415 never — **public** |
|
|
798
|
+
| `GET` | `/archives/:infoHash/archive.torrent` | The `.torrent` a torrent-aware client joins with — **public** |
|
|
799
|
+
| `GET` | `/archives/:infoHash/archive.pmtiles` | The archive itself, by byte range — what any PMTiles reader wants, and a valid web seed. Only where [`serveArchive`](docs/configuration.md#servearchive) is on — **public**. Complete archives, unless [`serveArchiveFromSwarm`](docs/configuration.md#servearchivefromswarm) is set, which answers a bounded range from the swarm instead |
|
|
800
|
+
| `GET` | `/archives/:infoHash/preview` | Map preview for one archive — admin, not public |
|
|
801
|
+
| `GET` | `/latest/` | Every category and the endpoints that resolve to its newest build — **public**, the index of everything below |
|
|
802
|
+
| `GET` | `/latest/:category/tiles.json`, `/:name.torrent`, `/magnet` | The newest build in a category — **public**. The torrent name is yours to choose, so a link can read `planetiler-openmaptiles-latest.torrent`; it redirects to the immutable URL, which names the download after the build it actually is |
|
|
803
|
+
| `GET` | `/latest/:category/archive.pmtiles` | The newest build in a category as a file, by byte range — **public**, and gated by [`serveArchive`](docs/configuration.md#servearchive). What to point a PMTiles reader at when you want "whichever is current" rather than one build |
|
|
804
|
+
| `GET` | `/api/stacks` | Every stack, with what each source resolved to and why one cannot be served yet |
|
|
805
|
+
| `GET` | `/api/torrents/:infoHash/stacks` | Which stacks would break if this archive were removed, and how |
|
|
806
|
+
| `GET`, `PUT`, `DELETE` | `/api/stacks/:id/raw`, `/api/stacks/:id` | Read a recipe as written, save one, or remove it. `/raw` is the recipe; `/api/stacks` is what it resolved to |
|
|
807
|
+
| `GET` | `/stacks/:id/preview` | A map of a stack, for looking at it — **public** |
|
|
808
|
+
| `GET` | `/stacks/:id/tiles.json` | TileJSON for a stack — **public**. `maxzoom` is the maximum over its sources, not the minimum |
|
|
809
|
+
| `GET` | `/stacks/:id/:size/:z/:x/:y.:ext` | The same tile at 256 or 512 px. A tile's coordinates are an extent rather than a pixel count, so this is the same ground on a finer or coarser grid |
|
|
810
|
+
| `GET` | `/stacks/:id/:z/:x/:y.:ext` | One tile of a stack — **public**. Answered by the topmost source holding it; `X-Stack-Sources` says which were asked and what each said |
|
|
811
|
+
| `GET` | `/feed.xml`, `/feed/:category.xml`, `/latest/:category.xml` | RSS — **public** |
|
|
794
812
|
|
|
795
813
|
Everything under `/api/` is guarded once a credential is configured; tiles, TileJSON and the feeds
|
|
796
814
|
never are. A `peer` token may read but not change, and may be narrowed to some categories. See
|
package/docs/configuration.md
CHANGED
|
@@ -95,7 +95,7 @@ the question entirely. See [haproxy.md](haproxy.md).
|
|
|
95
95
|
| `dataDir` | `'./data'` | catalog, generated `.torrent` files and keys |
|
|
96
96
|
| `savePath` | unset | where archive data lives |
|
|
97
97
|
| `cacheSavePath` | unset | separate path for cache-mode pieces |
|
|
98
|
-
| `savePathLayout` | `'flat'` | `'flat'` or `'
|
|
98
|
+
| `savePathLayout` | `'flat'` | `'flat'`, `'infohash'` or `'name'` |
|
|
99
99
|
| `locations` | `[]` | named places for data to land: `[{ name, path }]` |
|
|
100
100
|
| `incompleteSuffix` | `'.incomplete'` | marker on an archive that is not whole yet |
|
|
101
101
|
| `completionCheckIntervalSeconds` | `15` | how often to look for finished downloads |
|
|
@@ -133,6 +133,8 @@ measurable and clearable as a directory.
|
|
|
133
133
|
- `'infohash'` — each archive under `<savePath>/<infohash>/`. Two builds of the
|
|
134
134
|
same map are both `planet.pmtiles`, and this is the only arrangement in which
|
|
135
135
|
that can never matter.
|
|
136
|
+
- `'name'` — each archive under `<savePath>/<archive name>/`. The same
|
|
137
|
+
separation, in a directory you can find without knowing the infohash.
|
|
136
138
|
|
|
137
139
|
Flat by default, because the collision it avoids is now refused outright when the
|
|
138
140
|
second archive is added — so the cost of flat is an error message at the moment
|
|
@@ -143,6 +145,24 @@ Only joined archives are placed. One created here keeps the file it was made
|
|
|
143
145
|
from, and web seed URLs are built from the published location rather than from
|
|
144
146
|
the save path.
|
|
145
147
|
|
|
148
|
+
Changing this places new arrivals only. Everything already held keeps the save
|
|
149
|
+
path recorded for it, so nothing moves on disk and nothing has to be re-checked.
|
|
150
|
+
|
|
151
|
+
Under `'name'` the directory is settled when the archive is added, from whatever
|
|
152
|
+
name is known then, and never revised afterwards:
|
|
153
|
+
|
|
154
|
+
- The name is taken from the metainfo, or from a magnet's `dn=`. Every magnet
|
|
155
|
+
this node hands out carries one.
|
|
156
|
+
- A **bare magnet** carries no name, so that archive gets its infohash instead.
|
|
157
|
+
The name learned later over BEP 9 does not move it — the data is already
|
|
158
|
+
there, and a rename to tidy the directory is not worth a re-check of it.
|
|
159
|
+
- A name already held by another archive gets the first eight characters of the
|
|
160
|
+
infohash appended, which is the rebuild case: same name, new infohash.
|
|
161
|
+
- The name is written by whoever built the torrent, so it is sanitised down to
|
|
162
|
+
one path segment before it is used. Separators, control characters and the
|
|
163
|
+
ones Windows refuses become `-`; a name that survives none of that falls back
|
|
164
|
+
to the infohash.
|
|
165
|
+
|
|
146
166
|
### `incompleteSuffix`
|
|
147
167
|
|
|
148
168
|
Set to an empty string to switch it off, in which case a partial archive is
|
|
@@ -407,7 +427,7 @@ on. This is that address.
|
|
|
407
427
|
Narrower than `publicUrl` on purpose: `publicUrl` overrides every URL the node
|
|
408
428
|
emits and so gives up the multi-domain behaviour, while this overrides only the
|
|
409
429
|
ones that have to be permanent. Everything else — TileJSON, tile templates,
|
|
410
|
-
`.torrent` links,
|
|
430
|
+
`.torrent` links, source URLs, the feeds — goes on naming whichever host the
|
|
411
431
|
request arrived as.
|
|
412
432
|
|
|
413
433
|
```json
|
package/docs/internals.md
CHANGED
|
@@ -103,6 +103,13 @@ writing pieces into one file, and neither ends up with the archive it thinks it
|
|
|
103
103
|
has. This is caught when the second archive is added, where it can still be
|
|
104
104
|
answered by choosing somewhere else to put it.
|
|
105
105
|
|
|
106
|
+
`savePathLayout` is the standing answer rather than the per-archive one. Under
|
|
107
|
+
`'infohash'` the collision cannot arise. Under `'name'` it still can — the names
|
|
108
|
+
are what collide — so the second archive takes `<name>-<first eight of the
|
|
109
|
+
infohash>`, and an archive with no usable name at all takes its infohash. Both
|
|
110
|
+
are decided once, when the archive is added; a name learned later over BEP 9
|
|
111
|
+
leaves the directory alone rather than moving the data to match it.
|
|
112
|
+
|
|
106
113
|
### Staging, for an archive fetched from a URL
|
|
107
114
|
|
|
108
115
|
An archive fetched over HTTP has no infohash while it is being fetched — the
|
|
@@ -725,7 +732,7 @@ own and asks for nothing guarded.
|
|
|
725
732
|
|
|
726
733
|
Three things joined the public list to make it work, and each is a read of
|
|
727
734
|
something already published: `/api/categories`, which groups the same archives
|
|
728
|
-
and carries the
|
|
735
|
+
and carries the source URL for each; the per-archive `/preview`; and `/vendor/`,
|
|
729
736
|
which is the MapLibre bundle the preview renders with. The preview used to be
|
|
730
737
|
excluded on the grounds that it is console furniture and would not render
|
|
731
738
|
without `/vendor` anyway — both true, and both answered by publishing the pair
|
|
@@ -868,7 +875,7 @@ carry several categories on purpose — a planet build is both `basemaps` and
|
|
|
868
875
|
answer.
|
|
869
876
|
|
|
870
877
|
So the location is chosen rather than derived, and naming them is what makes that
|
|
871
|
-
bearable: `M:\
|
|
878
|
+
bearable: `M:\archives\finished-unsorted` is not something anyone should retype, and a
|
|
872
879
|
name survives the path changing underneath it.
|
|
873
880
|
|
|
874
881
|
Only new data is placed. An archive records where it was put and keeps it, so
|
package/docs/serving-tiles.md
CHANGED
|
@@ -45,7 +45,7 @@ See [internals.md](internals.md#serving-an-mbtiles-archive).
|
|
|
45
45
|
|
|
46
46
|
`GET /latest/` lists every category this node publishes, with the endpoints
|
|
47
47
|
that resolve to each one's newest build — the TileJSON, the `.torrent`, the
|
|
48
|
-
magnet, the per-category feed, and the
|
|
48
|
+
magnet, the per-category feed, and the source URL with the magnet in its
|
|
49
49
|
fragment.
|
|
50
50
|
|
|
51
51
|
Public, and deliberately so. Everything else under `/latest/` is — the
|
|
@@ -274,7 +274,7 @@ everywhere:
|
|
|
274
274
|
| torrent-aware | joins **before any network call**, and still can if the fetch fails |
|
|
275
275
|
|
|
276
276
|
The console's **Copy TileJSON URL + swarm** button produces exactly this, and so
|
|
277
|
-
does the `
|
|
277
|
+
does the `sourceUrl` on every row of `/api/categories` and `/latest/`.
|
|
278
278
|
|
|
279
279
|
### Two handles, and why both
|
|
280
280
|
|