pmtiles-swarm 0.99.2 ā 0.101.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +119 -0
- package/docs/tile-stacks.md +111 -0
- package/package.json +3 -1
- package/src/api.js +289 -0
- package/src/bake-jobs.js +165 -6
- package/src/bake-selection.js +257 -0
- package/src/bake.js +85 -4
- package/src/config.js +23 -0
- package/src/contour-mvt.js +213 -0
- package/src/contour-options.js +164 -0
- package/src/contour-tile.js +280 -0
- package/src/elevation.js +41 -0
- package/src/heights-cache.js +157 -0
- package/src/index.js +10 -0
- package/src/stack-tile.js +145 -3
- package/src/stacks.js +24 -1
- package/src/web/index.html +243 -2
- package/src/web/preview.html +50 -2
- package/src/web/public.html +1 -0
package/CHANGELOG.md
CHANGED
|
@@ -7,6 +7,125 @@
|
|
|
7
7
|
### š Bug fixes
|
|
8
8
|
- _...Add new stuff here..._
|
|
9
9
|
|
|
10
|
+
## 0.101.0
|
|
11
|
+
### ⨠Features and improvements
|
|
12
|
+
- **Contours, traced from what a stack merges.** `GET /stacks/<id>/contours/{z}/{x}/{y}.pbf` returns
|
|
13
|
+
a gzipped vector tile of contour lines. The interval comes from the recipe's `contours.thresholds`
|
|
14
|
+
or from `?interval=` on the request ā contours are a view of a stack rather than a property of one,
|
|
15
|
+
and the same terrain is wanted at 10 m on a walking map and 100 m on an atlas.
|
|
16
|
+
|
|
17
|
+
Drawn here rather than from an archive for one reason. A tool reading archives has to answer "what
|
|
18
|
+
is the elevation?" for ground no archive covers, and an encoded terrain tile cannot say
|
|
19
|
+
"nothing" ā every triple of bytes is a height. So it invents one, and a constant beside real
|
|
20
|
+
terrain is a cliff, which a tracer renders as lines packed tight along the seam. A stack has `NaN`
|
|
21
|
+
and hands its holes over unfilled, so a line stops at the edge of the data instead.
|
|
22
|
+
|
|
23
|
+
A level may name several intervals. `[100, 500]` draws every hundred metres and marks every fifth,
|
|
24
|
+
and each feature carries `level` ā how many intervals its height divides by ā so a style draws the
|
|
25
|
+
major lines thicker and labels only those, from one layer. That is `maplibre-contour`'s convention,
|
|
26
|
+
so a style written for its tiles works against these.
|
|
27
|
+
|
|
28
|
+
Worth knowing what it costs: a contour tile is traced from its own tile **plus its eight
|
|
29
|
+
neighbours**, because a line crossing an edge has to be traced from the ground on both sides or it
|
|
30
|
+
will not meet the line next door. Roughly nine merged terrain tiles each. See
|
|
31
|
+
[tile-stacks.md](docs/tile-stacks.md) ā "Contours from a stack".
|
|
32
|
+
|
|
33
|
+
A **Contours** button sits beside Preview and Terrain on any stack that draws as terrain, and the
|
|
34
|
+
preview draws the lines over the hillshade rather than instead of it ā a contour on its own says
|
|
35
|
+
nothing about which side of it is uphill. Major lines draw themselves thicker from `level`, with
|
|
36
|
+
no second layer and no filter. There are no labels: text needs a glyph server, and a preview that
|
|
37
|
+
fetched fonts from elsewhere would be a preview of that as much as of the archive. `ele` is on
|
|
38
|
+
every feature for a style that has fonts to spend.
|
|
39
|
+
|
|
40
|
+
**An export can write them too.** The export dialog's *What to make* list has its second entry,
|
|
41
|
+
and `POST /api/stacks/<id>/bake` takes `kind: "contours"` with an optional `thresholds`. The
|
|
42
|
+
archive is written as gzipped MVT with the right header, and the run narrows itself to the zooms
|
|
43
|
+
its thresholds actually draw at ā without that it walks every tile the sources hold at z0āz8 to
|
|
44
|
+
trace nothing, which on a planet is hours spent producing silence.
|
|
45
|
+
|
|
46
|
+
The interval is part of the export's revision, so a checkpoint cannot be resumed across a change
|
|
47
|
+
to it: a 20 m run continued into a 100 m one would splice two sets of lines into an archive
|
|
48
|
+
nothing downstream could tell apart.
|
|
49
|
+
|
|
50
|
+
**Merged heights are cached in memory**, under `stacks.heightsCacheBytes` (64 MiB, zero to turn
|
|
51
|
+
it off). A different cache from `stacks.cacheBytes` and for a different shape of reuse: that one
|
|
52
|
+
holds encoded tiles so a second request for one tile is free, this holds the numbers a tile was
|
|
53
|
+
merged from, because several *different* tiles are built out of the same neighbours. An NĆN
|
|
54
|
+
contour run needs (N+2)² merges and asks for 9N² ā measured at 4Ć fewer source reads and twice
|
|
55
|
+
the speed on a 4Ć4 block, approaching 9Ć on a large one. A feathered source benefits more mildly,
|
|
56
|
+
since four sibling tiles share the parents its ramp is measured against.
|
|
57
|
+
|
|
58
|
+
Each contour export gets its own, which goes when the job does. A bake deliberately does not
|
|
59
|
+
touch the disk cache ā a planet export would evict the serving node's entire cache with tiles
|
|
60
|
+
nobody will ask for again ā and that is exactly why it can have one of its own.
|
|
61
|
+
|
|
62
|
+
**Contours from an archive or a category too**, at `/archives/<infohash>/contours/{z}/{x}/{y}.pbf`
|
|
63
|
+
and `/latest/<category>/contours/{z}/{x}/{y}.pbf`. An archive is already terrain ā its pixels are
|
|
64
|
+
a packed height and it states the packing in its own metadata ā so these decode the tile and do
|
|
65
|
+
nothing else: no recipe to resolve, no sources to merge, no masks or clips, because there is no
|
|
66
|
+
recipe saying to. Nine reads a tile against the stack path's nine merges.
|
|
67
|
+
|
|
68
|
+
They also do not climb to a parent where the archive has no tile at that zoom, which the stack
|
|
69
|
+
path would: for contours that is the wrong favour, since a line traced from an upscaled parent is
|
|
70
|
+
the parent's line drawn twice as thick rather than detail the zoom has. An infohash is immutable
|
|
71
|
+
and cached as such; a category revalidates, and is tagged by the build it resolved to so a
|
|
72
|
+
rebuild lands as a new tag rather than the same one with different lines behind it.
|
|
73
|
+
|
|
74
|
+
The **Contours** button now appears anywhere terrain does ā a stack's row, an archive's panel, and
|
|
75
|
+
the public catalogue ā rather than on stacks alone. The preview rewrites its own path into the
|
|
76
|
+
contour endpoint, which is the same rewrite for all three.
|
|
77
|
+
- **An export can write part of a stack.** A zoom range, an area, or both, from the export dialog or
|
|
78
|
+
as `minzoom` / `maxzoom` / `bounds` on `POST /api/stacks/<id>/bake`. Absent still means all of it.
|
|
79
|
+
An export reads every tile its sources hold, which for a planet is hours and a file nobody wanted
|
|
80
|
+
all of.
|
|
81
|
+
|
|
82
|
+
The two narrow differently. PMTiles orders tile ids by zoom and then along a Hilbert curve, so a
|
|
83
|
+
zoom range is a contiguous run of ids and the scan **ends** at the deepest zoom asked for rather
|
|
84
|
+
than filtering past it ā which matters, because not enumerating unwanted zooms is why the export
|
|
85
|
+
iterates coverage in the first place. A box is not contiguous, so it is a test per tile; it runs
|
|
86
|
+
before the merge, so what it saves is the merge.
|
|
87
|
+
|
|
88
|
+
A tile is written when it **overlaps** the box, not when it sits inside it, so an archive reaches
|
|
89
|
+
its own edge instead of stopping up to a tile short. The dialog can also fill the box in from a
|
|
90
|
+
`z/x/y` tile, which is how a planet is usually split: regions that tile evenly, never overlap, and
|
|
91
|
+
have a name to agree on.
|
|
92
|
+
|
|
93
|
+
Three things move with the selection, each a silent fault otherwise. The **revision**, so a
|
|
94
|
+
checkpoint taken under one selection is not resumed under another ā the stream of ids is different
|
|
95
|
+
and resuming would skip whatever the new selection adds below the mark. Both **names**, since two
|
|
96
|
+
exports of one recipe over different ground are two archives and a date does not tell them apart.
|
|
97
|
+
And the header's **bounds**, which the writer defaults to the whole world: an archive claiming a
|
|
98
|
+
planet and holding one country is one a client keeps asking for tiles that were never written. The
|
|
99
|
+
zooms are still read off the tiles actually written, which is more honest than the request.
|
|
100
|
+
|
|
101
|
+
### š Bug fixes
|
|
102
|
+
|
|
103
|
+
## 0.100.0
|
|
104
|
+
### ⨠Features and improvements
|
|
105
|
+
- **A nested stack takes `maskColors`, and fades into what is under it.** Two things a stack could
|
|
106
|
+
not do that every other kind of source could, both for the same reason: it is evaluated rather
|
|
107
|
+
than stored, so it has no bytes. That is a fact about storage, not about meaning, and it was
|
|
108
|
+
showing up in the recipe as a source with a different set of options.
|
|
109
|
+
|
|
110
|
+
A colour is now decoded into the height it names, under the encoding the inner stack packs its
|
|
111
|
+
own output in, and masked as a height ā through `decodeHeights` rather than arithmetic written a
|
|
112
|
+
second time. `maskColors: ["#0186a0"]` and `maskValues: [0]` say the same thing to a
|
|
113
|
+
mapbox-encoded stack, so a source keeps its mask when it is swapped between an archive and a
|
|
114
|
+
stack. `encoding`, `baseVal`, `interval` and the custom factors stay refused: those describe how
|
|
115
|
+
to unpack channels into a number, and the number arrived already made.
|
|
116
|
+
|
|
117
|
+
Feathering a nested source now works rather than silently doing nothing. A ramp is measured in
|
|
118
|
+
pixels and the pixels that say how far a hole reaches are partly in the next tile ā which for an
|
|
119
|
+
archive means reading its parent and for a stack means evaluating it again, which was never
|
|
120
|
+
implemented. Validation also stopped asking a nested source for a mask before it would accept a
|
|
121
|
+
feather: its holes are already an edge, and the mask being demanded would have made a second one.
|
|
122
|
+
|
|
123
|
+
The representation is unchanged. A nested stack is still merged as heights, because a hole is
|
|
124
|
+
`NaN` and no encoding has one ā encoding it would turn every hole into a sentinel the recipe
|
|
125
|
+
above had to mask back out, which is the problem nesting avoids by construction.
|
|
126
|
+
|
|
127
|
+
### š Bug fixes
|
|
128
|
+
|
|
10
129
|
## 0.99.2
|
|
11
130
|
### ⨠Features and improvements
|
|
12
131
|
|
package/docs/tile-stacks.md
CHANGED
|
@@ -55,6 +55,7 @@ of its parts.
|
|
|
55
55
|
- [What a mask has to match](#what-a-mask-has-to-match)
|
|
56
56
|
- [Feathering a seam](#feathering-a-seam)
|
|
57
57
|
- [A stack as a source](#a-stack-as-a-source)
|
|
58
|
+
- [A colour is a height said another way](#a-colour-is-a-height-said-another-way)
|
|
58
59
|
- [A source read straight from a URL](#a-source-read-straight-from-a-url)
|
|
59
60
|
- [Importing a list of URLs](#importing-a-list-of-urls)
|
|
60
61
|
- [Exporting on a schedule](#exporting-on-a-schedule)
|
|
@@ -62,6 +63,7 @@ of its parts.
|
|
|
62
63
|
- [Syncing a stack to another node](#syncing-a-stack-to-another-node)
|
|
63
64
|
- [What the offline merge got wrong](#what-the-offline-merge-got-wrong)
|
|
64
65
|
- [The stack editor](#the-stack-editor)
|
|
66
|
+
- [Contours from a stack](#contours-from-a-stack)
|
|
65
67
|
- [Merging vector sources](#merging-vector-sources)
|
|
66
68
|
- [Staging](#staging)
|
|
67
69
|
- [Open questions](#open-questions)
|
|
@@ -1539,6 +1541,27 @@ height mask has to round; a stack that was never stored has no such channels to
|
|
|
1539
1541
|
compare, and masking the heights it decoded to would be a different operation
|
|
1540
1542
|
wearing the same name.
|
|
1541
1543
|
|
|
1544
|
+
### A colour is a height said another way
|
|
1545
|
+
|
|
1546
|
+
`maskColors` compares the bytes a source was stored as, and a nested stack has
|
|
1547
|
+
none ā it is evaluated, so what arrives is already metres. Refusing the field
|
|
1548
|
+
there would leave one kind of source unable to say a thing every other kind can,
|
|
1549
|
+
for a reason that is about storage rather than about meaning.
|
|
1550
|
+
|
|
1551
|
+
So the colour is decoded into the height it names, under the encoding the inner
|
|
1552
|
+
stack packs its own output in, and masked as a height. Through `decodeHeights`
|
|
1553
|
+
rather than arithmetic written a second time: it is the same question the merge
|
|
1554
|
+
asks of every pixel, and a second copy of it is a second thing to keep in step
|
|
1555
|
+
with `terrarium` and the custom factors.
|
|
1556
|
+
|
|
1557
|
+
`maskColors: ["#0186a0"]` and `maskValues: [0]` therefore say the same thing to
|
|
1558
|
+
a mapbox-encoded stack, which is what lets a source keep its mask when it is
|
|
1559
|
+
swapped between an archive and a stack.
|
|
1560
|
+
|
|
1561
|
+
What stays refused is the rest of that family ā `encoding`, `baseVal`,
|
|
1562
|
+
`interval` and the custom factors. Those describe how to unpack channels into a
|
|
1563
|
+
number, and there are no channels: the number arrived already made.
|
|
1564
|
+
|
|
1542
1565
|
### Nothing passes through
|
|
1543
1566
|
|
|
1544
1567
|
The short-circuit that hands back a source's own bytes cannot apply: there are
|
|
@@ -2298,6 +2321,94 @@ Debounced, and it should not follow the map past the stack's `maxzoom`: above
|
|
|
2298
2321
|
that the client overzooms and there is nothing new to see, but the tiles are
|
|
2299
2322
|
still requested and still composited.
|
|
2300
2323
|
|
|
2324
|
+
## Contours from a stack
|
|
2325
|
+
|
|
2326
|
+
A contour tile is drawn from the heights a stack merges, not from an archive of
|
|
2327
|
+
terrain. That distinction is the whole reason to do it here.
|
|
2328
|
+
|
|
2329
|
+
A tool that reads archives has to answer "what is the elevation here?" for
|
|
2330
|
+
ground no archive covers, and an encoded terrain tile has no way to say
|
|
2331
|
+
"nothing" ā every triple of bytes is some height. So it invents one. A constant
|
|
2332
|
+
beside real terrain is a **cliff**, and a cliff under a contour tracer comes out
|
|
2333
|
+
as lines packed arbitrarily tight along the seam. Filling with `-10000` rather
|
|
2334
|
+
than `0` only moves the cliff and makes it taller. That is not a fault in the
|
|
2335
|
+
tool; it is the shape of its input.
|
|
2336
|
+
|
|
2337
|
+
A stack has `NaN`, and `stackHeights` deliberately leaves its holes unfilled ā
|
|
2338
|
+
the same property that lets one stack show through another. `maplibre-contour`
|
|
2339
|
+
already understands it: `HeightTile.fromRawDem` maps anything invalid to `NaN`,
|
|
2340
|
+
the tracer skips it, and `combineNeighbors` answers `NaN` for a neighbour that
|
|
2341
|
+
is missing altogether rather than throwing. So a tile the stack covers none of
|
|
2342
|
+
is passed as `undefined` and reads as no-data. Nothing is invented anywhere, and
|
|
2343
|
+
a line stops at the edge of the data instead of diving off a cliff.
|
|
2344
|
+
|
|
2345
|
+
Where you want contours to run on past the coast, put a base under them ā the
|
|
2346
|
+
merge is what makes them continuous, and `featherMetres` is what stops the seam
|
|
2347
|
+
between two sources becoming its own little cliff. For contours that feather is
|
|
2348
|
+
load-bearing rather than cosmetic.
|
|
2349
|
+
|
|
2350
|
+
### Nine tiles per tile
|
|
2351
|
+
|
|
2352
|
+
A contour crossing a tile edge has to be traced from the ground on both sides,
|
|
2353
|
+
or it will not meet the line in the next tile. So a contour tile is drawn from
|
|
2354
|
+
its own tile plus its eight neighbours: **roughly nine merged terrain tiles
|
|
2355
|
+
each**, asked for together rather than in turn.
|
|
2356
|
+
|
|
2357
|
+
Every one of those is a tile some neighbouring contour tile also wants, so a
|
|
2358
|
+
cache in front of the merge is what decides whether a run of these is affordable
|
|
2359
|
+
ā not an optimisation to add later. Baking deliberately bypasses the merged-tile
|
|
2360
|
+
cache, which is right for terrain, where each tile is written once and never
|
|
2361
|
+
wanted again. It is not right here.
|
|
2362
|
+
|
|
2363
|
+
### How far apart the lines go
|
|
2364
|
+
|
|
2365
|
+
Per zoom, because one interval is wrong at both ends of a map: at z8 a 20 m
|
|
2366
|
+
contour is a band of ink, and at z15 a 500 m one is a blank tile through most of
|
|
2367
|
+
the world. A recipe says either a number, meaning that interval wherever
|
|
2368
|
+
contours are drawn at all, or a table of zoom to intervals. Saying nothing gets
|
|
2369
|
+
a built-in table.
|
|
2370
|
+
|
|
2371
|
+
A level may name more than one interval. `[100, 500]` draws a line every hundred
|
|
2372
|
+
metres and marks every fifth, and each feature carries `level` ā how many of the
|
|
2373
|
+
intervals its height divides by, so 500 outranks 100. A style reads that to draw
|
|
2374
|
+
the major lines thicker and label only those, from one layer rather than two
|
|
2375
|
+
passes. The convention is `maplibre-contour`'s, so a style written against its
|
|
2376
|
+
tiles works against these.
|
|
2377
|
+
|
|
2378
|
+
A zoom the table skips reads as the entry above it: a table naming 12 and 14
|
|
2379
|
+
means 12 and 13 share a setting. Below the shallowest entry nothing is drawn at
|
|
2380
|
+
all, and that is checked _before_ the nine merges ā at z5 a tile is most of a
|
|
2381
|
+
continent, and nine merges is an expensive way to answer nothing.
|
|
2382
|
+
|
|
2383
|
+
### What the endpoint claims
|
|
2384
|
+
|
|
2385
|
+
The zoom range is the thresholds', not the stack's. A stack serving z0āz16 draws
|
|
2386
|
+
no contours at z2, and a client told otherwise fetches empty tiles all the way
|
|
2387
|
+
down. It is never deeper than the stack has ground for either: a contour traced
|
|
2388
|
+
from an upscaled parent is the parent's line drawn twice as thick, not new
|
|
2389
|
+
detail.
|
|
2390
|
+
|
|
2391
|
+
### Why the encoding is written here
|
|
2392
|
+
|
|
2393
|
+
`maplibre-contour` has a vector tile encoder and does not export it, and
|
|
2394
|
+
`vt-pbf` ā the obvious dependency ā is built against `pbf` 3 while this tree
|
|
2395
|
+
resolves `pbf` 5, whose `Pbf` default export no longer exists. So `contour-mvt.js`
|
|
2396
|
+
writes the tile directly against `pbf` 5. It is narrow enough to be worth
|
|
2397
|
+
owning: one layer of line strings with two numeric properties, where a general
|
|
2398
|
+
encoder carries polygons, points, mixed property types and many layers.
|
|
2399
|
+
|
|
2400
|
+
Heights are written in ascending order, so two runs over the same contours
|
|
2401
|
+
produce the same bytes whatever order the tracer closed its fragments in ā which
|
|
2402
|
+
is what lets an export be resumed and a tile be keyed by content. A tile no
|
|
2403
|
+
contour crossed is no tile at all rather than an empty layer, which a client
|
|
2404
|
+
would pay to fetch and draw nothing from.
|
|
2405
|
+
|
|
2406
|
+
`maplibre-contour` itself is loaded through `createRequire`: the published 0.1.0
|
|
2407
|
+
declares no `import` condition, so `import 'maplibre-contour'` fails outright.
|
|
2408
|
+
The fix is in upstream main and unreleased. A git dependency would push that
|
|
2409
|
+
requirement onto everyone installing this package, so the `require` path is
|
|
2410
|
+
taken instead, and becomes an ordinary import when a release carries the fix.
|
|
2411
|
+
|
|
2301
2412
|
## Merging vector sources
|
|
2302
2413
|
|
|
2303
2414
|
**Not built.** This is the design, written down before anything is started so
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "pmtiles-swarm",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.101.0",
|
|
4
4
|
"description": "BitTorrent distribution for PMTiles map archives: create torrents, watch folders, publish and subscribe to RSS feeds, and seed through qBittorrent or an embedded client",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"main": "src/index.js",
|
|
@@ -46,8 +46,10 @@
|
|
|
46
46
|
"create-torrent": "^6.1.0",
|
|
47
47
|
"express": "^5.2.1",
|
|
48
48
|
"fs-chunk-store": "^5.0.1",
|
|
49
|
+
"maplibre-contour": "^0.1.0",
|
|
49
50
|
"maplibre-gl": "^6.2.0",
|
|
50
51
|
"parse-torrent": "^11.0.24",
|
|
52
|
+
"pbf": "^5.1.2",
|
|
51
53
|
"pmtiles": "^4.4.1",
|
|
52
54
|
"pmtiles-torrent": "^0.11.1",
|
|
53
55
|
"webtorrent": "^3.0.21"
|
package/src/api.js
CHANGED
|
@@ -57,6 +57,13 @@ import {
|
|
|
57
57
|
stacksAffectedBy,
|
|
58
58
|
} from './stacks.js';
|
|
59
59
|
import { bakeRevision } from './bake.js';
|
|
60
|
+
import zlib from 'node:zlib';
|
|
61
|
+
import {
|
|
62
|
+
contourTile,
|
|
63
|
+
heightsFromArchive,
|
|
64
|
+
heightsFromStack,
|
|
65
|
+
} from './contour-tile.js';
|
|
66
|
+
import { contourProblems } from './contour-options.js';
|
|
60
67
|
|
|
61
68
|
const here = path.dirname(fileURLToPath(import.meta.url));
|
|
62
69
|
|
|
@@ -251,6 +258,7 @@ export function createApp({
|
|
|
251
258
|
tiles,
|
|
252
259
|
stacks,
|
|
253
260
|
stackCache,
|
|
261
|
+
heightsCache,
|
|
254
262
|
cutlines,
|
|
255
263
|
bakes,
|
|
256
264
|
warm,
|
|
@@ -3528,6 +3536,15 @@ export function createApp({
|
|
|
3528
3536
|
publicDownload: req.body?.publicDownload,
|
|
3529
3537
|
keep: req.body?.keep,
|
|
3530
3538
|
keepDays: req.body?.keepDays,
|
|
3539
|
+
// Which part of the stack to write. Absent is all of it, which is
|
|
3540
|
+
// what an export has always meant. See src/bake-selection.js.
|
|
3541
|
+
minzoom: req.body?.minzoom,
|
|
3542
|
+
maxzoom: req.body?.maxzoom,
|
|
3543
|
+
bounds: req.body?.bounds,
|
|
3544
|
+
// And what to make of it. Terrain unless asked otherwise, so
|
|
3545
|
+
// nothing already written has to start saying so.
|
|
3546
|
+
kind: req.body?.kind,
|
|
3547
|
+
thresholds: req.body?.thresholds,
|
|
3531
3548
|
});
|
|
3532
3549
|
return res.status(202).json({ bake: job });
|
|
3533
3550
|
} catch (error) {
|
|
@@ -3952,6 +3969,56 @@ export function createApp({
|
|
|
3952
3969
|
* @param {number} y - Row.
|
|
3953
3970
|
* @returns {void}
|
|
3954
3971
|
*/
|
|
3972
|
+
/**
|
|
3973
|
+
* Gzips a vector tile, the way the archive tile route does.
|
|
3974
|
+
* @param {Buffer} data - The tile.
|
|
3975
|
+
* @returns {Promise<Buffer>} - The compressed tile.
|
|
3976
|
+
*/
|
|
3977
|
+
const gzip = (data) =>
|
|
3978
|
+
new Promise((resolve, reject) =>
|
|
3979
|
+
zlib.gzip(data, (error, out) => (error ? reject(error) : resolve(out))),
|
|
3980
|
+
);
|
|
3981
|
+
|
|
3982
|
+
/**
|
|
3983
|
+
* What contour intervals to draw a stack at.
|
|
3984
|
+
*
|
|
3985
|
+
* The recipe's, unless the request asked for something else. A query
|
|
3986
|
+
* parameter rather than only a recipe field because contours are a view of a
|
|
3987
|
+
* stack rather than a property of one -- the same terrain is wanted at 10 m
|
|
3988
|
+
* on a walking map and 100 m on an atlas, and neither is the stack's
|
|
3989
|
+
* business.
|
|
3990
|
+
* @param {object} stack - The stack definition.
|
|
3991
|
+
* @param {object} [query] - The request's query parameters.
|
|
3992
|
+
* @returns {number|object|undefined} - What the recipe or the caller named.
|
|
3993
|
+
*/
|
|
3994
|
+
const contourThresholdsFor = (stack, query = {}) => {
|
|
3995
|
+
const asked = query.interval ?? query.thresholds;
|
|
3996
|
+
if (asked !== undefined && asked !== '') {
|
|
3997
|
+
const flat = Number(asked);
|
|
3998
|
+
if (Number.isFinite(flat)) return flat;
|
|
3999
|
+
try {
|
|
4000
|
+
return JSON.parse(asked);
|
|
4001
|
+
} catch {
|
|
4002
|
+
// Left to `contourProblems` to refuse and name, rather than answering
|
|
4003
|
+
// a different question than the one that was asked.
|
|
4004
|
+
return asked;
|
|
4005
|
+
}
|
|
4006
|
+
}
|
|
4007
|
+
return stack.contours?.thresholds;
|
|
4008
|
+
};
|
|
4009
|
+
|
|
4010
|
+
/**
|
|
4011
|
+
* A short, stable tag for one set of thresholds.
|
|
4012
|
+
* @param {number|object} [thresholds] - What is being drawn.
|
|
4013
|
+
* @returns {string} - Twelve hex characters.
|
|
4014
|
+
*/
|
|
4015
|
+
const contourTag = (thresholds) =>
|
|
4016
|
+
crypto
|
|
4017
|
+
.createHash('sha1')
|
|
4018
|
+
.update(JSON.stringify(thresholds ?? null))
|
|
4019
|
+
.digest('hex')
|
|
4020
|
+
.slice(0, 12);
|
|
4021
|
+
|
|
3955
4022
|
const stackHeaders = (res, resolved, contributors, z, x, y) => {
|
|
3956
4023
|
res.setHeader('access-control-allow-origin', '*');
|
|
3957
4024
|
res.setHeader('x-stack-sources', contributors.join(', '));
|
|
@@ -4099,6 +4166,228 @@ export function createApp({
|
|
|
4099
4166
|
return res.send(answer.body);
|
|
4100
4167
|
});
|
|
4101
4168
|
|
|
4169
|
+
/**
|
|
4170
|
+
* Contour lines traced from what a stack merges.
|
|
4171
|
+
*
|
|
4172
|
+
* Its own endpoint rather than a format the tile route answers in, because
|
|
4173
|
+
* it is a different map: vector where that is raster, drawn at the zooms the
|
|
4174
|
+
* thresholds name rather than the ones the stack has ground for, and costing
|
|
4175
|
+
* nine merges a tile rather than one. Sharing a route would mean one set of
|
|
4176
|
+
* headers describing two things.
|
|
4177
|
+
*
|
|
4178
|
+
* See docs/tile-stacks.md -- "Contours from a stack".
|
|
4179
|
+
*/
|
|
4180
|
+
/**
|
|
4181
|
+
* Answers one contour tile, for whatever produced the resolution.
|
|
4182
|
+
*
|
|
4183
|
+
* A stack, one archive, or whichever build a category currently points at.
|
|
4184
|
+
* The three differ only in where the heights come from and what identifies
|
|
4185
|
+
* them, so both arrive as functions and everything else -- the coordinate
|
|
4186
|
+
* checks, the codec refusal, the abort, the headers -- is written once.
|
|
4187
|
+
* @param {import('express').Request} req - The request.
|
|
4188
|
+
* @param {import('express').Response} res - The response.
|
|
4189
|
+
* @param {object} options - `heightsAt`, `tag`, `thresholds`, `immutable`.
|
|
4190
|
+
* @returns {Promise<void>} - Resolves once answered.
|
|
4191
|
+
*/
|
|
4192
|
+
const answerContours = async (req, res, options = {}) => {
|
|
4193
|
+
// Always: contours are traced from heights, so every source has to be
|
|
4194
|
+
// decoded whatever would otherwise have been passed through.
|
|
4195
|
+
const codec = await loadCodec();
|
|
4196
|
+
if (!codec) {
|
|
4197
|
+
return res.status(501).json({
|
|
4198
|
+
error:
|
|
4199
|
+
'tracing contours means decoding pixels, and this node has no codec',
|
|
4200
|
+
hint: 'npm install sharp',
|
|
4201
|
+
});
|
|
4202
|
+
}
|
|
4203
|
+
|
|
4204
|
+
const z = Number(req.params.z);
|
|
4205
|
+
const x = Number(req.params.x);
|
|
4206
|
+
const y = Number(req.params.y);
|
|
4207
|
+
if (![z, x, y].every(Number.isInteger)) {
|
|
4208
|
+
return res.status(400).json({ error: 'z, x and y must be integers' });
|
|
4209
|
+
}
|
|
4210
|
+
const limit = 2 ** z;
|
|
4211
|
+
if (z < 0 || z > 26 || x < 0 || y < 0 || x >= limit || y >= limit) {
|
|
4212
|
+
return res.status(400).json({ error: 'tile coordinates out of range' });
|
|
4213
|
+
}
|
|
4214
|
+
|
|
4215
|
+
const thresholds = options.thresholds;
|
|
4216
|
+
const wrong = contourProblems({ thresholds });
|
|
4217
|
+
if (wrong.length) {
|
|
4218
|
+
return res.status(400).json({ error: wrong.join('; ') });
|
|
4219
|
+
}
|
|
4220
|
+
|
|
4221
|
+
const controller = new AbortController();
|
|
4222
|
+
// Nine merges deep, so an abandoned request here is nine abandoned reads
|
|
4223
|
+
// per source. A panning map does that constantly.
|
|
4224
|
+
res.on('close', () => {
|
|
4225
|
+
if (!res.writableEnded) controller.abort();
|
|
4226
|
+
});
|
|
4227
|
+
|
|
4228
|
+
let tile;
|
|
4229
|
+
try {
|
|
4230
|
+
tile = await contourTile({
|
|
4231
|
+
heightsAt: options.heightsAt(codec, controller.signal),
|
|
4232
|
+
z,
|
|
4233
|
+
x,
|
|
4234
|
+
y,
|
|
4235
|
+
thresholds,
|
|
4236
|
+
});
|
|
4237
|
+
} catch (error) {
|
|
4238
|
+
if (error?.name === 'AbortError') return;
|
|
4239
|
+
throw error;
|
|
4240
|
+
}
|
|
4241
|
+
|
|
4242
|
+
res.setHeader('access-control-allow-origin', '*');
|
|
4243
|
+
res.setHeader('x-stack-sources', 'contours');
|
|
4244
|
+
// The thresholds decide the bytes as much as the ground does, so they go
|
|
4245
|
+
// in the tag. Without that a style asking for 50 m lines is answered from
|
|
4246
|
+
// a cache holding 100 m ones.
|
|
4247
|
+
res.setHeader(
|
|
4248
|
+
'etag',
|
|
4249
|
+
`W/"${options.tag(z, x, y)}-${contourTag(thresholds)}"`,
|
|
4250
|
+
);
|
|
4251
|
+
// Pinned content can never change; a category moves when its newest build
|
|
4252
|
+
// does, and a stack when its recipe does. The same split the tile routes
|
|
4253
|
+
// make, for the same reason.
|
|
4254
|
+
res.setHeader(
|
|
4255
|
+
'cache-control',
|
|
4256
|
+
options.immutable
|
|
4257
|
+
? 'public, max-age=31536000, immutable'
|
|
4258
|
+
: 'public, max-age=300, must-revalidate',
|
|
4259
|
+
);
|
|
4260
|
+
// Nothing crossed a threshold here. 404 rather than an empty tile, which
|
|
4261
|
+
// a client pays to fetch and draws nothing from.
|
|
4262
|
+
if (!tile) return res.status(404).end();
|
|
4263
|
+
|
|
4264
|
+
res.type('application/x-protobuf');
|
|
4265
|
+
res.setHeader('content-encoding', 'gzip');
|
|
4266
|
+
return res.send(await gzip(tile));
|
|
4267
|
+
};
|
|
4268
|
+
|
|
4269
|
+
/**
|
|
4270
|
+
* Whether an archive's tiles are heights at all.
|
|
4271
|
+
*
|
|
4272
|
+
* The same test the console uses to decide whether to offer a terrain
|
|
4273
|
+
* preview: an archive naming no encoding is a picture, and tracing a
|
|
4274
|
+
* picture's channels as though they were a number produces lines that mean
|
|
4275
|
+
* nothing rather than an error.
|
|
4276
|
+
* @param {object} entry - A catalog entry.
|
|
4277
|
+
* @returns {boolean} - Whether contours can be traced from it.
|
|
4278
|
+
*/
|
|
4279
|
+
const tracesAsTerrain = (entry) =>
|
|
4280
|
+
['terrarium', 'mapbox', 'custom'].includes(entry?.pmtiles?.encoding);
|
|
4281
|
+
|
|
4282
|
+
/**
|
|
4283
|
+
* Contour lines traced from what a stack merges.
|
|
4284
|
+
*
|
|
4285
|
+
* Its own endpoint rather than a format the tile route answers in, because
|
|
4286
|
+
* it is a different map: vector where that is raster, drawn at the zooms the
|
|
4287
|
+
* thresholds name rather than the ones the stack has ground for, and costing
|
|
4288
|
+
* nine merges a tile rather than one. Sharing a route would mean one set of
|
|
4289
|
+
* headers describing two things.
|
|
4290
|
+
*
|
|
4291
|
+
* See docs/tile-stacks.md -- "Contours from a stack".
|
|
4292
|
+
*/
|
|
4293
|
+
const serveStackContours = route(async (req, res) => {
|
|
4294
|
+
await stacks?.refresh();
|
|
4295
|
+
const resolved = stackOr404(req, res);
|
|
4296
|
+
if (!resolved) return;
|
|
4297
|
+
if (resolved.stack.space === 'rgba') {
|
|
4298
|
+
return res.status(400).json({
|
|
4299
|
+
error: 'this stack is imagery, and a colour has no contours to trace',
|
|
4300
|
+
});
|
|
4301
|
+
}
|
|
4302
|
+
return answerContours(req, res, {
|
|
4303
|
+
thresholds: contourThresholdsFor(resolved.stack, req.query),
|
|
4304
|
+
immutable: isPinned(resolved),
|
|
4305
|
+
tag: (z, x, y) => stackEtag(resolved, z, x, y).slice(1, -1),
|
|
4306
|
+
heightsAt: (codec, signal) =>
|
|
4307
|
+
heightsFromStack({
|
|
4308
|
+
resolved,
|
|
4309
|
+
tiles,
|
|
4310
|
+
codec,
|
|
4311
|
+
cutlines,
|
|
4312
|
+
signal,
|
|
4313
|
+
heightsCache,
|
|
4314
|
+
}),
|
|
4315
|
+
});
|
|
4316
|
+
});
|
|
4317
|
+
|
|
4318
|
+
/** Contours traced from one archive, which is a stack of one source. */
|
|
4319
|
+
const serveArchiveContours = route(async (req, res) => {
|
|
4320
|
+
const entry = catalog.get(req.params.infoHash);
|
|
4321
|
+
if (!entry) return res.status(404).json({ error: 'unknown archive' });
|
|
4322
|
+
if (!tracesAsTerrain(entry)) {
|
|
4323
|
+
return res.status(400).json({
|
|
4324
|
+
error:
|
|
4325
|
+
'this archive does not say its pixels are heights, so there is ' +
|
|
4326
|
+
'nothing to trace contours from',
|
|
4327
|
+
hint: 'an archive states that as `encoding` in its metadata',
|
|
4328
|
+
});
|
|
4329
|
+
}
|
|
4330
|
+
return answerContours(req, res, {
|
|
4331
|
+
thresholds: contourThresholdsFor({}, req.query),
|
|
4332
|
+
// An infohash names one build and can never come to mean another.
|
|
4333
|
+
immutable: true,
|
|
4334
|
+
tag: (z, x, y) => `${entry.infoHash.slice(0, 20)}-${z}/${x}/${y}`,
|
|
4335
|
+
heightsAt: (codec, signal) =>
|
|
4336
|
+
heightsFromArchive({ entry, tiles, codec, signal, heightsCache }),
|
|
4337
|
+
});
|
|
4338
|
+
});
|
|
4339
|
+
|
|
4340
|
+
/** The same, for whichever build a category currently points at. */
|
|
4341
|
+
const serveCategoryContours = route(async (req, res) => {
|
|
4342
|
+
const entry = newestIn(req.params.category, req);
|
|
4343
|
+
if (!entry) {
|
|
4344
|
+
return res.status(404).json({ error: 'nothing is in that category' });
|
|
4345
|
+
}
|
|
4346
|
+
if (!tracesAsTerrain(entry)) {
|
|
4347
|
+
return res.status(400).json({
|
|
4348
|
+
error:
|
|
4349
|
+
"this category's newest build does not say its pixels are heights",
|
|
4350
|
+
});
|
|
4351
|
+
}
|
|
4352
|
+
return answerContours(req, res, {
|
|
4353
|
+
thresholds: contourThresholdsFor({}, req.query),
|
|
4354
|
+
// A category moves when a newer build lands, so it revalidates.
|
|
4355
|
+
immutable: false,
|
|
4356
|
+
// Tagged by the build it resolved to rather than by the category, so a
|
|
4357
|
+
// rebuild lands as a new tag rather than as the same one with different
|
|
4358
|
+
// lines behind it.
|
|
4359
|
+
tag: (z, x, y) => `${entry.infoHash.slice(0, 20)}-${z}/${x}/${y}`,
|
|
4360
|
+
heightsAt: (codec, signal) =>
|
|
4361
|
+
heightsFromArchive({ entry, tiles, codec, signal, heightsCache }),
|
|
4362
|
+
});
|
|
4363
|
+
});
|
|
4364
|
+
|
|
4365
|
+
/**
|
|
4366
|
+
* The three ways in. A contour tile is a vector tile whatever produced it,
|
|
4367
|
+
* so the extension is checked once and in one place.
|
|
4368
|
+
* @param {Function} handler - What answers it.
|
|
4369
|
+
* @returns {Function} - An express handler.
|
|
4370
|
+
*/
|
|
4371
|
+
const contourRoute = (handler) => (req, res, next) => {
|
|
4372
|
+
if (!['pbf', 'mvt'].includes(req.params.ext)) {
|
|
4373
|
+
return res.status(400).json({ error: 'contours are served as pbf' });
|
|
4374
|
+
}
|
|
4375
|
+
return handler(req, res, next);
|
|
4376
|
+
};
|
|
4377
|
+
|
|
4378
|
+
app.get(
|
|
4379
|
+
'/stacks/:id/contours/:z/:x/:y.:ext',
|
|
4380
|
+
contourRoute(serveStackContours),
|
|
4381
|
+
);
|
|
4382
|
+
app.get(
|
|
4383
|
+
'/archives/:infoHash/contours/:z/:x/:y.:ext',
|
|
4384
|
+
contourRoute(serveArchiveContours),
|
|
4385
|
+
);
|
|
4386
|
+
app.get(
|
|
4387
|
+
'/latest/:category/contours/:z/:x/:y.:ext',
|
|
4388
|
+
contourRoute(serveCategoryContours),
|
|
4389
|
+
);
|
|
4390
|
+
|
|
4102
4391
|
app.get('/stacks/:id/:size/:z/:x/:y.:ext', (req, res, next) => {
|
|
4103
4392
|
// Anything that is not a size this serves is not a size at all -- and
|
|
4104
4393
|
// must not fall through to the shorter shape, where it would be read as a
|