pmtiles-swarm 0.74.0 → 0.75.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +28 -1
- package/docs/tile-stacks.md +69 -0
- package/package.json +1 -1
- package/src/elevation.js +75 -0
- package/src/stack-tile.js +8 -2
- package/src/stacks.js +24 -0
- package/src/web/index.html +32 -0
- package/tools/terrain-probe.mjs +250 -0
package/CHANGELOG.md
CHANGED
|
@@ -7,8 +7,35 @@
|
|
|
7
7
|
### 🐞 Bug fixes
|
|
8
8
|
- _...Add new stuff here..._
|
|
9
9
|
|
|
10
|
-
## 0.
|
|
10
|
+
## 0.75.0
|
|
11
11
|
### ✨ Features and improvements
|
|
12
|
+
- **`maskRange`, because nodata is a band and not a number.** `maskValues` and `maskColors` both
|
|
13
|
+
compare exactly, and cubic resampling overshoots at every edge it crosses - so a sea authored as
|
|
14
|
+
`0` arrives scattered across `-0.9` to `0`. The exact matches were masked and everything between
|
|
15
|
+
them was left standing.
|
|
16
|
+
|
|
17
|
+
Measured on a real merge, a terrain source over GEBCO at z13. The recipe masked `#018696` and
|
|
18
|
+
`#0186a0`, which decode to -1.0 m and 0.0 m; the nine colours between them, -0.9 m to -0.1 m,
|
|
19
|
+
were never masked. 1,227 pixels of one tile stood clear of all eight neighbours, the worst 25.1 m
|
|
20
|
+
above the bathymetry underneath - a scattering of 25 m pillars over open water, which is what
|
|
21
|
+
stipples a hillshade.
|
|
22
|
+
|
|
23
|
+
```json
|
|
24
|
+
{ "archive": "planet", "maskRange": [-1, 0] }
|
|
25
|
+
```
|
|
26
|
+
|
|
27
|
+
A band rather than a width either side of a value, because nodata is rarely symmetric about
|
|
28
|
+
anything: sea is everything up to zero and nothing above it, and a width reaching a metre down
|
|
29
|
+
reaches a metre up as well, into ground that is really there. It is also what a recipe naming two
|
|
30
|
+
colours was already reaching for - they are the ends of one. A list of bands is accepted for a
|
|
31
|
+
source with a sentinel as well as a range, and the edges are inclusive, compared in thousandths
|
|
32
|
+
so `[-0.2, 0]` includes the -0.2 a Float32Array stores as -0.20000000298.
|
|
33
|
+
|
|
34
|
+
Worth recording that no average shows the problem this solves. That merged tile measures 0.2 m of
|
|
35
|
+
roughness at the median and is smooth by every summary statistic, because half a per cent of
|
|
36
|
+
pixels never move the middle of a distribution. `tools/terrain-probe.mjs` reports the tail and
|
|
37
|
+
counts pixels standing clear of their neighbours, which is the shape it makes.
|
|
38
|
+
|
|
12
39
|
- **The holes a mask leaves are feathered now, not just a cutline's edge.** `feather` faded a source
|
|
13
40
|
in at its `cutline` or `bounds` and did nothing about `maskValues` or `maskColors` - which is
|
|
14
41
|
where most of these recipes actually stop, and why the field looked like it did nothing.
|
package/docs/tile-stacks.md
CHANGED
|
@@ -241,6 +241,7 @@ it differs from the snake_case rio-rgbify-merge uses.
|
|
|
241
241
|
| `encoding` | `mapbox` or `terrarium`. Elevation space only. |
|
|
242
242
|
| `baseVal` / `interval` | Mapbox decode offset and step. Default `-10000` and `0.1`. |
|
|
243
243
|
| `maskValues` | Decoded heights meaning "no data here". Elevation space only. |
|
|
244
|
+
| `maskRange` | `[low, high]` in metres, or a list of them. Everything inside is nodata. Elevation space only. |
|
|
244
245
|
| `heightAdjustment` | Metres, added **after** masking. Elevation space only. |
|
|
245
246
|
| `feather` | Pixels to fade in over at the edge of the source's shape. Needs a `cutline` or `bounds`. Max 64. |
|
|
246
247
|
| `opacity` | `0`–`1`, scales the source alpha. RGBA space only. |
|
|
@@ -1014,6 +1015,74 @@ missing. Serving a source unclipped because its cutline could not be found would
|
|
|
1014
1015
|
put back exactly the data somebody asked to remove, which is the one failure a
|
|
1015
1016
|
clip must not have.
|
|
1016
1017
|
|
|
1018
|
+
## What a mask has to match
|
|
1019
|
+
|
|
1020
|
+
`maskValues` and `maskColors` both compare exactly, and an archive that was
|
|
1021
|
+
resampled on its way to being built does not hold the number it was authored
|
|
1022
|
+
with. Cubic overshoots at every edge it crosses, so a sea authored as exactly
|
|
1023
|
+
`0` arrives as a field of `0` with `-0.4` and `0.1` scattered through it near
|
|
1024
|
+
the coast.
|
|
1025
|
+
|
|
1026
|
+
Masking by colour makes this sharper rather than softer, because a colour is one
|
|
1027
|
+
exact number and the resampled ground either side of it is several others. A
|
|
1028
|
+
recipe masking `#018696` and `#0186a0` is masking **-1.0 m** and **0.0 m** —
|
|
1029
|
+
and leaving `#018697` through `#01869f`, which are -0.9 m through -0.1 m,
|
|
1030
|
+
entirely alone.
|
|
1031
|
+
|
|
1032
|
+
Measured on a real merge — a planet DEM over GEBCO bathymetry, one tile at z13:
|
|
1033
|
+
|
|
1034
|
+
| | |
|
|
1035
|
+
| --------------------------------------------- | ------------- |
|
|
1036
|
+
| planet source, pixels at exactly `0` | 109,441 |
|
|
1037
|
+
| its lowest value | -0.4 m |
|
|
1038
|
+
| merged tile, pixels at exactly `0` | 13 |
|
|
1039
|
+
| merged tile, pixels clear of all 8 neighbours | 1,227 (0.47%) |
|
|
1040
|
+
| the worst of them | 25.1 m |
|
|
1041
|
+
|
|
1042
|
+
`maskValues: [0]` took the 109,441 zeroes and left the rest standing. With
|
|
1043
|
+
bathymetry underneath, each survivor became a spike as tall as the difference
|
|
1044
|
+
between the two sources — a scattering of 25 m pillars over open water, which
|
|
1045
|
+
is what stipples a hillshade.
|
|
1046
|
+
|
|
1047
|
+
`maskRange` says the band outright:
|
|
1048
|
+
|
|
1049
|
+
```json
|
|
1050
|
+
{ "archive": "planet", "maskRange": [-1, 0] }
|
|
1051
|
+
```
|
|
1052
|
+
|
|
1053
|
+
A band rather than a width either side of a value, because nodata is rarely
|
|
1054
|
+
symmetric about anything. Sea is everything up to zero and nothing above it,
|
|
1055
|
+
and a width reaching a metre down reaches a metre up as well, into ground that
|
|
1056
|
+
is really there. It is also what the recipe above was already reaching for: its
|
|
1057
|
+
two colours are the ends of one.
|
|
1058
|
+
|
|
1059
|
+
Several, where a source has a sentinel as well as a band:
|
|
1060
|
+
|
|
1061
|
+
```json
|
|
1062
|
+
{
|
|
1063
|
+
"archive": "planet",
|
|
1064
|
+
"maskRange": [
|
|
1065
|
+
[-1, 0],
|
|
1066
|
+
[-10001, -9999]
|
|
1067
|
+
]
|
|
1068
|
+
}
|
|
1069
|
+
```
|
|
1070
|
+
|
|
1071
|
+
The edges are inclusive and compared in thousandths, because a `Float32Array`
|
|
1072
|
+
holds -0.2 as -0.20000000298 and an edge that does not include the number
|
|
1073
|
+
written on it leaves a row of pixels behind.
|
|
1074
|
+
|
|
1075
|
+
It is a trade rather than a free win: a band wide enough to catch the
|
|
1076
|
+
resampling's overshoot also eats genuine ground inside it. Reaching down to the
|
|
1077
|
+
lowest value the sea arrives at is usually the right price; reaching up above
|
|
1078
|
+
zero is not, which is the asymmetry a band can express and a width cannot.
|
|
1079
|
+
|
|
1080
|
+
Worth knowing that neither a median nor a colour ramp shows this. The merged
|
|
1081
|
+
tile above measures 0.2 m of roughness at the median and looks smooth by every
|
|
1082
|
+
summary statistic — half a per cent of pixels never move the middle of a
|
|
1083
|
+
distribution. `tools/terrain-probe.mjs` reports the tail and counts pixels
|
|
1084
|
+
standing clear of their neighbours, which is the shape this makes.
|
|
1085
|
+
|
|
1017
1086
|
## Feathering a seam
|
|
1018
1087
|
|
|
1019
1088
|
_Built for a cutline and for `bounds`. A mask edge is not feathered yet, and
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "pmtiles-swarm",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.75.0",
|
|
4
4
|
"description": "BitTorrent distribution for PMTiles map archives: create torrents, watch folders, publish and subscribe to RSS feeds, and seed through qBittorrent or an embedded client",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"main": "src/index.js",
|
package/src/elevation.js
CHANGED
|
@@ -162,6 +162,77 @@ export function maskHeights(heights, maskValues) {
|
|
|
162
162
|
return heights;
|
|
163
163
|
}
|
|
164
164
|
|
|
165
|
+
/**
|
|
166
|
+
* Reads `maskRange` into a list of low-high pairs.
|
|
167
|
+
*
|
|
168
|
+
* One pair or several, because both read naturally: `[-1, 0]` is the common
|
|
169
|
+
* case and `[[-1, 0], [-10001, -9999]]` is a source with a sentinel as well as
|
|
170
|
+
* a band. A pair the wrong way round is taken as the same band rather than
|
|
171
|
+
* refused -- the recipe validation says so, and a merge that has got this far
|
|
172
|
+
* should not silently mask nothing.
|
|
173
|
+
* @param {Array<number[]>|number[]|undefined} maskRange - What the recipe said.
|
|
174
|
+
* @returns {Array<number[]>} - Pairs, low first.
|
|
175
|
+
*/
|
|
176
|
+
export function rangesOf(maskRange) {
|
|
177
|
+
if (!Array.isArray(maskRange) || maskRange.length === 0) return [];
|
|
178
|
+
const pairs = Array.isArray(maskRange[0]) ? maskRange : [maskRange];
|
|
179
|
+
const out = [];
|
|
180
|
+
for (const pair of pairs) {
|
|
181
|
+
if (!Array.isArray(pair) || pair.length < 2) continue;
|
|
182
|
+
const low = Number(pair[0]);
|
|
183
|
+
const high = Number(pair[1]);
|
|
184
|
+
if (!Number.isFinite(low) || !Number.isFinite(high)) continue;
|
|
185
|
+
out.push(low <= high ? [low, high] : [high, low]);
|
|
186
|
+
}
|
|
187
|
+
return out;
|
|
188
|
+
}
|
|
189
|
+
|
|
190
|
+
/**
|
|
191
|
+
* Blanks every height inside a band.
|
|
192
|
+
*
|
|
193
|
+
* The shape nodata actually has. An archive resampled on its way to being
|
|
194
|
+
* built does not hold the number it was authored with -- cubic overshoots at
|
|
195
|
+
* every edge it crosses, so a sea authored as exactly 0 arrives as a field of
|
|
196
|
+
* 0 with -0.4 and 0.1 scattered through it near the coast. Masking the exact
|
|
197
|
+
* values takes some of them and leaves the rest standing, and where another
|
|
198
|
+
* source is underneath each survivor becomes a spike as tall as the difference
|
|
199
|
+
* between them: measured at 25 m on a real merge, on half a per cent of the
|
|
200
|
+
* tile, which is what stipples a hillshade.
|
|
201
|
+
*
|
|
202
|
+
* A band rather than a width either side of a value, because nodata is rarely
|
|
203
|
+
* symmetric about anything. Sea is everything from the deepest sentinel up to
|
|
204
|
+
* zero and nothing above it, and a width wide enough to reach the bottom of
|
|
205
|
+
* that reaches the same distance into real ground. See docs/tile-stacks.md --
|
|
206
|
+
* "What a mask has to match".
|
|
207
|
+
* @param {Float32Array} heights - Metres, modified in place.
|
|
208
|
+
* @param {Array<number[]>|number[]} [maskRange] - A `[low, high]` pair, or a list of them.
|
|
209
|
+
* @returns {Float32Array} - The same array.
|
|
210
|
+
*/
|
|
211
|
+
export function maskRanges(heights, maskRange) {
|
|
212
|
+
const bands = rangesOf(maskRange).map(([low, high]) => [
|
|
213
|
+
Math.round(low * 1000),
|
|
214
|
+
Math.round(high * 1000),
|
|
215
|
+
]);
|
|
216
|
+
if (bands.length === 0) return heights;
|
|
217
|
+
|
|
218
|
+
// Compared in thousandths, the same way maskHeights compares. A Float32Array
|
|
219
|
+
// holds -0.2 as -0.20000000298, which falls outside a band written as
|
|
220
|
+
// [-0.2, 0] -- and an edge that does not include the number written on it is
|
|
221
|
+
// a mask that quietly leaves a row of pixels behind.
|
|
222
|
+
for (let i = 0; i < heights.length; i += 1) {
|
|
223
|
+
const height = heights[i];
|
|
224
|
+
if (Number.isNaN(height)) continue;
|
|
225
|
+
const at = Math.round(height * 1000);
|
|
226
|
+
for (const [low, high] of bands) {
|
|
227
|
+
if (at >= low && at <= high) {
|
|
228
|
+
heights[i] = Number.NaN;
|
|
229
|
+
break;
|
|
230
|
+
}
|
|
231
|
+
}
|
|
232
|
+
}
|
|
233
|
+
return heights;
|
|
234
|
+
}
|
|
235
|
+
|
|
165
236
|
/**
|
|
166
237
|
* Reads one colour from a recipe into a packed 24-bit value.
|
|
167
238
|
*
|
|
@@ -638,8 +709,12 @@ export function mergeElevation(contributions, options) {
|
|
|
638
709
|
// Both masks say the same thing -- nothing here -- and both have to run
|
|
639
710
|
// before the height adjustment, which would otherwise shift the values
|
|
640
711
|
// out from under the comparison.
|
|
712
|
+
//
|
|
641
713
|
maskHeights(heights, source.maskValues);
|
|
642
714
|
maskColors(heights, contribution.raster, source.maskColors);
|
|
715
|
+
// A band as well as, not instead of: a source may have a sentinel it names
|
|
716
|
+
// exactly and a range of ground it does not want either.
|
|
717
|
+
maskRanges(heights, source.maskRange);
|
|
643
718
|
if (source.heightAdjustment) {
|
|
644
719
|
for (let i = 0; i < heights.length; i += 1) {
|
|
645
720
|
heights[i] += source.heightAdjustment;
|
package/src/stack-tile.js
CHANGED
|
@@ -5,6 +5,7 @@ import {
|
|
|
5
5
|
fillNodata,
|
|
6
6
|
maskColors,
|
|
7
7
|
maskHeights,
|
|
8
|
+
maskRanges,
|
|
8
9
|
mergeElevation,
|
|
9
10
|
} from './elevation.js';
|
|
10
11
|
import { paddedKnown, parentsFor } from './mask-edge.js';
|
|
@@ -450,7 +451,11 @@ async function gather({ reads, z, x, y, tiles, codec, signal, size, rgba }) {
|
|
|
450
451
|
* @returns {boolean} - True when it masks anything.
|
|
451
452
|
*/
|
|
452
453
|
function masksAnything(recipe) {
|
|
453
|
-
return Boolean(
|
|
454
|
+
return Boolean(
|
|
455
|
+
recipe?.maskValues?.length ||
|
|
456
|
+
recipe?.maskColors?.length ||
|
|
457
|
+
recipe?.maskRange?.length,
|
|
458
|
+
);
|
|
454
459
|
}
|
|
455
460
|
|
|
456
461
|
/**
|
|
@@ -506,12 +511,13 @@ async function readMaskEdges({
|
|
|
506
511
|
.catch(() => null);
|
|
507
512
|
if (!raster) return;
|
|
508
513
|
|
|
509
|
-
// The same
|
|
514
|
+
// The same masking the merge applies, asked of the parent. A ramp
|
|
510
515
|
// measured against a different idea of where the holes are would fade
|
|
511
516
|
// toward ground that is not a hole.
|
|
512
517
|
const heights = decodeHeights(raster, recipe);
|
|
513
518
|
maskHeights(heights, recipe.maskValues);
|
|
514
519
|
maskColors(heights, raster, recipe.maskColors);
|
|
520
|
+
maskRanges(heights, recipe.maskRange);
|
|
515
521
|
|
|
516
522
|
const flags = new Uint8Array(heights.length);
|
|
517
523
|
for (let i = 0; i < flags.length; i += 1) {
|
package/src/stacks.js
CHANGED
|
@@ -35,6 +35,8 @@ import { BLEND_MODES, isBlendMode } from './rgba.js';
|
|
|
35
35
|
* @property {number} [baseVal] - Mapbox decode offset.
|
|
36
36
|
* @property {number} [interval] - Mapbox decode step.
|
|
37
37
|
* @property {number[]} [maskValues] - Heights meaning "no data here".
|
|
38
|
+
* @property {Array<number[]>|number[]} [maskRange] - A `[low, high]` band of
|
|
39
|
+
* heights meaning "no data here", or a list of them.
|
|
38
40
|
* @property {Array<string|number[]>} [maskColors] - Pixel colours meaning the
|
|
39
41
|
* same, as "#rrggbb" or [r, g, b]. Exact, where a height mask has to round.
|
|
40
42
|
* @property {number} [heightAdjustment] - Metres, added after masking.
|
|
@@ -127,6 +129,28 @@ export function validateStack(stack) {
|
|
|
127
129
|
if (source?.maskValues !== undefined && !Array.isArray(source.maskValues)) {
|
|
128
130
|
problems.push(`sources[${index}].maskValues must be a list`);
|
|
129
131
|
}
|
|
132
|
+
if (source?.maskRange !== undefined) {
|
|
133
|
+
// One pair or a list of them. Anything else is a range nobody can read,
|
|
134
|
+
// and a mask that silently matches nothing is the failure this whole
|
|
135
|
+
// feature is most prone to.
|
|
136
|
+
const given = Array.isArray(source.maskRange) ? source.maskRange : null;
|
|
137
|
+
const pairs =
|
|
138
|
+
given && given.length && Array.isArray(given[0]) ? given : [given];
|
|
139
|
+
const usable =
|
|
140
|
+
given &&
|
|
141
|
+
given.length > 0 &&
|
|
142
|
+
pairs.every(
|
|
143
|
+
(pair) =>
|
|
144
|
+
Array.isArray(pair) &&
|
|
145
|
+
pair.length === 2 &&
|
|
146
|
+
pair.every((edge) => Number.isFinite(Number(edge))),
|
|
147
|
+
);
|
|
148
|
+
if (!usable) {
|
|
149
|
+
problems.push(
|
|
150
|
+
`sources[${index}].maskRange must be [low, high], or a list of them`,
|
|
151
|
+
);
|
|
152
|
+
}
|
|
153
|
+
}
|
|
130
154
|
if (source?.feather !== undefined) {
|
|
131
155
|
const feather = Number(source.feather);
|
|
132
156
|
if (!Number.isFinite(feather) || feather < 0 || feather > MAX_FEATHER) {
|
package/src/web/index.html
CHANGED
|
@@ -1904,6 +1904,21 @@
|
|
|
1904
1904
|
({ '&': '&', '<': '<', '>': '>', '"': '"', "'": ''' })[c],
|
|
1905
1905
|
);
|
|
1906
1906
|
|
|
1907
|
+
/**
|
|
1908
|
+
* A mask range as the field shows it.
|
|
1909
|
+
*
|
|
1910
|
+
* One pair is written plainly; a recipe carrying several is shown as
|
|
1911
|
+
* JSON, because the editor offers one band and the file may hold more
|
|
1912
|
+
* than the editor can draw.
|
|
1913
|
+
* @param {*} range - What the recipe says.
|
|
1914
|
+
* @returns {string} - What to put in the box.
|
|
1915
|
+
*/
|
|
1916
|
+
const rangeText = (range) => {
|
|
1917
|
+
if (!Array.isArray(range) || range.length === 0) return '';
|
|
1918
|
+
if (Array.isArray(range[0])) return JSON.stringify(range);
|
|
1919
|
+
return range.slice(0, 2).join(', ');
|
|
1920
|
+
};
|
|
1921
|
+
|
|
1907
1922
|
/**
|
|
1908
1923
|
* Redraws the open detail pane, when it is one that changes by itself.
|
|
1909
1924
|
*
|
|
@@ -7618,6 +7633,13 @@ Every piece is hashed against the ` +
|
|
|
7618
7633
|
<input style="width:12rem" placeholder="-10000, 0, -1"
|
|
7619
7634
|
data-stack-field="maskValues" data-stack-index="${index}"
|
|
7620
7635
|
value="${escapeHtml((source.maskValues ?? []).join(', '))}" />
|
|
7636
|
+
</label>
|
|
7637
|
+
<label class="choice"
|
|
7638
|
+
title="Mask every height inside a band, written low, high. Nodata is rarely one number: an archive resampled on its way to being built does not hold what it was authored with, so a sea authored as 0 arrives scattered across -0.9 m to 0 — and masking the two ends of that leaves everything between, standing proud of whatever is underneath. A band says what you mean, and asymmetrically: sea is everything up to zero and nothing above it.">
|
|
7639
|
+
Mask range
|
|
7640
|
+
<input style="width:9rem" placeholder="-1, 0"
|
|
7641
|
+
data-stack-field="maskRange" data-stack-index="${index}"
|
|
7642
|
+
value="${escapeHtml(rangeText(source.maskRange))}" /> m
|
|
7621
7643
|
</label>`;
|
|
7622
7644
|
|
|
7623
7645
|
return `
|
|
@@ -7739,6 +7761,16 @@ Every piece is hashed against the ` +
|
|
|
7739
7761
|
.filter(Boolean);
|
|
7740
7762
|
} else if (field === 'required') {
|
|
7741
7763
|
source.required = event.target.checked;
|
|
7764
|
+
} else if (field === 'maskRange') {
|
|
7765
|
+
// Held while it is half-typed, the way the clip box is: four numbers
|
|
7766
|
+
// arrive one keystroke at a time and a field that cleared itself
|
|
7767
|
+
// after the first comma could never be filled in.
|
|
7768
|
+
const numbers = event.target.value
|
|
7769
|
+
.split(',')
|
|
7770
|
+
.map((part) => Number(part.trim()))
|
|
7771
|
+
.filter((n) => Number.isFinite(n));
|
|
7772
|
+
if (numbers.length >= 2) source.maskRange = numbers.slice(0, 2);
|
|
7773
|
+
else if (event.target.value.trim() === '') delete source.maskRange;
|
|
7742
7774
|
} else if (field === 'feather') {
|
|
7743
7775
|
const pixels = Number(event.target.value);
|
|
7744
7776
|
if (event.target.value === '' || !(pixels > 0)) delete source.feather;
|
|
@@ -0,0 +1,250 @@
|
|
|
1
|
+
#!/usr/bin/env node
|
|
2
|
+
/**
|
|
3
|
+
* What is actually in a terrain tile, in metres.
|
|
4
|
+
*
|
|
5
|
+
* A colour ramp and a hillshade disagree about what looks wrong, and both of
|
|
6
|
+
* them lie about scale: quantisation of a tenth of a metre dithers a ramp's
|
|
7
|
+
* band edge into a checkerboard, and a hillshade turns the same tenth of a
|
|
8
|
+
* metre into a field of bumps if the ground is flat enough. Neither tells you
|
|
9
|
+
* which you are looking at. This prints the numbers.
|
|
10
|
+
*
|
|
11
|
+
* node tools/terrain-probe.mjs <tile-url> [--encoding mapbox] [--interval 0.1]
|
|
12
|
+
*
|
|
13
|
+
* where <tile-url> is one tile, for instance
|
|
14
|
+
*
|
|
15
|
+
* http://127.0.0.1:8090/archives/<infohash>/10/512/380.webp
|
|
16
|
+
* http://127.0.0.1:8090/stacks/<id>/10/512/380.webp
|
|
17
|
+
*
|
|
18
|
+
* The two lines worth reading are the step histogram and the flat-ground
|
|
19
|
+
* roughness. If the steps are all one quantum, what you are seeing is the
|
|
20
|
+
* encoding and the renderer, not the data.
|
|
21
|
+
*/
|
|
22
|
+
|
|
23
|
+
import { loadCodec } from '../src/codec.js';
|
|
24
|
+
import { decodeHeights } from '../src/elevation.js';
|
|
25
|
+
|
|
26
|
+
const args = process.argv.slice(2);
|
|
27
|
+
const url = args.find((a) => !a.startsWith('--'));
|
|
28
|
+
if (!url) {
|
|
29
|
+
console.error(
|
|
30
|
+
'usage: node tools/terrain-probe.mjs <tile-url> [--encoding mapbox] [--interval 0.1] [--baseVal -10000]',
|
|
31
|
+
);
|
|
32
|
+
process.exit(2);
|
|
33
|
+
}
|
|
34
|
+
|
|
35
|
+
/**
|
|
36
|
+
* One named option off the command line.
|
|
37
|
+
* @param {string} name - Its name, without the dashes.
|
|
38
|
+
* @param {string} fallback - What it is when absent.
|
|
39
|
+
* @returns {string} - The value.
|
|
40
|
+
*/
|
|
41
|
+
const option = (name, fallback) => {
|
|
42
|
+
const at = args.indexOf(`--${name}`);
|
|
43
|
+
return at >= 0 && args[at + 1] ? args[at + 1] : fallback;
|
|
44
|
+
};
|
|
45
|
+
|
|
46
|
+
const source = {
|
|
47
|
+
encoding: option('encoding', 'mapbox'),
|
|
48
|
+
interval: Number(option('interval', '0.1')),
|
|
49
|
+
baseVal: Number(option('baseVal', '-10000')),
|
|
50
|
+
};
|
|
51
|
+
|
|
52
|
+
const codec = await loadCodec();
|
|
53
|
+
if (!codec) {
|
|
54
|
+
console.error('this needs sharp: npm install sharp');
|
|
55
|
+
process.exit(1);
|
|
56
|
+
}
|
|
57
|
+
|
|
58
|
+
const response = await fetch(url);
|
|
59
|
+
if (!response.ok) {
|
|
60
|
+
console.error(`${url} answered ${response.status}`);
|
|
61
|
+
process.exit(1);
|
|
62
|
+
}
|
|
63
|
+
const raster = await codec.decode(Buffer.from(await response.arrayBuffer()), {
|
|
64
|
+
channels: 3,
|
|
65
|
+
});
|
|
66
|
+
const heights = decodeHeights(raster, source);
|
|
67
|
+
const { width } = raster;
|
|
68
|
+
// Off the URL, so roughness can be reported against the ground a pixel covers
|
|
69
|
+
// rather than in the abstract. A metre between neighbours means one thing at
|
|
70
|
+
// z8 and another at z16.
|
|
71
|
+
const coordinates = url.match(/\/(\d+)\/(\d+)\/(\d+)\.[a-z]+$/);
|
|
72
|
+
const zoom = Number(coordinates?.[1] ?? 0);
|
|
73
|
+
const tileRow = Number(coordinates?.[3] ?? 0);
|
|
74
|
+
|
|
75
|
+
// Walked rather than spread into Math.min: a 512px tile is 262,144 numbers and
|
|
76
|
+
// that many arguments overflows the call stack.
|
|
77
|
+
let low = Infinity;
|
|
78
|
+
let high = -Infinity;
|
|
79
|
+
let known = 0;
|
|
80
|
+
let zeroes = 0;
|
|
81
|
+
for (const value of heights) {
|
|
82
|
+
if (!Number.isFinite(value)) continue;
|
|
83
|
+
known += 1;
|
|
84
|
+
if (value === 0) zeroes += 1;
|
|
85
|
+
if (value < low) low = value;
|
|
86
|
+
if (value > high) high = value;
|
|
87
|
+
}
|
|
88
|
+
if (known === 0) {
|
|
89
|
+
console.log(`${url}
|
|
90
|
+
every pixel is nodata`);
|
|
91
|
+
process.exit(0);
|
|
92
|
+
}
|
|
93
|
+
|
|
94
|
+
// Every step between neighbouring pixels, left to right and top to bottom.
|
|
95
|
+
const steps = [];
|
|
96
|
+
for (let row = 0; row < width; row += 1) {
|
|
97
|
+
for (let column = 1; column < width; column += 1) {
|
|
98
|
+
steps.push(
|
|
99
|
+
Math.abs(
|
|
100
|
+
heights[row * width + column] - heights[row * width + column - 1],
|
|
101
|
+
),
|
|
102
|
+
);
|
|
103
|
+
}
|
|
104
|
+
}
|
|
105
|
+
for (let row = 1; row < width; row += 1) {
|
|
106
|
+
for (let column = 0; column < width; column += 1) {
|
|
107
|
+
steps.push(
|
|
108
|
+
Math.abs(
|
|
109
|
+
heights[row * width + column] - heights[(row - 1) * width + column],
|
|
110
|
+
),
|
|
111
|
+
);
|
|
112
|
+
}
|
|
113
|
+
}
|
|
114
|
+
|
|
115
|
+
const quantum = source.encoding === 'terrarium' ? 1 / 256 : source.interval;
|
|
116
|
+
const inQuanta = steps.map((s) => Math.round(s / quantum));
|
|
117
|
+
const counts = new Map();
|
|
118
|
+
for (const q of inQuanta) counts.set(q, (counts.get(q) ?? 0) + 1);
|
|
119
|
+
|
|
120
|
+
console.log(`${url}`);
|
|
121
|
+
console.log(
|
|
122
|
+
` ${width}x${width}, ${source.encoding}, one quantum = ${quantum} m`,
|
|
123
|
+
);
|
|
124
|
+
console.log(` heights ${low.toFixed(2)} m to ${high.toFixed(2)} m`);
|
|
125
|
+
console.log(` exactly zero: ${zeroes} of ${known} pixels`);
|
|
126
|
+
|
|
127
|
+
console.log('\n step between neighbouring pixels:');
|
|
128
|
+
const ordered = [...counts.entries()].sort((a, b) => a[0] - b[0]).slice(0, 8);
|
|
129
|
+
for (const [q, n] of ordered) {
|
|
130
|
+
const share = n / steps.length;
|
|
131
|
+
console.log(
|
|
132
|
+
` ${String(q).padStart(4)} quanta (${(q * quantum).toFixed(2)} m) ` +
|
|
133
|
+
`${'#'.repeat(Math.round(share * 40)).padEnd(40)} ${(share * 100).toFixed(1)}%`,
|
|
134
|
+
);
|
|
135
|
+
}
|
|
136
|
+
|
|
137
|
+
// A median cannot see sparse spikes, and sparse spikes are what stipple a
|
|
138
|
+
// hillshade: a scattering of pixels standing well clear of their neighbours,
|
|
139
|
+
// too few to move the middle of any distribution. So the tail is what gets
|
|
140
|
+
// reported, and separately the count of pixels that disagree with everything
|
|
141
|
+
// around them.
|
|
142
|
+
const flat = [];
|
|
143
|
+
let spikes = 0;
|
|
144
|
+
let worstSpike = 0;
|
|
145
|
+
for (let row = 1; row < width - 1; row += 1) {
|
|
146
|
+
for (let column = 1; column < width - 1; column += 1) {
|
|
147
|
+
const at = row * width + column;
|
|
148
|
+
const here = heights[at];
|
|
149
|
+
if (!Number.isFinite(here)) continue;
|
|
150
|
+
|
|
151
|
+
const across = heights[at + 1] - 2 * here + heights[at - 1];
|
|
152
|
+
const down = heights[at + width] - 2 * here + heights[at - width];
|
|
153
|
+
if (Number.isFinite(across) && Number.isFinite(down)) {
|
|
154
|
+
flat.push(Math.abs(across) + Math.abs(down));
|
|
155
|
+
}
|
|
156
|
+
|
|
157
|
+
// Standing clear of every neighbour, in the same direction. Terrain does
|
|
158
|
+
// not do this; a pixel that survived a mask its neighbours did not, or one
|
|
159
|
+
// the encoding placed a quantum out, does.
|
|
160
|
+
let above = 0;
|
|
161
|
+
let below = 0;
|
|
162
|
+
let seen = 0;
|
|
163
|
+
let nearest = Infinity;
|
|
164
|
+
for (const [dy, dx] of [
|
|
165
|
+
[-1, -1],
|
|
166
|
+
[-1, 0],
|
|
167
|
+
[-1, 1],
|
|
168
|
+
[0, -1],
|
|
169
|
+
[0, 1],
|
|
170
|
+
[1, -1],
|
|
171
|
+
[1, 0],
|
|
172
|
+
[1, 1],
|
|
173
|
+
]) {
|
|
174
|
+
const neighbour = heights[(row + dy) * width + column + dx];
|
|
175
|
+
if (!Number.isFinite(neighbour)) continue;
|
|
176
|
+
seen += 1;
|
|
177
|
+
if (here > neighbour) above += 1;
|
|
178
|
+
else if (here < neighbour) below += 1;
|
|
179
|
+
nearest = Math.min(nearest, Math.abs(here - neighbour));
|
|
180
|
+
}
|
|
181
|
+
if (seen === 8 && (above === 8 || below === 8) && nearest > quantum * 1.5) {
|
|
182
|
+
spikes += 1;
|
|
183
|
+
worstSpike = Math.max(worstSpike, nearest);
|
|
184
|
+
}
|
|
185
|
+
}
|
|
186
|
+
}
|
|
187
|
+
|
|
188
|
+
flat.sort((one, two) => one - two);
|
|
189
|
+
/**
|
|
190
|
+
* A value at a percentile of the sorted roughness.
|
|
191
|
+
* @param {number} share - Where to look, 0 to 1.
|
|
192
|
+
* @returns {number} - Metres.
|
|
193
|
+
*/
|
|
194
|
+
const at = (share) =>
|
|
195
|
+
flat[Math.min(flat.length - 1, Math.floor(flat.length * share))] ?? 0;
|
|
196
|
+
|
|
197
|
+
// How much ground one pixel covers, so roughness can be read as a slope.
|
|
198
|
+
// Web mercator shrinks with latitude, and the tile's row is where that comes
|
|
199
|
+
// from -- a metre between neighbours means one thing at z8 and another at z16.
|
|
200
|
+
const rows = 2 ** zoom;
|
|
201
|
+
// Clamped: a row outside the pyramid would put the latitude past the pole and
|
|
202
|
+
// report no ground at all, which reads as a broken tile rather than a typo.
|
|
203
|
+
const row = Math.min(Math.max(tileRow, 0), rows - 1);
|
|
204
|
+
const latitude = Math.atan(Math.sinh(Math.PI * (1 - (2 * row) / rows)));
|
|
205
|
+
const groundMetres = (40075016.686 * Math.cos(latitude)) / rows / width;
|
|
206
|
+
|
|
207
|
+
console.log('\n roughness, as the second difference between neighbours:');
|
|
208
|
+
for (const [label, share] of [
|
|
209
|
+
['half of them are under', 0.5],
|
|
210
|
+
['nine in ten under', 0.9],
|
|
211
|
+
['ninety-nine in a hundred under', 0.99],
|
|
212
|
+
['the worst', 1],
|
|
213
|
+
]) {
|
|
214
|
+
console.log(
|
|
215
|
+
` ${label.padEnd(32)} ${at(share).toFixed(2).padStart(9)} m` +
|
|
216
|
+
` (${(at(share) / quantum).toFixed(0)} quanta)`,
|
|
217
|
+
);
|
|
218
|
+
}
|
|
219
|
+
|
|
220
|
+
console.log(
|
|
221
|
+
`\n relief across this tile: ${(high - low).toFixed(1)} m` +
|
|
222
|
+
`, about ${groundMetres.toFixed(0)} m of ground per pixel`,
|
|
223
|
+
);
|
|
224
|
+
console.log(
|
|
225
|
+
` pixels standing clear of all eight neighbours: ${spikes}` +
|
|
226
|
+
` (${((spikes / known) * 100).toFixed(2)}%)` +
|
|
227
|
+
(spikes ? `, worst ${worstSpike.toFixed(1)} m` : ''),
|
|
228
|
+
);
|
|
229
|
+
|
|
230
|
+
console.log('');
|
|
231
|
+
if (spikes / known > 0.002 && worstSpike > quantum * 10) {
|
|
232
|
+
console.log(
|
|
233
|
+
' -> a scattering of pixels stands well clear of everything around it.\n' +
|
|
234
|
+
' That is what stipples a hillshade, and terrain does not do it. Look\n' +
|
|
235
|
+
' for something applied per pixel: a mask that matched some pixels and\n' +
|
|
236
|
+
' not their neighbours, or two sources meeting one pixel at a time.',
|
|
237
|
+
);
|
|
238
|
+
} else if (at(0.99) <= quantum * 2.5) {
|
|
239
|
+
console.log(
|
|
240
|
+
" -> smooth to the encoding's own resolution, in the tail as well as the\n" +
|
|
241
|
+
' middle. A checkerboard here is the colour ramp dithering across a\n' +
|
|
242
|
+
' band edge, and a hillshade exaggerating a tenth of a metre.',
|
|
243
|
+
);
|
|
244
|
+
} else {
|
|
245
|
+
console.log(
|
|
246
|
+
` -> rough, but evenly so, against ${(high - low).toFixed(0)} m of relief across\n` +
|
|
247
|
+
' the tile. That is what terrain looks like; compare a tile of flat\n' +
|
|
248
|
+
' ground before calling it noise.',
|
|
249
|
+
);
|
|
250
|
+
}
|