pmtiles-swarm 0.75.0 → 0.77.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -7,6 +7,73 @@
7
7
  ### 🐞 Bug fixes
8
8
  - _...Add new stuff here..._
9
9
 
10
+ ## 0.77.0
11
+ ### ✨ Features and improvements
12
+ - **A fade can be written in metres of ground.** `featherMetres` on a source says how far to blend
13
+ it in as a distance, and the merge works out the pixels for each tile it builds -
14
+ `40075016.686 x cos(latitude) / 2^zoom / tileSize` of them. What a fade has to hide is two sources
15
+ disagreeing about the height of the same ground, which is a fixed number of metres, while a fade
16
+ in pixels is a different distance at every zoom.
17
+
18
+ It matters because a hillshade reads slope rather than height. `feather: 8` over a 7 m
19
+ disagreement is a gradient of 0.08 at z12 and 1.28 at z16 - invisible at one end and a saturated
20
+ band at the other, wider than the cliff it replaced and no less visible. The same 50 m holds 0.14
21
+ at every zoom, which is ordinary hillside. Below the zoom where the fade is a pixel wide it rounds
22
+ to nothing, and it is capped at a quarter of the tile - 128 pixels on a 512px grid, which is where
23
+ the old 64 came from. `featherMeters` is read as well, and the console's fade field now takes a
24
+ unit rather than always meaning pixels.
25
+
26
+ - **A tool that measures the step a fade has to hide.** `tools/coast-step.mjs` reports the height
27
+ difference where one source hands over to another, and says which kind of disagreement it is.
28
+ Steps that cluster are a vertical datum offset, which one `heightAdjustment` corrects. Steps that
29
+ scatter are a coarse source averaging land and water together in every cell that straddles a
30
+ coast - worst at the shore, gone offshore - which no single number corrects and a fade can only
31
+ hide.
32
+
33
+ ### 🐞 Bug fixes
34
+ - **A mask range was not an edge to fade at.** A source whose only mask was a `maskRange` was told
35
+ its `feather` had nothing to act on, and refused - while the merge had been fading exactly that
36
+ edge all along. The recipe validation and the console's own warning both listed `maskValues` and
37
+ `maskColors` and neither had been taught about the band.
38
+
39
+ ## 0.76.0
40
+ ### ✨ Features and improvements
41
+ - **A stopped export stays stopped.** Stopping one left a checkpoint, and a checkpoint says what was
42
+ in progress but not why it stopped - so the next restart could not tell somebody pressing Stop
43
+ from a crash, and picked it up again. Stopping now writes a marker beside the checkpoint, and a
44
+ restart reports what it is holding rather than resuming it. Starting or resuming clears the marker.
45
+
46
+ And the work can be thrown away, which it could not be before: `DELETE /api/stacks/:id/bake/work`
47
+ removes the working directory, which for an abandoned export is hundreds of gigabytes of buffered
48
+ tiles that previously had to be found by hand. Refused while a merge is running rather than pulled
49
+ out from under it.
50
+
51
+ ### 🐞 Bug fixes
52
+ - **An edit inside one clock tick went unnoticed.** `stacks.json` is re-read when its modification
53
+ time changes, and a filesystem's clock is coarser than an edit: on NTFS the tick is about 15 ms,
54
+ measured here at 36 rapid rewrites in 40 landing on the same timestamp. Two edits that close
55
+ together left the mtime alone and the second was never read.
56
+
57
+ The size is compared as well now, which catches the ones that changed the file's length - most of
58
+ them - and costs nothing, since the stat was already being made.
59
+
60
+ This is also what made the reload test flake, roughly one run in three: it wrote the file twice in
61
+ quick succession and then depended on how those two writes fell against the clock. It stamps both
62
+ writes to the same instant now and asserts they really are the same, so it tests whether a change
63
+ is noticed rather than whether the clock happened to tick.
64
+ - **A stack with a shallow global source served holes above z14.** How far the merge would climb
65
+ for a source with no tile at this zoom was a fixed six levels. GEBCO is z0-8 and the sea floor has
66
+ no more detail to give, so a stack serving z16 has to upscale that z8 tile eight levels - and at
67
+ z15 the climb stopped one short of the only tile that existed. Over open water, where the other
68
+ source was sparse and had nothing either, no source contributed at all and the stack correctly
69
+ answered no-tile. A rectangular hole, one tile wide, in the middle of the sea.
70
+
71
+ It is derived now rather than fixed: the deepest zoom the stack serves, less the shallowest source
72
+ under it. Nothing to set and nothing to get wrong - the right answer is computable, and a smaller
73
+ one would only punch holes. Somebody who wants the merge to stop climbing says so with `maxzoom`,
74
+ which stops the stack serving that deep at all: the same wish, said where it also stops the work.
75
+ Never below the old six, so no stack reaches less far than it did.
76
+
10
77
  ## 0.75.0
11
78
  ### ✨ Features and improvements
12
79
  - **`maskRange`, because nodata is a band and not a number.** `maskValues` and `maskColors` both
@@ -50,6 +50,8 @@ of its parts.
50
50
  - [Starting one, and watching it](#starting-one-and-watching-it)
51
51
  - [What exists now](#what-exists-now)
52
52
  - [Clipping a source to a shape](#clipping-a-source-to-a-shape)
53
+ - [What a mask has to match](#what-a-mask-has-to-match)
54
+ - [Feathering a seam](#feathering-a-seam)
53
55
  - [Finding a stack](#finding-a-stack)
54
56
  - [Syncing a stack to another node](#syncing-a-stack-to-another-node)
55
57
  - [What the offline merge got wrong](#what-the-offline-merge-got-wrong)
@@ -233,19 +235,20 @@ it differs from the snake_case rio-rgbify-merge uses.
233
235
 
234
236
  ### Source fields
235
237
 
236
- | Field | Meaning |
237
- | ---------------------- | ---------------------------------------------------------------------------------------------------------- |
238
- | `category` | Resolve to the newest build in this category. |
239
- | `archive` | Or pin one infohash. Exactly one of the two. |
240
- | `required` | A tile fails rather than being served without this source. Defaults true for the bottom-most, false above. |
241
- | `encoding` | `mapbox` or `terrarium`. Elevation space only. |
242
- | `baseVal` / `interval` | Mapbox decode offset and step. Default `-10000` and `0.1`. |
243
- | `maskValues` | Decoded heights meaning "no data here". Elevation space only. |
244
- | `maskRange` | `[low, high]` in metres, or a list of them. Everything inside is nodata. Elevation space only. |
245
- | `heightAdjustment` | Metres, added **after** masking. Elevation space only. |
246
- | `feather` | Pixels to fade in over at the edge of the source's shape. Needs a `cutline` or `bounds`. Max 64. |
247
- | `opacity` | `0`–`1`, scales the source alpha. RGBA space only. |
248
- | `blend` | `normal`, `multiply`, `screen`, `overlay`, `darken`, `lighten`. RGBA space only. |
238
+ | Field | Meaning |
239
+ | ---------------------- | ------------------------------------------------------------------------------------------------------------ |
240
+ | `category` | Resolve to the newest build in this category. |
241
+ | `archive` | Or pin one infohash. Exactly one of the two. |
242
+ | `required` | A tile fails rather than being served without this source. Defaults true for the bottom-most, false above. |
243
+ | `encoding` | `mapbox` or `terrarium`. Elevation space only. |
244
+ | `baseVal` / `interval` | Mapbox decode offset and step. Default `-10000` and `0.1`. |
245
+ | `maskValues` | Decoded heights meaning "no data here". Elevation space only. |
246
+ | `maskRange` | `[low, high]` in metres, or a list of them. Everything inside is nodata. Elevation space only. |
247
+ | `heightAdjustment` | Metres, added **after** masking. Elevation space only. |
248
+ | `feather` | Pixels to fade in over wherever the source stops: a `cutline`, `bounds`, or the holes a mask leaves. Max 64. |
249
+ | `featherMetres` | The same fade written as metres of ground, worked out per tile. Usually the one to reach for. |
250
+ | `opacity` | `0`–`1`, scales the source alpha. RGBA space only. |
251
+ | `blend` | `normal`, `multiply`, `screen`, `overlay`, `darken`, `lighten`. RGBA space only. |
249
252
 
250
253
  `attribution` is not optional in practice. A stack is a derived work of every
251
254
  source in it, and the thing that reliably gets lost when tiles are combined is
@@ -1085,8 +1088,8 @@ standing clear of their neighbours, which is the shape this makes.
1085
1088
 
1086
1089
  ## Feathering a seam
1087
1090
 
1088
- _Built for a cutline and for `bounds`. A mask edge is not feathered yet, and
1089
- the last section says what that still needs._
1091
+ _Built for a cutline, for `bounds`, and for the holes a mask leaves. A source
1092
+ that vanishes at a tile edge is still open, and the last section says why._
1090
1093
 
1091
1094
  Smoothing hides the terracing **inside** an upscaled area. It does nothing about
1092
1095
  the artefact the original discussion actually named — "artefacts at tiles and
@@ -1236,18 +1239,40 @@ lookup rather than a decode — and it is not done.
1236
1239
 
1237
1240
  `feather` on a source, in pixels, `0` and absent meaning the edge is a switch as
1238
1241
  before. It fades that source in over that many pixels measured inward from
1239
- wherever it stops — the holes `maskValues` and `maskColors` leave, and the edge
1240
- of a `cutline` or `bounds`:
1242
+ wherever it stops — the holes `maskValues`, `maskRange` and `maskColors` leave,
1243
+ and the edge of a `cutline` or `bounds`:
1241
1244
 
1242
1245
  ```json
1243
1246
  { "archive": "swissalti", "maskValues": [0], "feather": 16 }
1244
1247
  ```
1245
1248
 
1246
- The step left at the seam is the height difference divided by the feather, which
1247
- is what makes the number predictable: two sources 40 m apart at the border, faded
1248
- over 16 pixels, step 2.5 m a pixel instead of 40 m at once. Sixty-four is the
1249
- most it takes — past that the ramp does not reach full weight anywhere inside a
1250
- 256px tile, and the source is being turned down rather than blended in.
1249
+ The step left at the seam is the height difference divided by the feather: two
1250
+ sources 40 m apart at the border, faded over 16 pixels, step 2.5 m a pixel
1251
+ instead of 40 m at once. Sixty-four is the most it takes — past that the ramp
1252
+ does not reach full weight anywhere inside a 256px tile, and the source is being
1253
+ turned down rather than blended in.
1254
+
1255
+ **A smaller step is not the same as an invisible one**, and that is the thing to
1256
+ know before picking a number. A hillshade does not read height, it reads slope:
1257
+ the drop divided by the ground underneath it. What decides whether a seam
1258
+ disappears is therefore metres per metre rather than metres per pixel — and a
1259
+ pixel is a different number of metres at every zoom. The same `feather: 8`, over
1260
+ the same 7 m disagreement, at 55°N:
1261
+
1262
+ | zoom | m/pixel | gradient |
1263
+ | ---- | ------- | -------- |
1264
+ | z12 | 11.0 | 0.08 |
1265
+ | z13 | 5.5 | 0.16 |
1266
+ | z14 | 2.7 | 0.32 |
1267
+ | z15 | 1.4 | 0.64 |
1268
+ | z16 | 0.7 | 1.28 |
1269
+
1270
+ Natural terrain is rarely over about 0.5 and a hillshade saturates around there,
1271
+ so a fade that vanishes at z12 is a bright band by z15 — wider than the cliff it
1272
+ replaced and no less visible. Widening it does help at any one zoom, since the
1273
+ gradient is the drop over the whole fade; the difficulty is that the number
1274
+ which works at z16 is eight times the one that works at z13, and a recipe has
1275
+ one number.
1251
1276
 
1252
1277
  Three things fall out of the implementation and are worth stating.
1253
1278
 
@@ -1268,22 +1293,52 @@ envelope — linear in the tile's width, and chosen over a chamfer approximation
1268
1293
  because a chamfer's error is largest on diagonals and a national boundary is
1269
1294
  mostly diagonals.
1270
1295
 
1271
- ### What it does not do yet
1296
+ ### A fade in metres
1297
+
1298
+ `featherMetres` says the same thing as a distance on the ground, and the merge
1299
+ works out the pixels for each tile it builds:
1272
1300
 
1273
- **A mask edge.** `maskValues` and `maskColors` are where most seams actually come
1274
- from, and they are not feathered. The reason is the margin: a cutline is known in
1275
- full, so the ramp beside a tile can be computed by rasterising a few extra rows,
1276
- while a mask edge lives in the source's own pixels and the ramp needs the
1277
- neighbouring tiles to be read and decoded — up to eight of them, for that source,
1278
- on that tile.
1279
-
1280
- That is the piece with a real cost attached, and the one to measure before
1281
- building. Two things make it more tractable than it looks: in a bake the tiles
1282
- are walked in Hilbert order, which is chosen for locality, so a cache of decoded
1283
- source tiles would be hit far more often than missed; and the ramp is smooth by
1284
- construction, so it can be computed from the source's _parent_ at half resolution
1285
- and upscaled, which turns eight reads at this zoom into a handful at the one
1286
- above, shared between four children.
1301
+ ```json
1302
+ { "archive": "gebco", "maskRange": [-1, 0], "featherMetres": 50 }
1303
+ ```
1304
+
1305
+ Web Mercator makes that arithmetic rather than a lookup: one pixel covers
1306
+ `40075016.686 × cos(latitude) / 2^zoom / tileSize` metres, so the conversion
1307
+ needs a multiply and the coordinates of the tile being built. `featherMeters` is
1308
+ read as well, because a key that quietly does nothing when spelled the other way
1309
+ is the failure this feature is most prone to.
1310
+
1311
+ Fifty metres, over the same 7 m disagreement, at 55°N on 512px tiles:
1312
+
1313
+ | zoom | m/pixel | pixels | ground | gradient |
1314
+ | ---- | ------- | ------ | ------ | -------- |
1315
+ | z12 | 11.0 | 5 | 55 m | 0.13 |
1316
+ | z13 | 5.5 | 9 | 49 m | 0.14 |
1317
+ | z14 | 2.7 | 18 | 49 m | 0.14 |
1318
+ | z15 | 1.4 | 36 | 49 m | 0.14 |
1319
+ | z16 | 0.7 | 73 | 50 m | 0.14 |
1320
+
1321
+ One number, the same slope at every zoom, and 0.14 is ordinary hillside rather
1322
+ than an edge.
1323
+
1324
+ It stops being exact at both ends, benignly. Below the zoom where the fade is a
1325
+ pixel wide it rounds to nothing — 50 m at z8 is a sixth of a pixel, and the whole
1326
+ coastline is inside one pixel there anyway. Above a quarter of the tile it is
1327
+ capped, which is 128 pixels on a 512px grid and is where the old 64 came from:
1328
+ past that the ramp reaches full weight nowhere inside the tile. At 55°N the cap
1329
+ first binds at z17.
1330
+
1331
+ **Picking the number.** It is the disagreement being hidden that sets it, not the
1332
+ coastline being drawn across. `tools/coast-step.mjs` measures the first against a
1333
+ real tile: it reports the step where one source hands over to the other, and
1334
+ whether those steps cluster — in which case they are a datum offset and
1335
+ `heightAdjustment` is the honest fix — or scatter, in which case nothing corrects
1336
+ them and a fade is for hiding what cannot be corrected. Divide the step by the
1337
+ gradient wanted: 7 m at 0.15 is about 50 m, and the same step at 0.05 would need
1338
+ 140 m, which is wide enough to start flattening ground either side of the coast
1339
+ that was never wrong.
1340
+
1341
+ ### What it does not do yet
1287
1342
 
1288
1343
  **A source that vanishes at a tile edge.** A sparse archive simply has no tile
1289
1344
  outside its extent, so the seam lands exactly on a tile boundary — the most
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "pmtiles-swarm",
3
- "version": "0.75.0",
3
+ "version": "0.77.0",
4
4
  "description": "BitTorrent distribution for PMTiles map archives: create torrents, watch folders, publish and subscribe to RSS feeds, and seed through qBittorrent or an embedded client",
5
5
  "type": "module",
6
6
  "main": "src/index.js",
package/src/api.js CHANGED
@@ -3303,6 +3303,31 @@ export function createApp({
3303
3303
  }),
3304
3304
  );
3305
3305
 
3306
+ /**
3307
+ * Throws away what a stopped export had done.
3308
+ *
3309
+ * Separate from stopping, and deliberately a second decision. Stopping keeps
3310
+ * the work because an export may be hours in and somebody may want it back;
3311
+ * this is for when they do not, and until now the only way to be rid of
3312
+ * hundreds of gigabytes of buffered tiles was to find the directory by hand.
3313
+ */
3314
+ app.delete(
3315
+ '/api/stacks/:id/bake/work',
3316
+ route(async (req, res) => {
3317
+ if (!bakes) {
3318
+ return res.status(501).json({ error: 'this node does not bake' });
3319
+ }
3320
+ const discarded = await bakes.discard(req.params.id);
3321
+ if (!discarded) {
3322
+ return res.status(409).json({
3323
+ error:
3324
+ 'there is no stopped export for that stack, or one is still running',
3325
+ });
3326
+ }
3327
+ return res.json({ discarded: true });
3328
+ }),
3329
+ );
3330
+
3306
3331
  /**
3307
3332
  * Creates or replaces a stack.
3308
3333
  *
package/src/bake-jobs.js CHANGED
@@ -7,7 +7,11 @@ import {
7
7
  bakeStack,
8
8
  bakedArchiveName,
9
9
  bakedName,
10
+ clearStopped,
11
+ discardCheckpoint,
12
+ markStopped,
10
13
  mergeTileFor,
14
+ wasStopped,
11
15
  } from './bake.js';
12
16
  import { PixelWorker } from './pixels.js';
13
17
  import { outputFormat } from './stack-tile.js';
@@ -124,6 +128,10 @@ export class BakeManager {
124
128
  #loadCodec;
125
129
  #cutlines;
126
130
  #jobs = new Map();
131
+ // Exports somebody stopped, whose work is still on disk. Kept so the console
132
+ // can offer to resume or discard them: after a restart there is no job in
133
+ // memory, only a directory nobody would think to look for.
134
+ #held = new Map();
127
135
 
128
136
  /**
129
137
  * @param {object} deps - The library, the tile store, the config and the codec probe.
@@ -152,11 +160,36 @@ export class BakeManager {
152
160
  */
153
161
  get(stackId) {
154
162
  const job = this.#jobs.get(stackId);
155
- return job ? this.#describe(job) : null;
163
+ if (job) return this.#describe(job);
164
+
165
+ // Nothing running, but work somebody stopped may still be there. Reported
166
+ // in the same shape so the console has one thing to read.
167
+ const stopped = this.#held.get(stackId);
168
+ if (!stopped) return null;
169
+ return {
170
+ stackId,
171
+ phase: 'stopped',
172
+ written: stopped.written,
173
+ resumable: true,
174
+ ...stopped.describe,
175
+ };
176
+ }
177
+
178
+ /**
179
+ * Every stack with work waiting that nobody has decided about.
180
+ * @returns {string[]} - Their ids.
181
+ */
182
+ heldStacks() {
183
+ return [...this.#held.keys()];
156
184
  }
157
185
 
158
186
  /**
159
- * Stops a bake, leaving its work where the next run can pick it up.
187
+ * Stops a bake, leaving its work for somebody to pick up or throw away.
188
+ *
189
+ * Marked as stopped on purpose, so the next start leaves it alone. The work
190
+ * is still there and `resume` takes it up again -- what changes is who
191
+ * decides, which for a job that may be hours from finishing should be a
192
+ * person rather than a restart.
160
193
  * @param {string} stackId - Which stack.
161
194
  * @returns {boolean} - True if there was one to stop.
162
195
  */
@@ -164,10 +197,46 @@ export class BakeManager {
164
197
  const job = this.#jobs.get(stackId);
165
198
  if (!job || job.finishedAt) return false;
166
199
  job.cancelling = true;
200
+ job.stoppedOnPurpose = true;
167
201
  job.controller.abort();
202
+ // Not awaited: the abort has to reach the merge now, and a mark written a
203
+ // moment later is still written long before anything restarts.
204
+ markStopped(workDirFor(job, this.#config)).catch(() => {});
168
205
  return true;
169
206
  }
170
207
 
208
+ /**
209
+ * Throws away what a stopped export had done.
210
+ *
211
+ * The counterpart to stopping. An export that will not be finished leaves
212
+ * hundreds of gigabytes of buffered tiles behind, and until now the only way
213
+ * to be rid of them was to find the directory by hand.
214
+ * @param {string} stackId - Which stack.
215
+ * @returns {Promise<boolean>} - True if there was work to discard.
216
+ */
217
+ async discard(stackId) {
218
+ const running = this.#jobs.get(stackId);
219
+ // Refused rather than raced. Removing the directory under a running merge
220
+ // would have it fail on its next write, reporting a disk problem for
221
+ // something somebody chose.
222
+ if (running && !running.finishedAt) return false;
223
+
224
+ let found = false;
225
+ for (const root of this.#workRoots()) {
226
+ const directory = path.join(root, WORK_DIR, stackId);
227
+ const there = await fs
228
+ .access(directory)
229
+ .then(() => true)
230
+ .catch(() => false);
231
+ if (!there) continue;
232
+ await discardCheckpoint(directory);
233
+ found = true;
234
+ }
235
+ this.#jobs.delete(stackId);
236
+ this.#held.delete(stackId);
237
+ return found;
238
+ }
239
+
171
240
  /**
172
241
  * Picks up exports a previous run did not finish.
173
242
  *
@@ -203,6 +272,21 @@ export class BakeManager {
203
272
  const resolved = resolve(stackId);
204
273
  if (!resolved || bakeRevision(resolved) !== state.revision) continue;
205
274
 
275
+ // Somebody stopped this one. It stays where it is until they say
276
+ // otherwise -- an export begun again by a restart is the opposite of
277
+ // what pressing Stop meant.
278
+ if (await wasStopped(path.join(directory, stackId))) {
279
+ this.#held.set(stackId, {
280
+ written: state.written ?? 0,
281
+ describe: state.describe,
282
+ });
283
+ console.log(
284
+ `[bake] ${stackId} was stopped on purpose; leaving its ` +
285
+ `${state.written ?? 0} tiles for you to resume or discard`,
286
+ );
287
+ continue;
288
+ }
289
+
206
290
  try {
207
291
  const job = await this.start({ resolved, ...state.describe });
208
292
  started.push(job);
@@ -367,6 +451,12 @@ export class BakeManager {
367
451
  const destination = path.join(workDir, job.name);
368
452
  const format = outputFormat(resolved);
369
453
 
454
+ // Running again is the answer to having been stopped, so the mark goes.
455
+ // Left behind, an export somebody restarted by hand would be passed over
456
+ // by the next restart, which is the same surprise the other way round.
457
+ await clearStopped(workDir);
458
+ this.#held.delete(job.stackId);
459
+
370
460
  // Sized with the batch, so every merge in flight has a thread to do its
371
461
  // arithmetic on rather than queueing behind one. Only where there is pixel
372
462
  // work to move: a passthrough bake hands bytes straight through and would
package/src/bake.js CHANGED
@@ -42,6 +42,7 @@ const DEFAULT_CONCURRENCY = 4;
42
42
  /** What the working directory holds while a bake is in progress. */
43
43
  const FILES = Object.freeze({
44
44
  state: 'bake-state.json',
45
+ stopped: 'bake-stopped.json',
45
46
  entries: 'bake-entries.bin',
46
47
  tiles: 'bake-tiles.bin',
47
48
  });
@@ -59,6 +60,64 @@ export function checkpointPaths(workDir) {
59
60
  };
60
61
  }
61
62
 
63
+ /**
64
+ * Records that an export was stopped on purpose.
65
+ *
66
+ * A checkpoint says what was in progress, not why it stopped, so a crash and
67
+ * somebody pressing Stop look identical on disk -- and the node picks both up
68
+ * on the next start. That is right for the crash and wrong for the person, who
69
+ * stopped it and then watched it begin again.
70
+ *
71
+ * Written beside the checkpoint rather than into it, so it cannot be lost to a
72
+ * half-finished write of the state the export is still making.
73
+ * @param {string} workDir - The bake's working directory.
74
+ * @returns {Promise<void>} - Resolves once it is written.
75
+ */
76
+ export async function markStopped(workDir) {
77
+ await fs
78
+ .writeFile(
79
+ path.join(workDir, FILES.stopped),
80
+ JSON.stringify({ at: new Date().toISOString() }),
81
+ )
82
+ .catch(() => {});
83
+ }
84
+
85
+ /**
86
+ * Forgets that an export was stopped, because it is running again.
87
+ * @param {string} workDir - The bake's working directory.
88
+ * @returns {Promise<void>} - Resolves once it is gone.
89
+ */
90
+ export async function clearStopped(workDir) {
91
+ await fs
92
+ .rm(path.join(workDir, FILES.stopped), { force: true })
93
+ .catch(() => {});
94
+ }
95
+
96
+ /**
97
+ * Whether an export was stopped on purpose rather than interrupted.
98
+ * @param {string} workDir - The bake's working directory.
99
+ * @returns {Promise<boolean>} - True when somebody stopped it.
100
+ */
101
+ export async function wasStopped(workDir) {
102
+ return fs
103
+ .access(path.join(workDir, FILES.stopped))
104
+ .then(() => true)
105
+ .catch(() => false);
106
+ }
107
+
108
+ /**
109
+ * Forgets an export's unfinished work.
110
+ *
111
+ * The whole directory, because a checkpoint is only meaningful with the tiles
112
+ * it names -- leaving either behind is leaving something that will be picked
113
+ * up and found wanting.
114
+ * @param {string} workDir - The bake's working directory.
115
+ * @returns {Promise<void>} - Resolves once it is gone.
116
+ */
117
+ export async function discardCheckpoint(workDir) {
118
+ await fs.rm(workDir, { recursive: true, force: true });
119
+ }
120
+
62
121
  /**
63
122
  * One entry, as a checkpoint stores it.
64
123
  *
package/src/cutline.js CHANGED
@@ -45,6 +45,31 @@ export function worldY(lat) {
45
45
  );
46
46
  }
47
47
 
48
+ /** The world across the equator, in metres: what a zoom 0 tile spans. */
49
+ const EQUATOR = 40075016.686;
50
+
51
+ /**
52
+ * How much ground one pixel of a tile covers, at that tile's latitude.
53
+ *
54
+ * Web Mercator holds a pixel to a fixed fraction of the world, so the ground
55
+ * under it shrinks toward the poles and halves at every zoom. That is why a
56
+ * fade written in pixels is a different width of ground everywhere it is used,
57
+ * and this is the conversion that lets one written in metres mean the same
58
+ * thing. Taken at the middle of the tile: the scale changes across it, but a
59
+ * tile is a small piece of the world at any zoom where a fade is more than a
60
+ * pixel wide.
61
+ * @param {number} z - Zoom.
62
+ * @param {number} y - Row.
63
+ * @param {number} size - Pixels per side of the tile.
64
+ * @returns {number} - Metres per pixel.
65
+ */
66
+ export function metresPerPixel(z, y, size) {
67
+ const tiles = 2 ** z;
68
+ const middle = (y + 0.5) / tiles;
69
+ const latitude = Math.atan(Math.sinh(Math.PI * (1 - 2 * middle)));
70
+ return (EQUATOR * Math.cos(latitude)) / (tiles * size);
71
+ }
72
+
48
73
  /**
49
74
  * Reads the rings out of GeoJSON, whatever shape it arrived in.
50
75
  *
package/src/cutlines.js CHANGED
@@ -1,6 +1,6 @@
1
1
  import fs from 'node:fs/promises';
2
2
  import path from 'node:path';
3
- import { fromBounds, fromGeoJSON } from './cutline.js';
3
+ import { fromBounds, fromGeoJSON, metresPerPixel } from './cutline.js';
4
4
 
5
5
  /**
6
6
  * The cutlines a node has, by name.
@@ -101,16 +101,63 @@ export class CutlineStore {
101
101
  */
102
102
  export const MAX_FEATHER = 64;
103
103
 
104
+ /** Past this a fade in metres is a typo rather than a distance. */
105
+ export const MAX_FEATHER_METRES = 100000;
106
+
107
+ /**
108
+ * The widest fade a grid of this size will take.
109
+ *
110
+ * A quarter of the tile, which is where 64 came from and what it still means on
111
+ * a 512px one: past that the ramp reaches full weight nowhere inside the tile,
112
+ * and the source is being turned down rather than blended in. It binds on a
113
+ * fade in metres, which asks for more pixels at every zoom.
114
+ * @param {number} size - Pixels per side of the grid the ramp runs on.
115
+ * @returns {number} - The most pixels a fade may be.
116
+ */
117
+ function capFor(size) {
118
+ return Math.max(MAX_FEATHER, Math.round((size ?? 0) / 4));
119
+ }
120
+
104
121
  /**
105
- * How far a source fades in at the edge of its shape.
122
+ * How far a source fades in, as metres of ground, if it is written that way.
123
+ *
124
+ * Both spellings are read. A recipe key that does nothing when it is spelled
125
+ * the other way is the failure this whole feature is most prone to, and the
126
+ * prose here says metres while half the world's config files say meters.
127
+ * @param {object} recipe - One source out of a recipe.
128
+ * @returns {number} - Metres, 0 when the fade is not written in them.
129
+ */
130
+ export function featherMetresFor(recipe) {
131
+ const asked = Number(recipe?.featherMetres ?? recipe?.featherMeters);
132
+ return Number.isFinite(asked) && asked > 0 ? asked : 0;
133
+ }
134
+
135
+ /**
136
+ * How far a source fades in at the edge of its shape, in this tile's pixels.
106
137
  *
107
138
  * Zero is the old behaviour and the default: the shape is a switch, and a pixel
108
139
  * is either the source or what is underneath it. Anything more makes it a ramp.
109
- * See docs/tile-stacks.md -- "Feathering a seam".
140
+ *
141
+ * A fade in metres is converted here, which is why this wants the tile. What
142
+ * the fade has to hide is two sources disagreeing about the height of the same
143
+ * ground, which is a fixed number of metres -- so the same number of pixels is
144
+ * too wide at one zoom and too steep at the next. See docs/tile-stacks.md --
145
+ * "Feathering a seam".
110
146
  * @param {object} recipe - One source out of a recipe.
147
+ * @param {object} [tile] - `{z, y, size}` of the tile being built.
111
148
  * @returns {number} - Pixels, 0 when it does not fade.
112
149
  */
113
- export function featherFor(recipe) {
150
+ export function featherFor(recipe, tile) {
151
+ const size = tile?.size > 0 ? tile.size : 256;
152
+ const metres = featherMetresFor(recipe);
153
+ if (metres > 0) {
154
+ // With no tile there is no scale to convert against, and a fade that
155
+ // guessed one would be a different width from the tiles beside it.
156
+ if (!Number.isFinite(tile?.z)) return 0;
157
+ const perPixel = metresPerPixel(tile.z, tile.y ?? 0, size);
158
+ return Math.min(Math.round(metres / perPixel), capFor(size));
159
+ }
160
+
114
161
  const asked = Number(recipe?.feather);
115
162
  if (!Number.isFinite(asked) || asked <= 0) return 0;
116
163
  return Math.min(Math.round(asked), MAX_FEATHER);
package/src/stack-tile.js CHANGED
@@ -36,9 +36,42 @@ import { TileReadError } from './tiles.js';
36
36
  * a tile in an archive or a hole where one is not needed.
37
37
  */
38
38
 
39
- /** How far up the pyramid a merge will climb for a source with no tile here. */
39
+ /**
40
+ * How far up the pyramid a merge will climb for a source with no tile here,
41
+ * where the stack does not say and nothing can be worked out.
42
+ */
40
43
  const PARENT_LIMIT = 6;
41
44
 
45
+ /**
46
+ * How far this stack has to climb for its shallowest source to keep working.
47
+ *
48
+ * A global source is shallow on purpose -- GEBCO is z0-8 and the sea floor has
49
+ * no more detail to give -- so serving a stack to z16 means upscaling that z8
50
+ * tile eight levels. A fixed limit truncates exactly the arrangement this
51
+ * feature is for: at z15 the climb stopped one level short of the only tile
52
+ * that exists, no source contributed, and the stack answered no-tile over open
53
+ * water.
54
+ *
55
+ * So it is derived from what the stack spans rather than assumed or set: the
56
+ * right answer is computable, and a recipe naming a smaller one would only
57
+ * punch holes in itself. Somebody who wants the merge to stop climbing says so
58
+ * with `maxzoom`, which stops the stack serving that deep at all -- the same
59
+ * wish, said where it also stops the work.
60
+ *
61
+ * Never below the old fixed limit, so no stack reaches less far than it did.
62
+ * @param {object} resolved - The resolved stack.
63
+ * @returns {number} - Levels a source may climb.
64
+ */
65
+ export function parentLimitFor(resolved) {
66
+ const { maxzoom } = stackCoverage(resolved);
67
+ const shallowest = resolved.sources
68
+ .map((source) => source.entry?.pmtiles?.maxZoom)
69
+ .filter((zoom) => Number.isFinite(zoom));
70
+ if (!shallowest.length || !Number.isFinite(maxzoom)) return PARENT_LIMIT;
71
+
72
+ return Math.max(PARENT_LIMIT, maxzoom - Math.min(...shallowest));
73
+ }
74
+
42
75
  /** The deepest zoom a tile id is defined for. */
43
76
  const MAX_ZOOM = 26;
44
77
 
@@ -118,7 +151,7 @@ export function clipsFor(resolved, cutlines, z, x, y, size = 256) {
118
151
  // The feather reaches inward from the edge, so a tile wholly inside but
119
152
  // near it still has a ramp across part of itself and cannot take the cheap
120
153
  // answer.
121
- const feather = featherFor(recipe);
154
+ const feather = featherFor(recipe, { z, y, size });
122
155
  return {
123
156
  shape,
124
157
  feather,
@@ -141,8 +174,8 @@ export function clipsFor(resolved, cutlines, z, x, y, size = 256) {
141
174
  * @param {object} options - Source, coordinates, the tile store and whether to climb.
142
175
  * @returns {Promise<object|null>} - The tile and the zoom it came from.
143
176
  */
144
- async function readFrom({ source, z, x, y, tiles, climb, signal }) {
145
- const floor = climb ? Math.max(0, z - PARENT_LIMIT) : z;
177
+ async function readFrom({ source, z, x, y, tiles, climb, signal, limit }) {
178
+ const floor = climb ? Math.max(0, z - (limit ?? PARENT_LIMIT)) : z;
146
179
  for (let at = z; at >= floor; at -= 1) {
147
180
  const shift = z - at;
148
181
  const tile = await tiles.getTile(
@@ -322,6 +355,7 @@ export function passThroughRead({
322
355
  * @returns {Promise<object>} - `{contributors, contributions}` or `{error}`.
323
356
  */
324
357
  async function readAll({ resolved, z, x, y, tiles, signal, clips }) {
358
+ const limit = parentLimitFor(resolved);
325
359
  return Promise.all(
326
360
  resolved.sources.map(async (source, index) => {
327
361
  if (!source.entry) return { source, found: null };
@@ -340,6 +374,7 @@ async function readAll({ resolved, z, x, y, tiles, signal, clips }) {
340
374
  tiles,
341
375
  climb: true,
342
376
  signal,
377
+ limit,
343
378
  }),
344
379
  };
345
380
  } catch (error) {
@@ -489,10 +524,11 @@ async function readMaskEdges({
489
524
  contributions.map(async (contribution) => {
490
525
  if (!contribution) return;
491
526
  const recipe = contribution.source ?? {};
492
- const feather = featherFor(recipe);
493
- if (!feather || !masksAnything(recipe)) return;
494
-
527
+ // On the source's own grid rather than the output's, because that is
528
+ // what the ramp will be measured on.
495
529
  const grid = contribution.raster?.width ?? size;
530
+ const feather = featherFor(recipe, { z, y, size: grid });
531
+ if (!feather || !masksAnything(recipe)) return;
496
532
  const layout = parentsFor({ z, x, y }, grid, feather);
497
533
  if (!layout) return;
498
534
 
@@ -577,10 +613,14 @@ async function merge({
577
613
  // decides what the pixels inside it weigh. The shape is known in full, so
578
614
  // this costs the extra rows and nothing else -- unlike a mask read out of
579
615
  // a source's own pixels, which would need its neighbours.
580
- const margin = clip.feather ?? 0;
616
+ //
617
+ // Asked again rather than taken from the clip: `clipsFor` may have been
618
+ // given a grid the merge did not end up using, and a fade in metres is a
619
+ // different number of pixels on each of them.
620
+ const margin = featherFor(contribution.source, { z, y, size: grid });
581
621
  const mask = rasterizeTile(clip.shape, z, x, y, grid, margin);
582
622
  contribution.coverage = cropMask(
583
- featherMask(mask, grid + margin * 2, clip.feather ?? 0),
623
+ featherMask(mask, grid + margin * 2, margin),
584
624
  grid,
585
625
  margin,
586
626
  );
@@ -694,7 +734,8 @@ export async function answerStackTile(options) {
694
734
  // 256 where the request did not say, which is the cautious guess: the
695
735
  // margin is in pixels of the output grid, so assuming a smaller grid makes
696
736
  // it wider on the ground, and too wide only costs a rasterise that was not
697
- // needed while too narrow misses a ramp.
737
+ // needed while too narrow misses a ramp. A fade in metres does not mind
738
+ // either way -- fewer pixels each covering more ground is the same ground.
698
739
  const clips = clipsFor(resolved, cutlines, z, x, y, size);
699
740
  const format = options.format ?? outputFormat(resolved);
700
741
  const rgba = resolved.stack.space === 'rgba';
package/src/stacks.js CHANGED
@@ -8,7 +8,11 @@ import {
8
8
  isResampling,
9
9
  parseColor,
10
10
  } from './elevation.js';
11
- import { MAX_FEATHER } from './cutlines.js';
11
+ import {
12
+ MAX_FEATHER,
13
+ MAX_FEATHER_METRES,
14
+ featherMetresFor,
15
+ } from './cutlines.js';
12
16
  import { normalizeCategories } from './catalog.js';
13
17
  import { BLEND_MODES, isBlendMode } from './rgba.js';
14
18
 
@@ -42,6 +46,8 @@ import { BLEND_MODES, isBlendMode } from './rgba.js';
42
46
  * @property {number} [heightAdjustment] - Metres, added after masking.
43
47
  * @property {number} [feather] - Pixels to fade in over at the edge of the
44
48
  * source's shape. 0, the default, makes the edge a switch.
49
+ * @property {number} [featherMetres] - Metres of ground to fade in over
50
+ * instead, worked out per tile. `featherMeters` is read as well.
45
51
  * @property {number} [opacity] - 0-1, scales source alpha. RGBA only.
46
52
  * @property {string} [blend] - Blend operator. RGBA only.
47
53
  * @typedef {object} Stack
@@ -159,6 +165,18 @@ export function validateStack(stack) {
159
165
  );
160
166
  }
161
167
  }
168
+ const metres = source?.featherMetres ?? source?.featherMeters;
169
+ if (metres !== undefined) {
170
+ // Loose, because the pixel cap is what actually bounds the ramp at any
171
+ // one zoom. This only has to catch a number nobody meant to type.
172
+ const width = Number(metres);
173
+ if (!Number.isFinite(width) || width < 0 || width > MAX_FEATHER_METRES) {
174
+ problems.push(
175
+ `sources[${index}].featherMetres must be between 0 and ` +
176
+ `${MAX_FEATHER_METRES}`,
177
+ );
178
+ }
179
+ }
162
180
  // A fade with no edge to fade at. Refused rather than ignored: it reads as
163
181
  // a source that blends into what is under it, and it does nothing at all.
164
182
  // A mask is an edge as much as a cutline is -- the hole it leaves is where
@@ -167,8 +185,9 @@ export function validateStack(stack) {
167
185
  source?.cutline ||
168
186
  source?.bounds ||
169
187
  source?.maskValues?.length ||
170
- source?.maskColors?.length;
171
- if (source?.feather && !fades) {
188
+ source?.maskColors?.length ||
189
+ source?.maskRange?.length;
190
+ if ((source?.feather || featherMetresFor(source)) && !fades) {
172
191
  problems.push(
173
192
  `sources[${index}].feather needs a cutline, bounds, or a mask to fade at`,
174
193
  );
@@ -528,6 +547,7 @@ export class StackStore {
528
547
  #stacks = new Map();
529
548
  #problems = new Map();
530
549
  #mtime = null;
550
+ #size = null;
531
551
  #checkedAt = 0;
532
552
 
533
553
  /**
@@ -552,10 +572,13 @@ export class StackStore {
552
572
  let raw;
553
573
  try {
554
574
  raw = JSON.parse(await fs.readFile(this.#file, 'utf8'));
555
- this.#mtime = (await fs.stat(this.#file)).mtimeMs;
575
+ const stat = await fs.stat(this.#file);
576
+ this.#mtime = stat.mtimeMs;
577
+ this.#size = stat.size;
556
578
  } catch (error) {
557
579
  if (error.code === 'ENOENT') {
558
580
  this.#mtime = null;
581
+ this.#size = null;
559
582
  return;
560
583
  }
561
584
  throw error;
@@ -588,9 +611,15 @@ export class StackStore {
588
611
  if (now - this.#checkedAt < 1000) return false;
589
612
  this.#checkedAt = now;
590
613
 
614
+ // Size as well as the timestamp. A filesystem's clock is coarser than an
615
+ // edit: on NTFS the tick is about 15 ms, so two writes in quick succession
616
+ // land on the same mtime and the second would never be seen. Comparing the
617
+ // length as well catches the ones that changed it, which is most of them --
618
+ // and it costs nothing, since the stat was made anyway.
591
619
  const stat = await fs.stat(this.#file).catch(() => null);
592
620
  const mtime = stat?.mtimeMs ?? null;
593
- if (mtime === this.#mtime) return false;
621
+ const size = stat?.size ?? null;
622
+ if (mtime === this.#mtime && size === this.#size) return false;
594
623
  await this.load();
595
624
  return true;
596
625
  }
@@ -676,6 +705,8 @@ export class StackStore {
676
705
  `,
677
706
  );
678
707
  await fs.rename(temp, this.#file);
679
- this.#mtime = (await fs.stat(this.#file)).mtimeMs;
708
+ const written = await fs.stat(this.#file);
709
+ this.#mtime = written.mtimeMs;
710
+ this.#size = written.size;
680
711
  }
681
712
  }
@@ -1905,18 +1905,21 @@
1905
1905
  );
1906
1906
 
1907
1907
  /**
1908
- * A mask range as the field shows it.
1908
+ * One end of a mask range, for the box that shows it.
1909
1909
  *
1910
- * One pair is written plainly; a recipe carrying several is shown as
1911
- * JSON, because the editor offers one band and the file may hold more
1912
- * than the editor can draw.
1910
+ * A recipe may hold several bands where the editor offers one. The first
1911
+ * is what the boxes show, and the row says how many more there are --
1912
+ * silently dropping them would be worse than saying the editor cannot
1913
+ * draw them.
1913
1914
  * @param {*} range - What the recipe says.
1915
+ * @param {number} which - 0 for the low end, 1 for the high.
1914
1916
  * @returns {string} - What to put in the box.
1915
1917
  */
1916
- const rangeText = (range) => {
1918
+ const rangeEdge = (range, which) => {
1917
1919
  if (!Array.isArray(range) || range.length === 0) return '';
1918
- if (Array.isArray(range[0])) return JSON.stringify(range);
1919
- return range.slice(0, 2).join(', ');
1920
+ const first = Array.isArray(range[0]) ? range[0] : range;
1921
+ const edge = first[which];
1922
+ return Number.isFinite(Number(edge)) ? String(edge) : '';
1920
1923
  };
1921
1924
 
1922
1925
  /**
@@ -7542,6 +7545,18 @@ Every piece is hashed against the ` +
7542
7545
  */
7543
7546
  const renderDraftSource = (source, index, rgba) => {
7544
7547
  const last = index === stackDraft.sources.length - 1;
7548
+ // A fade is written in one unit or the other, never both: the box
7549
+ // shows whichever the recipe used and the choice beside it says which.
7550
+ const featherMetres = source.featherMetres ?? source.featherMeters;
7551
+ const featherUnit =
7552
+ featherMetres === undefined && source.feather ? 'px' : 'm';
7553
+ const featherWidth = featherMetres ?? source.feather ?? '';
7554
+ const fadesAt =
7555
+ source.cutline ||
7556
+ source.bounds ||
7557
+ (source.maskValues ?? []).length ||
7558
+ (source.maskColors ?? []).length ||
7559
+ (source.maskRange ?? []).length;
7545
7560
  const role =
7546
7561
  stackDraft.sources.length === 1
7547
7562
  ? ''
@@ -7635,11 +7650,23 @@ Every piece is hashed against the ` +
7635
7650
  value="${escapeHtml((source.maskValues ?? []).join(', '))}" />
7636
7651
  </label>
7637
7652
  <label class="choice"
7638
- title="Mask every height inside a band, written low, high. Nodata is rarely one number: an archive resampled on its way to being built does not hold what it was authored with, so a sea authored as 0 arrives scattered across -0.9 m to 0 — and masking the two ends of that leaves everything between, standing proud of whatever is underneath. A band says what you mean, and asymmetrically: sea is everything up to zero and nothing above it.">
7639
- Mask range
7640
- <input style="width:9rem" placeholder="-1, 0"
7641
- data-stack-field="maskRange" data-stack-index="${index}"
7642
- value="${escapeHtml(rangeText(source.maskRange))}" /> m
7653
+ title="Mask every height between these two, inclusive. Separate from Mask heights, which takes values one at a time — nodata is often a band rather than a number, because an archive resampled on its way to being built does not hold what it was authored with. A sea authored as 0 arrives scattered across -0.9 m to 0, and masking the two ends of that leaves everything between, standing proud of whatever is underneath. Leave both empty for no band.">
7654
+ Mask between
7655
+ <input type="number" step="any" style="width:5.5rem" placeholder="low"
7656
+ data-stack-field="maskRangeLow" data-stack-index="${index}"
7657
+ value="${rangeEdge(source.maskRange, 0)}" />
7658
+ and
7659
+ <input type="number" step="any" style="width:5.5rem" placeholder="high"
7660
+ data-stack-field="maskRangeHigh" data-stack-index="${index}"
7661
+ value="${rangeEdge(source.maskRange, 1)}" /> m
7662
+ ${
7663
+ Array.isArray(source.maskRange) &&
7664
+ Array.isArray(source.maskRange[0]) &&
7665
+ source.maskRange.length > 1
7666
+ ? `<span class="sub">and ${source.maskRange.length - 1} more,
7667
+ which editing here replaces</span>`
7668
+ : ''
7669
+ }
7643
7670
  </label>`;
7644
7671
 
7645
7672
  return `
@@ -7695,19 +7722,23 @@ Every piece is hashed against the ` +
7695
7722
  : ''
7696
7723
  }
7697
7724
  <label class="choice"
7698
- title="Fade this source in over this many pixels wherever it stops — the holes a mask leaves, and the edge of a cutline or box. Where a high-resolution source meets a coarser one the two disagree, by their survey and by their vertical datum, and the step between them reads as a wall under a hillshade. The drop is divided by this: 40 m across an 8 px fade is 5 m a pixel. 0 is a hard edge.">
7725
+ title="Fade this source in wherever it stops — the holes a mask leaves, and the edge of a cutline or box. Where a high-resolution source meets a coarser one the two disagree, by their survey and by their vertical datum, and the step between them reads as a wall under a hillshade. Metres of ground is usually the one to pick: a hillshade reads slope, so a fade written in pixels is gentle at one zoom and a bright band at the next, while a distance on the ground holds the same slope at every zoom. 50 m suits a coastline where two sources are a few metres apart. 0 is a hard edge.">
7699
7726
  Fade in
7700
- <input type="number" min="0" max="64" step="1" style="width:4.5rem"
7727
+ <input type="number" min="0" step="any" style="width:4.5rem"
7701
7728
  placeholder="0"
7702
7729
  data-stack-field="feather" data-stack-index="${index}"
7703
- value="${source.feather ?? ''}" />
7704
- <span class="sub">px${
7705
- source.feather &&
7706
- !source.cutline &&
7707
- !source.bounds &&
7708
- !(source.maskValues ?? []).length &&
7709
- !(source.maskColors ?? []).length
7710
- ? ' — nothing to fade at yet: mask something, or clip it'
7730
+ value="${featherWidth}" />
7731
+ <select data-stack-field="featherUnit" data-stack-index="${index}">
7732
+ <option value="m"${
7733
+ featherUnit === 'm' ? ' selected' : ''
7734
+ }>m of ground</option>
7735
+ <option value="px"${
7736
+ featherUnit === 'px' ? ' selected' : ''
7737
+ }>px</option>
7738
+ </select>
7739
+ <span class="sub">${
7740
+ featherWidth && !fadesAt
7741
+ ? 'nothing to fade at yet: mask something, or clip it'
7711
7742
  : ''
7712
7743
  }</span>
7713
7744
  </label>
@@ -7761,20 +7792,40 @@ Every piece is hashed against the ` +
7761
7792
  .filter(Boolean);
7762
7793
  } else if (field === 'required') {
7763
7794
  source.required = event.target.checked;
7764
- } else if (field === 'maskRange') {
7765
- // Held while it is half-typed, the way the clip box is: four numbers
7766
- // arrive one keystroke at a time and a field that cleared itself
7767
- // after the first comma could never be filled in.
7768
- const numbers = event.target.value
7769
- .split(',')
7770
- .map((part) => Number(part.trim()))
7771
- .filter((n) => Number.isFinite(n));
7772
- if (numbers.length >= 2) source.maskRange = numbers.slice(0, 2);
7773
- else if (event.target.value.trim() === '') delete source.maskRange;
7774
- } else if (field === 'feather') {
7775
- const pixels = Number(event.target.value);
7776
- if (event.target.value === '' || !(pixels > 0)) delete source.feather;
7777
- else source.feather = Math.min(64, Math.round(pixels));
7795
+ } else if (field === 'maskRangeLow' || field === 'maskRangeHigh') {
7796
+ // A band needs both ends, so one on its own is a half-typed thought
7797
+ // rather than a range -- read them together, and write nothing until
7798
+ // there are two numbers to write.
7799
+ const row = event.target.closest('.card');
7800
+ const edge = (name) => {
7801
+ const box = row?.querySelector(`[data-stack-field="${name}"]`);
7802
+ return box && box.value.trim() !== '' ? Number(box.value) : null;
7803
+ };
7804
+ const low = edge('maskRangeLow');
7805
+ const high = edge('maskRangeHigh');
7806
+ if (Number.isFinite(low) && Number.isFinite(high)) {
7807
+ source.maskRange = low <= high ? [low, high] : [high, low];
7808
+ } else if (low === null && high === null) {
7809
+ delete source.maskRange;
7810
+ }
7811
+ } else if (field === 'feather' || field === 'featherUnit') {
7812
+ // The width and its unit are read together and only one key is
7813
+ // written, because a source carrying a fade in both would be asking
7814
+ // the merge a question it has no answer to.
7815
+ const row = event.target.closest('.card');
7816
+ const box = row?.querySelector('[data-stack-field="feather"]');
7817
+ const unit = row?.querySelector('[data-stack-field="featherUnit"]');
7818
+ const width = Number(box?.value);
7819
+ delete source.feather;
7820
+ delete source.featherMetres;
7821
+ delete source.featherMeters;
7822
+ if (box && box.value.trim() !== '' && width > 0) {
7823
+ if (unit?.value === 'px') {
7824
+ source.feather = Math.min(64, Math.round(width));
7825
+ } else {
7826
+ source.featherMetres = width;
7827
+ }
7828
+ }
7778
7829
  } else if (field === 'baseVal' || field === 'interval') {
7779
7830
  // Blank means "the default", not zero -- an interval of 0 would make
7780
7831
  // every height the base value.
@@ -7812,6 +7863,12 @@ Every piece is hashed against the ` +
7812
7863
  renderStackDraft();
7813
7864
  return;
7814
7865
  }
7866
+ // Redrawn so the row's own warning about having nothing to fade at
7867
+ // follows the unit rather than the value it was drawn with.
7868
+ if (event.target.dataset.stackField === 'featherUnit') {
7869
+ renderStackDraft();
7870
+ return;
7871
+ }
7815
7872
  if (event.target.dataset.stackField !== 'encoding') return;
7816
7873
  // Redrawn so the four custom fields appear or disappear with the
7817
7874
  // choice, rather than sitting there meaning nothing.
@@ -0,0 +1,200 @@
1
+ #!/usr/bin/env node
2
+ /**
3
+ * How far two sources disagree where one hands over to the other.
4
+ *
5
+ * A land DEM says its shoreline is about zero. A global bathymetry at 450 m
6
+ * cells averages land and water together in every cell that straddles a coast,
7
+ * so its shoreline sits well below that -- and the merge steps between them in
8
+ * one pixel. Feathering spreads that step without reducing it, and a hillshade
9
+ * reads slope, so at high zoom a spread step is a wider bright band rather
10
+ * than a fainter one.
11
+ *
12
+ * Two quite different things produce that step and they look identical in a
13
+ * picture. A vertical datum offset is the same wherever the two meet, and one
14
+ * `heightAdjustment` fixes it. Averaging is wrong by whatever each cell
15
+ * happened to cover -- worst at the shore, gone offshore -- and no single
16
+ * number fixes that; subtracting the median would meet the coast and lift the
17
+ * open ocean with it.
18
+ *
19
+ * They are told apart by how much the steps scatter, which is what this
20
+ * prints. It is worth knowing that the measurement is taken exactly where a
21
+ * coarse source is least trustworthy, so a clustered result is evidence and a
22
+ * scattered one is close to proof.
23
+ *
24
+ * node tools/coast-step.mjs --stack <url> --land <url> [--range -1,0] [--values 0]
25
+ *
26
+ * `--stack` is the merged tile and `--land` the source that wins on land, both
27
+ * at the same z/x/y. The mask arguments are that source's, so the tool knows
28
+ * which of its pixels are sea.
29
+ */
30
+
31
+ import { loadCodec } from '../src/codec.js';
32
+ import { decodeHeights, maskHeights, maskRanges } from '../src/elevation.js';
33
+
34
+ const args = process.argv.slice(2);
35
+
36
+ /**
37
+ * One named option off the command line.
38
+ * @param {string} name - Its name, without the dashes.
39
+ * @param {string} [fallback] - What it is when absent.
40
+ * @returns {string|undefined} - The value.
41
+ */
42
+ const option = (name, fallback) => {
43
+ const at = args.indexOf(`--${name}`);
44
+ return at >= 0 && args[at + 1] ? args[at + 1] : fallback;
45
+ };
46
+
47
+ const stackUrl = option('stack');
48
+ const landUrl = option('land');
49
+ if (!stackUrl || !landUrl) {
50
+ console.error(
51
+ 'usage: node tools/coast-step.mjs --stack <url> --land <url> [--range low,high] [--values a,b]',
52
+ );
53
+ process.exit(2);
54
+ }
55
+
56
+ /** The masks the land source is configured with, so its sea can be found. */
57
+ const values = option('values')
58
+ ? option('values').split(',').map(Number)
59
+ : undefined;
60
+ const range = option('range')
61
+ ? option('range').split(',').map(Number)
62
+ : undefined;
63
+ if (!values && !range) {
64
+ console.error(
65
+ 'give --values, --range, or both: without them no pixel is sea',
66
+ );
67
+ process.exit(2);
68
+ }
69
+
70
+ const codec = await loadCodec();
71
+ if (!codec) {
72
+ console.error('this needs sharp: npm install sharp');
73
+ process.exit(1);
74
+ }
75
+
76
+ /**
77
+ * One tile, decoded to metres.
78
+ * @param {string} url - Where it is.
79
+ * @returns {Promise<object>} - `{heights, width}`.
80
+ */
81
+ async function tileAt(url) {
82
+ const response = await fetch(url);
83
+ if (!response.ok) {
84
+ console.error(`${url} answered ${response.status}`);
85
+ process.exit(1);
86
+ }
87
+ const raster = await codec.decode(Buffer.from(await response.arrayBuffer()), {
88
+ channels: 3,
89
+ });
90
+ return { heights: decodeHeights(raster, {}), width: raster.width };
91
+ }
92
+
93
+ const merged = await tileAt(stackUrl);
94
+ const land = await tileAt(landUrl);
95
+ if (merged.width !== land.width) {
96
+ console.error(
97
+ `different sizes: the stack is ${merged.width}px and the land source ${land.width}px`,
98
+ );
99
+ process.exit(1);
100
+ }
101
+
102
+ // The land source's own idea of where it has nothing, which is where the other
103
+ // source shows through in the merged tile.
104
+ const sea = Float32Array.from(land.heights);
105
+ maskHeights(sea, values);
106
+ maskRanges(sea, range);
107
+
108
+ const { width } = merged;
109
+ const steps = [];
110
+ for (let row = 1; row < width - 1; row += 1) {
111
+ for (let column = 1; column < width - 1; column += 1) {
112
+ const at = row * width + column;
113
+ // A pixel the land source covers, beside one it does not: the handover,
114
+ // found from that source rather than guessed from the merged heights.
115
+ if (Number.isNaN(sea[at])) continue;
116
+ for (const step of [-1, 1, -width, width]) {
117
+ if (!Number.isNaN(sea[at + step])) continue;
118
+ const above = merged.heights[at];
119
+ const below = merged.heights[at + step];
120
+ if (Number.isFinite(above) && Number.isFinite(below)) {
121
+ steps.push(above - below);
122
+ }
123
+ }
124
+ }
125
+ }
126
+
127
+ if (steps.length === 0) {
128
+ console.log('no handover in this tile: it is all land, or all sea');
129
+ process.exit(0);
130
+ }
131
+
132
+ steps.sort((one, two) => one - two);
133
+
134
+ /**
135
+ * A value at a percentile of the sorted steps.
136
+ * @param {number} share - Where to look, 0 to 1.
137
+ * @returns {number} - Metres.
138
+ */
139
+ const at = (share) =>
140
+ steps[Math.min(steps.length - 1, Math.floor(steps.length * share))];
141
+
142
+ const median = at(0.5);
143
+ const spread = at(0.75) - at(0.25);
144
+
145
+ console.log(`${steps.length} pixels where one source hands over to the other`);
146
+ console.log('\n the step across it, land minus what is underneath:');
147
+ for (const [label, share] of [
148
+ ['a quarter are under', 0.25],
149
+ ['half are under', 0.5],
150
+ ['three quarters under', 0.75],
151
+ ['the worst', 1],
152
+ ]) {
153
+ console.log(` ${label.padEnd(24)} ${at(share).toFixed(1).padStart(8)} m`);
154
+ }
155
+
156
+ console.log(
157
+ `\n median ${median.toFixed(1)} m, middle half spread over ${spread.toFixed(1)} m`,
158
+ );
159
+
160
+ // Whether one number could fix it. A datum offset is the same wherever the two
161
+ // meet, so the steps cluster; a coarse source averaging land and water is wrong
162
+ // by whatever its cell covered, so they scatter.
163
+ if (Math.abs(spread) < Math.abs(median) * 0.5) {
164
+ console.log(
165
+ [
166
+ '',
167
+ ' -> the steps cluster, so the two disagree by about the same amount',
168
+ ' wherever they meet. That looks like a datum offset, which one',
169
+ ' number fixes. On the source underneath, not the land:',
170
+ '',
171
+ ` "heightAdjustment": ${median.toFixed(1)}`,
172
+ '',
173
+ ' It shifts that source everywhere, so the deep ocean floor moves by',
174
+ ' the same amount -- against thousands of metres, nothing. Re-run',
175
+ ' afterwards and the median should come out near zero.',
176
+ '',
177
+ ' Worth one check first: this was measured at the coastline, which is',
178
+ ' the one place a coarse source is guaranteed to be wrong. If its',
179
+ ' cells are much larger than a pixel here, a clustered result can',
180
+ ' still be averaging rather than datum -- compare a stretch of steep',
181
+ ' coast against a flat one, and a real offset will agree.',
182
+ ].join('\n'),
183
+ );
184
+ } else {
185
+ console.log(
186
+ [
187
+ '',
188
+ ' -> the steps scatter more than they cluster, so this is not one offset.',
189
+ ' A coarse source averages land and water together in the cells that',
190
+ ' straddle a coast, so how wrong it is depends on what each cell',
191
+ ' happened to cover: worst at the shore, gone offshore. Subtracting',
192
+ ' the median would meet the coast and lift the open ocean with it,',
193
+ ' trading a seam you can see for an error you cannot.',
194
+ '',
195
+ ' What helps is a nearshore source that has the detail, or leaving the',
196
+ ' line alone. At this zoom the coarse source is being asked something',
197
+ ' its cells cannot answer, and no adjustment invents the answer.',
198
+ ].join('\n'),
199
+ );
200
+ }