pmtiles-swarm 0.76.0 → 0.78.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -7,6 +7,66 @@
7
7
  ### 🐞 Bug fixes
8
8
  - _...Add new stuff here..._
9
9
 
10
+ ## 0.78.0
11
+ ### ✨ Features and improvements
12
+ - **Somewhere to clear the caches and the files nothing is waiting for.** A merged tile cache with
13
+ no way to empty it is a directory somebody eventually finds by hand, and a temporary file a crash
14
+ left behind is one nobody finds at all. Settings has a **Storage** tab now: what this node is
15
+ holding, what each thing costs to lose, and a button per row.
16
+
17
+ Five things — merged stack tiles, left-over temporary files, stopped exports, traffic history and
18
+ the tile counters. `GET /api/storage` reports them, `DELETE /api/storage/:what` lets go of one.
19
+ Everything on the list is derived and can be rebuilt, which is what makes a button reasonable:
20
+ none of it asks whether you meant it, because none of it is the only copy of anything.
21
+
22
+ The archives are deliberately not on it, and neither is the resume data beside them. Both look
23
+ like housekeeping and neither is: retiring an archive is a decision made from its own panel with
24
+ what it seeds in view, and resume data thrown away is a rehash of every byte on disk.
25
+
26
+ A sweep picks files by name and by age together, because either alone is wrong: `*.tmp` and
27
+ `pmtiles-write-*`, untouched for an hour. Every write that uses one renames within milliseconds,
28
+ so the margin is not for slowness — it is because this runs while the node is serving, and a sweep
29
+ with none at all could take the file a catalog write is halfway through renaming into place.
30
+ `torrents-data` is skipped outright, being terabytes of payload with no working files in it.
31
+
32
+ Stopped exports are read off the disk rather than from what the process remembers, so one left by
33
+ a stack somebody has since deleted is found as well. A running export is left alone: removing the
34
+ directory under a running merge would have it fail on its next write, reporting a disk problem for
35
+ something somebody chose. This is also the first way to discard one without finding the directory
36
+ by hand — the API for it shipped in 0.76.0 with nothing calling it.
37
+
38
+ ### 🐞 Bug fixes
39
+ - _...Add new stuff here..._
40
+
41
+ ## 0.77.0
42
+ ### ✨ Features and improvements
43
+ - **A fade can be written in metres of ground.** `featherMetres` on a source says how far to blend
44
+ it in as a distance, and the merge works out the pixels for each tile it builds -
45
+ `40075016.686 x cos(latitude) / 2^zoom / tileSize` of them. What a fade has to hide is two sources
46
+ disagreeing about the height of the same ground, which is a fixed number of metres, while a fade
47
+ in pixels is a different distance at every zoom.
48
+
49
+ It matters because a hillshade reads slope rather than height. `feather: 8` over a 7 m
50
+ disagreement is a gradient of 0.08 at z12 and 1.28 at z16 - invisible at one end and a saturated
51
+ band at the other, wider than the cliff it replaced and no less visible. The same 50 m holds 0.14
52
+ at every zoom, which is ordinary hillside. Below the zoom where the fade is a pixel wide it rounds
53
+ to nothing, and it is capped at a quarter of the tile - 128 pixels on a 512px grid, which is where
54
+ the old 64 came from. `featherMeters` is read as well, and the console's fade field now takes a
55
+ unit rather than always meaning pixels.
56
+
57
+ - **A tool that measures the step a fade has to hide.** `tools/coast-step.mjs` reports the height
58
+ difference where one source hands over to another, and says which kind of disagreement it is.
59
+ Steps that cluster are a vertical datum offset, which one `heightAdjustment` corrects. Steps that
60
+ scatter are a coarse source averaging land and water together in every cell that straddles a
61
+ coast - worst at the shore, gone offshore - which no single number corrects and a fade can only
62
+ hide.
63
+
64
+ ### 🐞 Bug fixes
65
+ - **A mask range was not an edge to fade at.** A source whose only mask was a `maskRange` was told
66
+ its `feather` had nothing to act on, and refused - while the merge had been fading exactly that
67
+ edge all along. The recipe validation and the console's own warning both listed `maskValues` and
68
+ `maskColors` and neither had been taught about the band.
69
+
10
70
  ## 0.76.0
11
71
  ### ✨ Features and improvements
12
72
  - **A stopped export stays stopped.** Stopping one left a checkpoint, and a checkpoint says what was
package/README.md CHANGED
@@ -786,6 +786,8 @@ which the endpoint answers 501.
786
786
  | `GET` `POST` `DELETE` | `/api/tokens`, `/api/tokens/:id` | Mint, list and revoke access tokens |
787
787
  | `GET` `POST` | `/api/restart` | What a restart would do, and doing it |
788
788
  | `GET` `DELETE` | `/api/stats` | What this node has served — per-archive counters and the last N requests; `DELETE` clears them |
789
+ | `GET` | `/api/storage` | What this node is holding that it could let go of — merged tiles, left-over temporary files, stopped exports, traffic history, tile counters |
790
+ | `DELETE` | `/api/storage/:what` | Lets go of one of them. Everything listed is derived and can be rebuilt; the archives are not on the list |
789
791
  | `GET` | `/api/traffic` | Upload and download speed per archive over time, from `stats.db`; `infoHash`, `hours` and `buckets` narrow it, and `totals` ranks archives by bytes moved |
790
792
  | `GET` `PATCH` | `/api/config` | Read and change settings |
791
793
  | `POST` | `/api/login`, `/api/logout` | Console sign-in |
package/docs/internals.md CHANGED
@@ -33,6 +33,7 @@ Operator-facing documentation is elsewhere — see [publishing](publishing.md),
33
33
  - [Retiring old builds](#retiring-old-builds)
34
34
  - [Prewarming a freshly joined archive](#prewarming-a-freshly-joined-archive)
35
35
  - [Named save locations](#named-save-locations)
36
+ - [Storage](#storage)
36
37
 
37
38
  ## Marking incomplete archives
38
39
 
@@ -884,3 +885,60 @@ moving several hundred gigabytes is not something a settings screen should do as
884
885
  a side effect. A move checks free space before the engine is disturbed, since
885
886
  running out halfway through costs an hour, a partial file to clean up, and an
886
887
  archive that has to be put back.
888
+
889
+ ## Storage
890
+
891
+ A merged tile cache with no way to empty it is a directory somebody eventually
892
+ finds by hand, and a temporary file a crash left behind is one nobody finds at
893
+ all. `GET /api/storage` reports what a node is holding that it could let go of;
894
+ `DELETE /api/storage/:what` lets go of one of them. The console draws it as the
895
+ Storage tab under Settings.
896
+
897
+ Five things, and what each costs to lose:
898
+
899
+ | What | Where | Clearing it costs |
900
+ | ------------------------- | ------------------ | -------------------------------------------------------------- |
901
+ | Merged stack tiles | `stack-cache/` | The merge again: an archive read per source, and a decode each |
902
+ | Left-over temporary files | the data directory | Nothing — nobody is waiting for a file a crashed write left |
903
+ | Stopped exports | the archive's disk | The hours already spent, which is what resuming would use |
904
+ | Traffic history | `stats.db` | The graphs go back to empty |
905
+ | Tile counters | memory | The count starts again; frees nothing |
906
+
907
+ Everything on that list is derived and can be rebuilt, which is what makes a
908
+ button reasonable — none of it asks whether the operator meant it, because none
909
+ of it is the only copy of anything.
910
+
911
+ ### What is deliberately not on it
912
+
913
+ **The archives**, and the resume data beside them. Both look like housekeeping
914
+ and neither is. An archive is the data this node exists to serve, and retiring
915
+ one is a decision made from its own panel with what it seeds in view. Resume
916
+ data thrown away is a rehash of every byte on disk — and worse than useless as
917
+ a repair, since a _stale partial_ resume file is the thing that cancels seed
918
+ mode, which is a diagnosis rather than a sweep.
919
+
920
+ ### Which files a sweep is willing to touch
921
+
922
+ By name and by age together, because either alone is wrong: the name says what
923
+ a file was for and the age says whether anything still cares.
924
+
925
+ The names are `*.tmp` and `pmtiles-write-*`, which is what the write-then-rename
926
+ in `catalog.js`, `stacks.js`, `dht-state.js`, `stack-cache.js` and
927
+ `pmtiles-write.js` leaves behind when a process stops between the two steps. The
928
+ age is an hour. Every one of those renames happens within milliseconds, so an
929
+ hour is far past generous — the margin is not for slowness, it is because this
930
+ runs while the node is serving and a sweep with no margin could delete the file
931
+ a catalog write is halfway through renaming into place.
932
+
933
+ `torrents-data/` is skipped outright. It is terabytes of payload, nothing in it
934
+ is a working file, and a sweep that walks it is a sweep that spends minutes
935
+ finding nothing.
936
+
937
+ ### The bake working directories are read from disk
938
+
939
+ Not from what the process remembers. A working directory outlives the run that
940
+ made it, and one belonging to a stack somebody has since deleted is exactly the
941
+ kind nobody thinks to look for. Whether an export is _running_ is memory's to
942
+ say, and that is the only thing that decides whether it may be discarded —
943
+ removing the directory under a running merge would have it fail on its next
944
+ write, reporting a disk problem for something somebody chose.
@@ -50,6 +50,8 @@ of its parts.
50
50
  - [Starting one, and watching it](#starting-one-and-watching-it)
51
51
  - [What exists now](#what-exists-now)
52
52
  - [Clipping a source to a shape](#clipping-a-source-to-a-shape)
53
+ - [What a mask has to match](#what-a-mask-has-to-match)
54
+ - [Feathering a seam](#feathering-a-seam)
53
55
  - [Finding a stack](#finding-a-stack)
54
56
  - [Syncing a stack to another node](#syncing-a-stack-to-another-node)
55
57
  - [What the offline merge got wrong](#what-the-offline-merge-got-wrong)
@@ -233,19 +235,20 @@ it differs from the snake_case rio-rgbify-merge uses.
233
235
 
234
236
  ### Source fields
235
237
 
236
- | Field | Meaning |
237
- | ---------------------- | ---------------------------------------------------------------------------------------------------------- |
238
- | `category` | Resolve to the newest build in this category. |
239
- | `archive` | Or pin one infohash. Exactly one of the two. |
240
- | `required` | A tile fails rather than being served without this source. Defaults true for the bottom-most, false above. |
241
- | `encoding` | `mapbox` or `terrarium`. Elevation space only. |
242
- | `baseVal` / `interval` | Mapbox decode offset and step. Default `-10000` and `0.1`. |
243
- | `maskValues` | Decoded heights meaning "no data here". Elevation space only. |
244
- | `maskRange` | `[low, high]` in metres, or a list of them. Everything inside is nodata. Elevation space only. |
245
- | `heightAdjustment` | Metres, added **after** masking. Elevation space only. |
246
- | `feather` | Pixels to fade in over at the edge of the source's shape. Needs a `cutline` or `bounds`. Max 64. |
247
- | `opacity` | `0`–`1`, scales the source alpha. RGBA space only. |
248
- | `blend` | `normal`, `multiply`, `screen`, `overlay`, `darken`, `lighten`. RGBA space only. |
238
+ | Field | Meaning |
239
+ | ---------------------- | ------------------------------------------------------------------------------------------------------------ |
240
+ | `category` | Resolve to the newest build in this category. |
241
+ | `archive` | Or pin one infohash. Exactly one of the two. |
242
+ | `required` | A tile fails rather than being served without this source. Defaults true for the bottom-most, false above. |
243
+ | `encoding` | `mapbox` or `terrarium`. Elevation space only. |
244
+ | `baseVal` / `interval` | Mapbox decode offset and step. Default `-10000` and `0.1`. |
245
+ | `maskValues` | Decoded heights meaning "no data here". Elevation space only. |
246
+ | `maskRange` | `[low, high]` in metres, or a list of them. Everything inside is nodata. Elevation space only. |
247
+ | `heightAdjustment` | Metres, added **after** masking. Elevation space only. |
248
+ | `feather` | Pixels to fade in over wherever the source stops: a `cutline`, `bounds`, or the holes a mask leaves. Max 64. |
249
+ | `featherMetres` | The same fade written as metres of ground, worked out per tile. Usually the one to reach for. |
250
+ | `opacity` | `0`–`1`, scales the source alpha. RGBA space only. |
251
+ | `blend` | `normal`, `multiply`, `screen`, `overlay`, `darken`, `lighten`. RGBA space only. |
249
252
 
250
253
  `attribution` is not optional in practice. A stack is a derived work of every
251
254
  source in it, and the thing that reliably gets lost when tiles are combined is
@@ -1085,8 +1088,8 @@ standing clear of their neighbours, which is the shape this makes.
1085
1088
 
1086
1089
  ## Feathering a seam
1087
1090
 
1088
- _Built for a cutline and for `bounds`. A mask edge is not feathered yet, and
1089
- the last section says what that still needs._
1091
+ _Built for a cutline, for `bounds`, and for the holes a mask leaves. A source
1092
+ that vanishes at a tile edge is still open, and the last section says why._
1090
1093
 
1091
1094
  Smoothing hides the terracing **inside** an upscaled area. It does nothing about
1092
1095
  the artefact the original discussion actually named — "artefacts at tiles and
@@ -1236,18 +1239,40 @@ lookup rather than a decode — and it is not done.
1236
1239
 
1237
1240
  `feather` on a source, in pixels, `0` and absent meaning the edge is a switch as
1238
1241
  before. It fades that source in over that many pixels measured inward from
1239
- wherever it stops — the holes `maskValues` and `maskColors` leave, and the edge
1240
- of a `cutline` or `bounds`:
1242
+ wherever it stops — the holes `maskValues`, `maskRange` and `maskColors` leave,
1243
+ and the edge of a `cutline` or `bounds`:
1241
1244
 
1242
1245
  ```json
1243
1246
  { "archive": "swissalti", "maskValues": [0], "feather": 16 }
1244
1247
  ```
1245
1248
 
1246
- The step left at the seam is the height difference divided by the feather, which
1247
- is what makes the number predictable: two sources 40 m apart at the border, faded
1248
- over 16 pixels, step 2.5 m a pixel instead of 40 m at once. Sixty-four is the
1249
- most it takes — past that the ramp does not reach full weight anywhere inside a
1250
- 256px tile, and the source is being turned down rather than blended in.
1249
+ The step left at the seam is the height difference divided by the feather: two
1250
+ sources 40 m apart at the border, faded over 16 pixels, step 2.5 m a pixel
1251
+ instead of 40 m at once. Sixty-four is the most it takes — past that the ramp
1252
+ does not reach full weight anywhere inside a 256px tile, and the source is being
1253
+ turned down rather than blended in.
1254
+
1255
+ **A smaller step is not the same as an invisible one**, and that is the thing to
1256
+ know before picking a number. A hillshade does not read height, it reads slope:
1257
+ the drop divided by the ground underneath it. What decides whether a seam
1258
+ disappears is therefore metres per metre rather than metres per pixel — and a
1259
+ pixel is a different number of metres at every zoom. The same `feather: 8`, over
1260
+ the same 7 m disagreement, at 55°N:
1261
+
1262
+ | zoom | m/pixel | gradient |
1263
+ | ---- | ------- | -------- |
1264
+ | z12 | 11.0 | 0.08 |
1265
+ | z13 | 5.5 | 0.16 |
1266
+ | z14 | 2.7 | 0.32 |
1267
+ | z15 | 1.4 | 0.64 |
1268
+ | z16 | 0.7 | 1.28 |
1269
+
1270
+ Natural terrain is rarely over about 0.5 and a hillshade saturates around there,
1271
+ so a fade that vanishes at z12 is a bright band by z15 — wider than the cliff it
1272
+ replaced and no less visible. Widening it does help at any one zoom, since the
1273
+ gradient is the drop over the whole fade; the difficulty is that the number
1274
+ which works at z16 is eight times the one that works at z13, and a recipe has
1275
+ one number.
1251
1276
 
1252
1277
  Three things fall out of the implementation and are worth stating.
1253
1278
 
@@ -1268,22 +1293,60 @@ envelope — linear in the tile's width, and chosen over a chamfer approximation
1268
1293
  because a chamfer's error is largest on diagonals and a national boundary is
1269
1294
  mostly diagonals.
1270
1295
 
1271
- ### What it does not do yet
1296
+ ### A fade in metres
1297
+
1298
+ `featherMetres` says the same thing as a distance on the ground, and the merge
1299
+ works out the pixels for each tile it builds:
1272
1300
 
1273
- **A mask edge.** `maskValues` and `maskColors` are where most seams actually come
1274
- from, and they are not feathered. The reason is the margin: a cutline is known in
1275
- full, so the ramp beside a tile can be computed by rasterising a few extra rows,
1276
- while a mask edge lives in the source's own pixels and the ramp needs the
1277
- neighbouring tiles to be read and decoded up to eight of them, for that source,
1278
- on that tile.
1279
-
1280
- That is the piece with a real cost attached, and the one to measure before
1281
- building. Two things make it more tractable than it looks: in a bake the tiles
1282
- are walked in Hilbert order, which is chosen for locality, so a cache of decoded
1283
- source tiles would be hit far more often than missed; and the ramp is smooth by
1284
- construction, so it can be computed from the source's _parent_ at half resolution
1285
- and upscaled, which turns eight reads at this zoom into a handful at the one
1286
- above, shared between four children.
1301
+ ```json
1302
+ { "category": "planet-bathymetry", "maskRange": [-1, 0], "featherMetres": 50 }
1303
+ ```
1304
+
1305
+ That is the source **on top**the one whose mask leaves the hole. A weight
1306
+ says how much of what is underneath shows through, so it goes on the layer with
1307
+ the edge and not on the one filling the hole: a fade on the bathymetry
1308
+ underneath is read by nothing, because `paintHeights` takes the first layer that
1309
+ contributes whole and only consults a weight from the second onward. See "A
1310
+ feathered layer over a hole stands alone" above, which is the same rule seen
1311
+ from the other side.
1312
+
1313
+ Web Mercator makes that arithmetic rather than a lookup: one pixel covers
1314
+ `40075016.686 × cos(latitude) / 2^zoom / tileSize` metres, so the conversion
1315
+ needs a multiply and the coordinates of the tile being built. `featherMeters` is
1316
+ read as well, because a key that quietly does nothing when spelled the other way
1317
+ is the failure this feature is most prone to.
1318
+
1319
+ Fifty metres, over the same 7 m disagreement, at 55°N on 512px tiles:
1320
+
1321
+ | zoom | m/pixel | pixels | ground | gradient |
1322
+ | ---- | ------- | ------ | ------ | -------- |
1323
+ | z12 | 11.0 | 5 | 55 m | 0.13 |
1324
+ | z13 | 5.5 | 9 | 49 m | 0.14 |
1325
+ | z14 | 2.7 | 18 | 49 m | 0.14 |
1326
+ | z15 | 1.4 | 36 | 49 m | 0.14 |
1327
+ | z16 | 0.7 | 73 | 50 m | 0.14 |
1328
+
1329
+ One number, the same slope at every zoom, and 0.14 is ordinary hillside rather
1330
+ than an edge.
1331
+
1332
+ It stops being exact at both ends, benignly. Below the zoom where the fade is a
1333
+ pixel wide it rounds to nothing — 50 m at z8 is a sixth of a pixel, and the whole
1334
+ coastline is inside one pixel there anyway. Above a quarter of the tile it is
1335
+ capped, which is 128 pixels on a 512px grid and is where the old 64 came from:
1336
+ past that the ramp reaches full weight nowhere inside the tile. At 55°N the cap
1337
+ first binds at z17.
1338
+
1339
+ **Picking the number.** It is the disagreement being hidden that sets it, not the
1340
+ coastline being drawn across. `tools/coast-step.mjs` measures the first against a
1341
+ real tile: it reports the step where one source hands over to the other, and
1342
+ whether those steps cluster — in which case they are a datum offset and
1343
+ `heightAdjustment` is the honest fix — or scatter, in which case nothing corrects
1344
+ them and a fade is for hiding what cannot be corrected. Divide the step by the
1345
+ gradient wanted: 7 m at 0.15 is about 50 m, and the same step at 0.05 would need
1346
+ 140 m, which is wide enough to start flattening ground either side of the coast
1347
+ that was never wrong.
1348
+
1349
+ ### What it does not do yet
1287
1350
 
1288
1351
  **A source that vanishes at a tile edge.** A sparse archive simply has no tile
1289
1352
  outside its extent, so the seam lands exactly on a tile boundary — the most
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "pmtiles-swarm",
3
- "version": "0.76.0",
3
+ "version": "0.78.0",
4
4
  "description": "BitTorrent distribution for PMTiles map archives: create torrents, watch folders, publish and subscribe to RSS feeds, and seed through qBittorrent or an embedded client",
5
5
  "type": "module",
6
6
  "main": "src/index.js",
package/src/api.js CHANGED
@@ -37,6 +37,7 @@ import { SUMMARY_VERSION } from './pmtiles-probe.js';
37
37
  import { TileReadError } from './tiles.js';
38
38
  import { loadCodec } from './codec.js';
39
39
  import { answerStackTile, outputFormat, outputSize } from './stack-tile.js';
40
+ import { clearStorage, storageReport } from './storage.js';
40
41
  import {
41
42
  isPinned,
42
43
  needsCodec,
@@ -790,6 +791,38 @@ export function createApp({
790
791
  }),
791
792
  );
792
793
 
794
+ // What this node is holding that it could let go of. Separate from the
795
+ // settings it sits beside in the console: a setting says what to do next and
796
+ // this says what to do about what was already done.
797
+ app.get(
798
+ '/api/storage',
799
+ route(async (_req, res) => {
800
+ res.setHeader('cache-control', 'no-store');
801
+ res.json(
802
+ await storageReport({ config, stackCache, stats, traffic, bakes }),
803
+ );
804
+ }),
805
+ );
806
+
807
+ app.delete(
808
+ '/api/storage/:what',
809
+ route(async (req, res) => {
810
+ const gone = await clearStorage(req.params.what, {
811
+ config,
812
+ stackCache,
813
+ stats,
814
+ traffic,
815
+ bakes,
816
+ });
817
+ if (!gone) {
818
+ return res
819
+ .status(404)
820
+ .json({ error: 'nothing here is called that, or it is turned off' });
821
+ }
822
+ return res.json(gone);
823
+ }),
824
+ );
825
+
793
826
  // Settings. Everything read per request takes effect immediately because the
794
827
  // running config object is the one being mutated; everything bound at startup
795
828
  // is written to the file and reported back as needing a restart, rather than
package/src/bake-jobs.js CHANGED
@@ -205,6 +205,34 @@ export class BakeManager {
205
205
  return true;
206
206
  }
207
207
 
208
+ /**
209
+ * Every export with work still on disk, and where it is.
210
+ *
211
+ * Read from the filesystem rather than from what this process remembers: a
212
+ * working directory outlives the run that made it, and one belonging to a
213
+ * stack somebody has since deleted is exactly the kind nobody thinks to look
214
+ * for. Whether it is running is what decides if it may be discarded, and
215
+ * that is memory's to say.
216
+ * @returns {Promise<object[]>} - `{stackId, directory, running, stopped}`.
217
+ */
218
+ async heldWork() {
219
+ const found = [];
220
+ for (const root of this.#workRoots()) {
221
+ const base = path.join(root, WORK_DIR);
222
+ const names = await fs.readdir(base).catch(() => []);
223
+ for (const stackId of names) {
224
+ const job = this.#jobs.get(stackId);
225
+ found.push({
226
+ stackId,
227
+ directory: path.join(base, stackId),
228
+ running: Boolean(job && !job.finishedAt),
229
+ stopped: this.#held.has(stackId),
230
+ });
231
+ }
232
+ }
233
+ return found;
234
+ }
235
+
208
236
  /**
209
237
  * Throws away what a stopped export had done.
210
238
  *
package/src/cutline.js CHANGED
@@ -45,6 +45,31 @@ export function worldY(lat) {
45
45
  );
46
46
  }
47
47
 
48
+ /** The world across the equator, in metres: what a zoom 0 tile spans. */
49
+ const EQUATOR = 40075016.686;
50
+
51
+ /**
52
+ * How much ground one pixel of a tile covers, at that tile's latitude.
53
+ *
54
+ * Web Mercator holds a pixel to a fixed fraction of the world, so the ground
55
+ * under it shrinks toward the poles and halves at every zoom. That is why a
56
+ * fade written in pixels is a different width of ground everywhere it is used,
57
+ * and this is the conversion that lets one written in metres mean the same
58
+ * thing. Taken at the middle of the tile: the scale changes across it, but a
59
+ * tile is a small piece of the world at any zoom where a fade is more than a
60
+ * pixel wide.
61
+ * @param {number} z - Zoom.
62
+ * @param {number} y - Row.
63
+ * @param {number} size - Pixels per side of the tile.
64
+ * @returns {number} - Metres per pixel.
65
+ */
66
+ export function metresPerPixel(z, y, size) {
67
+ const tiles = 2 ** z;
68
+ const middle = (y + 0.5) / tiles;
69
+ const latitude = Math.atan(Math.sinh(Math.PI * (1 - 2 * middle)));
70
+ return (EQUATOR * Math.cos(latitude)) / (tiles * size);
71
+ }
72
+
48
73
  /**
49
74
  * Reads the rings out of GeoJSON, whatever shape it arrived in.
50
75
  *
package/src/cutlines.js CHANGED
@@ -1,6 +1,6 @@
1
1
  import fs from 'node:fs/promises';
2
2
  import path from 'node:path';
3
- import { fromBounds, fromGeoJSON } from './cutline.js';
3
+ import { fromBounds, fromGeoJSON, metresPerPixel } from './cutline.js';
4
4
 
5
5
  /**
6
6
  * The cutlines a node has, by name.
@@ -101,16 +101,63 @@ export class CutlineStore {
101
101
  */
102
102
  export const MAX_FEATHER = 64;
103
103
 
104
+ /** Past this a fade in metres is a typo rather than a distance. */
105
+ export const MAX_FEATHER_METRES = 100000;
106
+
107
+ /**
108
+ * The widest fade a grid of this size will take.
109
+ *
110
+ * A quarter of the tile, which is where 64 came from and what it still means on
111
+ * a 512px one: past that the ramp reaches full weight nowhere inside the tile,
112
+ * and the source is being turned down rather than blended in. It binds on a
113
+ * fade in metres, which asks for more pixels at every zoom.
114
+ * @param {number} size - Pixels per side of the grid the ramp runs on.
115
+ * @returns {number} - The most pixels a fade may be.
116
+ */
117
+ function capFor(size) {
118
+ return Math.max(MAX_FEATHER, Math.round((size ?? 0) / 4));
119
+ }
120
+
104
121
  /**
105
- * How far a source fades in at the edge of its shape.
122
+ * How far a source fades in, as metres of ground, if it is written that way.
123
+ *
124
+ * Both spellings are read. A recipe key that does nothing when it is spelled
125
+ * the other way is the failure this whole feature is most prone to, and the
126
+ * prose here says metres while half the world's config files say meters.
127
+ * @param {object} recipe - One source out of a recipe.
128
+ * @returns {number} - Metres, 0 when the fade is not written in them.
129
+ */
130
+ export function featherMetresFor(recipe) {
131
+ const asked = Number(recipe?.featherMetres ?? recipe?.featherMeters);
132
+ return Number.isFinite(asked) && asked > 0 ? asked : 0;
133
+ }
134
+
135
+ /**
136
+ * How far a source fades in at the edge of its shape, in this tile's pixels.
106
137
  *
107
138
  * Zero is the old behaviour and the default: the shape is a switch, and a pixel
108
139
  * is either the source or what is underneath it. Anything more makes it a ramp.
109
- * See docs/tile-stacks.md -- "Feathering a seam".
140
+ *
141
+ * A fade in metres is converted here, which is why this wants the tile. What
142
+ * the fade has to hide is two sources disagreeing about the height of the same
143
+ * ground, which is a fixed number of metres -- so the same number of pixels is
144
+ * too wide at one zoom and too steep at the next. See docs/tile-stacks.md --
145
+ * "Feathering a seam".
110
146
  * @param {object} recipe - One source out of a recipe.
147
+ * @param {object} [tile] - `{z, y, size}` of the tile being built.
111
148
  * @returns {number} - Pixels, 0 when it does not fade.
112
149
  */
113
- export function featherFor(recipe) {
150
+ export function featherFor(recipe, tile) {
151
+ const size = tile?.size > 0 ? tile.size : 256;
152
+ const metres = featherMetresFor(recipe);
153
+ if (metres > 0) {
154
+ // With no tile there is no scale to convert against, and a fade that
155
+ // guessed one would be a different width from the tiles beside it.
156
+ if (!Number.isFinite(tile?.z)) return 0;
157
+ const perPixel = metresPerPixel(tile.z, tile.y ?? 0, size);
158
+ return Math.min(Math.round(metres / perPixel), capFor(size));
159
+ }
160
+
114
161
  const asked = Number(recipe?.feather);
115
162
  if (!Number.isFinite(asked) || asked <= 0) return 0;
116
163
  return Math.min(Math.round(asked), MAX_FEATHER);
@@ -207,6 +207,25 @@ export class StackCache {
207
207
  }
208
208
  }
209
209
 
210
+ /**
211
+ * Throws away every tile it is holding.
212
+ *
213
+ * The hit and miss counters are left alone: they say what the cache has been
214
+ * doing since the process started, which emptying it does not unmake. The
215
+ * files go one at a time and forgivingly -- one that vanished underneath is
216
+ * one fewer to remove.
217
+ * @returns {Promise<number>} - How many tiles went.
218
+ */
219
+ async clear() {
220
+ const keys = [...this.#entries.keys()];
221
+ this.#entries.clear();
222
+ this.#bytes = 0;
223
+ for (const key of keys) {
224
+ await fs.rm(this.#pathFor(key), { force: true }).catch(() => {});
225
+ }
226
+ return keys.length;
227
+ }
228
+
210
229
  /**
211
230
  * What the cache is holding, for the console and for tests.
212
231
  * @returns {object} - entries, bytes, maxBytes, hits, misses.
package/src/stack-tile.js CHANGED
@@ -151,7 +151,7 @@ export function clipsFor(resolved, cutlines, z, x, y, size = 256) {
151
151
  // The feather reaches inward from the edge, so a tile wholly inside but
152
152
  // near it still has a ramp across part of itself and cannot take the cheap
153
153
  // answer.
154
- const feather = featherFor(recipe);
154
+ const feather = featherFor(recipe, { z, y, size });
155
155
  return {
156
156
  shape,
157
157
  feather,
@@ -524,10 +524,11 @@ async function readMaskEdges({
524
524
  contributions.map(async (contribution) => {
525
525
  if (!contribution) return;
526
526
  const recipe = contribution.source ?? {};
527
- const feather = featherFor(recipe);
528
- if (!feather || !masksAnything(recipe)) return;
529
-
527
+ // On the source's own grid rather than the output's, because that is
528
+ // what the ramp will be measured on.
530
529
  const grid = contribution.raster?.width ?? size;
530
+ const feather = featherFor(recipe, { z, y, size: grid });
531
+ if (!feather || !masksAnything(recipe)) return;
531
532
  const layout = parentsFor({ z, x, y }, grid, feather);
532
533
  if (!layout) return;
533
534
 
@@ -612,10 +613,14 @@ async function merge({
612
613
  // decides what the pixels inside it weigh. The shape is known in full, so
613
614
  // this costs the extra rows and nothing else -- unlike a mask read out of
614
615
  // a source's own pixels, which would need its neighbours.
615
- const margin = clip.feather ?? 0;
616
+ //
617
+ // Asked again rather than taken from the clip: `clipsFor` may have been
618
+ // given a grid the merge did not end up using, and a fade in metres is a
619
+ // different number of pixels on each of them.
620
+ const margin = featherFor(contribution.source, { z, y, size: grid });
616
621
  const mask = rasterizeTile(clip.shape, z, x, y, grid, margin);
617
622
  contribution.coverage = cropMask(
618
- featherMask(mask, grid + margin * 2, clip.feather ?? 0),
623
+ featherMask(mask, grid + margin * 2, margin),
619
624
  grid,
620
625
  margin,
621
626
  );
@@ -729,7 +734,8 @@ export async function answerStackTile(options) {
729
734
  // 256 where the request did not say, which is the cautious guess: the
730
735
  // margin is in pixels of the output grid, so assuming a smaller grid makes
731
736
  // it wider on the ground, and too wide only costs a rasterise that was not
732
- // needed while too narrow misses a ramp.
737
+ // needed while too narrow misses a ramp. A fade in metres does not mind
738
+ // either way -- fewer pixels each covering more ground is the same ground.
733
739
  const clips = clipsFor(resolved, cutlines, z, x, y, size);
734
740
  const format = options.format ?? outputFormat(resolved);
735
741
  const rgba = resolved.stack.space === 'rgba';