pmtiles-swarm 0.90.0 → 0.92.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -7,6 +7,83 @@
7
7
  ### 🐞 Bug fixes
8
8
  - _...Add new stuff here..._
9
9
 
10
+ ## 0.92.0
11
+ ### 🐞 Bug fixes
12
+ - **A stack's TileJSON published the addresses its sources are read from.** A URL source is named by
13
+ its URL and an S3 source by its bucket and key, and the document naming them is served to anybody
14
+ who can load the map — while the archives at the other end are read with credentials nobody else
15
+ has. The sources are gone from it entirely, which was the right answer anyway: they are the
16
+ ingredients of one endpoint, not a list for a client to join, and listing them invited fetching
17
+ those instead of the tiles. What is left is a count and a `revision` fingerprint covering the
18
+ recipe and what every source resolved to — the one thing the list was good for, in twenty bytes
19
+ rather than tens of kilobytes. The document for a 459-source stack is now 336 bytes.
20
+
21
+ - **Signing in threw away the view you asked for.** A link to `#stacks` on a guarded node asked for a
22
+ password and then showed the archives, with the address still reading `#stacks` — which is what
23
+ made it look broken rather than like a redirect. Both ways into the console now land where the
24
+ address says.
25
+
26
+ - **An archive the engine has no record of was drawn as 0%.** That is a different fact from "none of
27
+ it is here", and it is the more alarming one to get wrong: a library the engine failed to take back
28
+ after a restart read as a library that had lost its data. The row now says **not loaded**, and says
29
+ where the reason is logged and that nothing on disk has been touched.
30
+
31
+ ## 0.91.0
32
+ ### ✨ Features and improvements
33
+ - **A source at a URL can be added by hand, not only imported.** **Add source → an address you
34
+ type…** in the stack editor, writing the same source the importer does. The card asks for the
35
+ address and for the zoom range the archive holds — a catalog archive states its range in its own
36
+ header and this has no header anybody has read, so leaving it empty means every tile asks it.
37
+ Anything published as a plain HTTPS download works, an S3 object or presigned URL included, and so
38
+ does `s3://bucket/key.pmtiles` for a private bucket — see below.
39
+
40
+ - **A stack's source list folds up past five sources.** An imported list is several hundred rows,
41
+ which buried every other stack on the page under one of them. The fold says how many there are and
42
+ names the base; a stack of a base and a layer or two stays open, since folding that hides nothing
43
+ worth a click.
44
+
45
+
46
+ - **A stack source may be an object in a private S3 bucket.**
47
+ `s3://bucket/terrain.pmtiles`, read with a signed request per byte range. Only for a bucket that is
48
+ not public — a public object or a presigned URL is an ordinary HTTPS address and always worked.
49
+ `endpoint` is what makes it S3-compatible rather than S3: MinIO, Ceph, Garage, R2, Wasabi and B2
50
+ all answer the same protocol at their own address, and path-style addressing is the default because
51
+ it is what they speak. Credentials go under **Settings → Feeds → S3 buckets**, per bucket or once
52
+ for an account; with none configured the standard `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`,
53
+ `AWS_REGION` and `AWS_S3_ENDPOINT` variables are read, so a machine already set up to reach a
54
+ bucket needs nothing typed in.
55
+
56
+ Signed here rather than by an SDK: SigV4 for a GET is a hash, four HMACs and a string, all of which
57
+ `node:crypto` has. An AWS client would be a large dependency for one signature, and an optional one
58
+ would make reading a bucket work on some installs and not on others for no reason the operator
59
+ could see — so it is neither optional nor a dependency. The signer is checked against AWS's own
60
+ published test vectors rather than against itself.
61
+
62
+
63
+ ### 🐞 Bug fixes
64
+ - **A stack of remote sources could be served but not exported.** The bake refused any stack whose
65
+ sources resolved to no archive on this node, which is every stack read from URLs — refusing exactly
66
+ the export worth having, since baking is how terrain that lives somewhere else becomes something
67
+ this node holds and can seed. It also had no way to walk a remote archive's directories, which is
68
+ how a bake finds which tiles exist at all.
69
+
70
+ - **An archive whose deepest tiles repeat said it stopped a zoom short.** Identical tiles collapse
71
+ into one directory entry covering a range of ids, so the last entry's own id can sit at a shallower
72
+ zoom than the tiles it addresses — and the header was read off that id. A client asks for nothing
73
+ past a header's `maxZoom`, so the deepest zoom of such an archive was unreachable. Found baking a
74
+ stack over flat terrain, where every tile of the last zoom was the same tile.
75
+
76
+ - **A stack used as a source of another stack was reported as unresolved.** It resolves to a recipe
77
+ rather than to bytes, and the console's reader looked only for a catalog entry or an address — so a
78
+ working nested stack showed "does not resolve", with no zoom range and no box, and put a "sources
79
+ missing" badge on the stack above it. It now says what it stands for and how deep it reaches,
80
+ worked out from the sources that stack would serve.
81
+
82
+ - **A stack's advertised box ignored any source that stated none.** The union was taken over
83
+ whichever sources had a box, so a stack of a global base plus regional patches — an imported list,
84
+ exactly — advertised the patches as its extent and left the base out of it. A source with no box
85
+ is not one covering nothing, so it is now every box or none, and none means the world.
86
+
10
87
  ## 0.90.0
11
88
  ### ✨ Features and improvements
12
89
  - **A stack source may name a URL instead of a category, archive or stack.** `{ "url": "https://…" }`,
@@ -20,6 +20,7 @@ Some settings only take effect on restart. Those are marked **restart**.
20
20
  - [Scheduled sources](#scheduled-sources)
21
21
  - [Subscriptions](#subscriptions)
22
22
  - [Serving tiles](#serving-tiles)
23
+ - [S3 buckets](#s3-buckets)
23
24
  - [Mutable publishing](#mutable-publishing)
24
25
  - [Authentication](#authentication)
25
26
  - [Statistics](#statistics)
@@ -776,6 +777,96 @@ connection, a piece already in flight. Ten attempts later it is one piece at the
776
777
  far end of an archive nobody has finished, and asking every fifteen seconds
777
778
  achieves nothing but log lines.
778
779
 
780
+ ## S3 buckets
781
+
782
+ A stack source may be an address rather than an archive this node holds, and
783
+ that address may name an object in a bucket:
784
+
785
+ ```json
786
+ {
787
+ "sources": [{ "url": "s3://terrain/planet.pmtiles", "encoding": "terrarium" }]
788
+ }
789
+ ```
790
+
791
+ Only for a bucket that is **not public**. A public object, or a presigned URL,
792
+ is an ordinary HTTPS address: write it as one and none of this applies.
793
+
794
+ ```json
795
+ {
796
+ "s3": [
797
+ {
798
+ "bucket": "terrain",
799
+ "endpoint": "https://minio.lan:9000",
800
+ "region": "us-east-1",
801
+ "accessKeyId": "…",
802
+ "secretAccessKey": "…"
803
+ }
804
+ ]
805
+ }
806
+ ```
807
+
808
+ | field | default | |
809
+ | ----------------- | ------------------ | ---------------------------------------------------- |
810
+ | `bucket` | any | which bucket this row is for; unset matches all |
811
+ | `endpoint` | AWS for the region | what makes it S3-compatible rather than S3 |
812
+ | `region` | `us-east-1` | signed into every request, whether or not it is used |
813
+ | `accessKeyId` | — | |
814
+ | `secretAccessKey` | — | |
815
+ | `sessionToken` | unset | for temporary credentials |
816
+ | `pathStyle` | path, unless AWS | `false` puts the bucket in the hostname |
817
+
818
+ A source is read through the row naming its bucket, or through whichever row
819
+ names none: one set of credentials for a whole account is the ordinary case,
820
+ and writing the same keys once per bucket is busywork with a copy-paste
821
+ mistake in it.
822
+
823
+ ### The endpoint is the whole point
824
+
825
+ MinIO, Ceph, Garage, Cloudflare R2, Wasabi and Backblaze B2 all answer the
826
+ same protocol at their own address, so naming that address is the difference
827
+ between "S3" and "S3-compatible". Path-style addressing is what they speak and
828
+ is the default here; AWS withdrew it for buckets made after September 2020, so
829
+ an `amazonaws.com` endpoint puts the bucket in the hostname instead. Set
830
+ `pathStyle` only for a server that insists on the other one.
831
+
832
+ ### Credentials from the environment
833
+
834
+ With no rows configured at all, the standard variables are read — the same
835
+ ones the AWS CLI, rclone, go-pmtiles and tileserver-gl use, so a machine
836
+ already set up to reach a bucket needs nothing written here:
837
+
838
+ | variable | |
839
+ | -------------------------------------------- | ------------------------------- |
840
+ | `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY` | required for any of it to apply |
841
+ | `AWS_SESSION_TOKEN` | temporary credentials |
842
+ | `AWS_REGION`, `AWS_DEFAULT_REGION` | region |
843
+ | `AWS_S3_ENDPOINT`, `AWS_ENDPOINT_URL_S3` | endpoint |
844
+ | `AWS_S3_FORCE_PATH_STYLE` | addressing |
845
+
846
+ ### Signed here, not by an SDK
847
+
848
+ SigV4 for a GET is a hash, four HMACs and a string, all of which `node:crypto`
849
+ already has. An AWS client would be a large dependency for one signature, and
850
+ an optional one would make reading a bucket work on some installs and not on
851
+ others for no reason the operator could see — so it is neither optional nor a
852
+ dependency. The signer is checked against AWS's own published test vectors
853
+ rather than against itself.
854
+
855
+ Every request is signed as it goes out, including the range: the header, each
856
+ directory and each tile. Nothing is presigned and nothing is cached but the
857
+ bytes, so revoking a key stops reads at the next request rather than at the
858
+ next restart. Changing the settings drops the open readers for the same
859
+ reason.
860
+
861
+ ### What a bucket source is, and is not
862
+
863
+ It is a URL source in every other respect — see
864
+ [tile-stacks.md](tile-stacks.md), "A source read straight from a URL". It
865
+ carries its own `minzoom`, `maxzoom` and `bounds`, it is never seeded or
866
+ retired, and it can be baked: a stack over private buckets exports to an
867
+ ordinary archive with an infohash, which is the path from "terrain we hold in
868
+ a bucket" to "terrain we publish".
869
+
779
870
  ## Mutable publishing
780
871
 
781
872
  Announcing the current build of each category over the DHT (BEP 46).
@@ -60,6 +60,7 @@ of its parts.
60
60
  - [Syncing a stack to another node](#syncing-a-stack-to-another-node)
61
61
  - [What the offline merge got wrong](#what-the-offline-merge-got-wrong)
62
62
  - [The stack editor](#the-stack-editor)
63
+ - [Merging vector sources](#merging-vector-sources)
63
64
  - [Staging](#staging)
64
65
  - [Open questions](#open-questions)
65
66
 
@@ -541,21 +542,42 @@ Derived from the resolved sources rather than from any one archive:
541
542
  - `attribution` — every source's, concatenated.
542
543
  - `tiles` — `/stacks/<id>/{z}/{x}/{y}.<ext>`.
543
544
 
544
- Plus a non-standard `stack` block naming what it resolved to, mirroring the
545
- `latest` block on `/latest/<category>/tiles.json`, so a consumer can tell one
546
- resolution from the next:
545
+ Plus a non-standard `stack` block, mirroring the `latest` block on
546
+ `/latest/<category>/tiles.json`, so a consumer can tell one resolution from the
547
+ next:
547
548
 
548
549
  ```jsonc
549
550
  "stack": {
550
551
  "id": "planet-terrain",
551
- "revision": 7,
552
- "sources": [
553
- { "category": "gebco", "infohash": "a074186d…", "name": "GEBCO_2026_…" },
554
- { "category": "planet-bathymetry", "infohash": "…", "name": "…" }
555
- ]
552
+ "space": "elevation",
553
+ "sources": 2,
554
+ "revision": "9f2c1a77b3e04d16"
556
555
  }
557
556
  ```
558
557
 
558
+ `revision` covers the recipe and what every source resolved to, so it moves
559
+ when a category resolves to a new build — which is the whole reason the block
560
+ exists.
561
+
562
+ ### The sources themselves are not in it
563
+
564
+ They were once, and it was wrong twice over.
565
+
566
+ A stack's sources are not a client's to join. They are the ingredients of one
567
+ endpoint, and this document exists to point at that endpoint: a map reads
568
+ `tiles`, not the archives behind it. Listing them invites somebody to fetch
569
+ those instead, which is the one thing a stack is there to stop them having to
570
+ do.
571
+
572
+ And a source may be an **address**. A URL, or a bucket and a key, published in
573
+ a document that is served to anybody who can load the map — while the archive
574
+ at the other end of it is read with credentials nobody else has. The tiles are
575
+ public on purpose; where they come from is not.
576
+
577
+ What is left says the same thing the list said, in twenty bytes rather than
578
+ tens of kilobytes: how many sources there are, and a fingerprint that moves
579
+ when any of them does.
580
+
559
581
  ## When a source will not answer
560
582
 
561
583
  A cache-mode source with no reachable peers, a category that resolves to an
@@ -1681,6 +1703,22 @@ archive — Mapterhorn's own base stops at z12 — is upscaled for a deeper
1681
1703
  request exactly as GEBCO is, through the same code, because reading one is now
1682
1704
  a question of which store method answers and nothing else.
1683
1705
 
1706
+ ### Adding one by hand
1707
+
1708
+ **Add source → an address you type…** in the stack editor, which is the same
1709
+ source the importer writes and edits the same way. The card asks for two
1710
+ things nothing else can know: the address, and the zoom range the archive
1711
+ holds. A catalog archive states its range in its own header and a URL source
1712
+ has no header this node has read, so an unstated range means every tile asks
1713
+ it — correct, and the thing worth avoiding once there are several.
1714
+
1715
+ Anything published as a plain HTTPS download works, which includes an S3
1716
+ bucket that serves range requests: an object URL, or a presigned one, is an
1717
+ address like any other, and this asks for byte ranges the way any PMTiles
1718
+ reader does. What it cannot do is a private bucket named `s3://…`, which is
1719
+ not a URL a browser or a fetch can follow — that needs a signed request and
1720
+ somewhere to keep the credentials, and neither exists here yet.
1721
+
1684
1722
  ### Cheap to have hundreds of
1685
1723
 
1686
1724
  The one thing that makes hundreds of these practical rather than merely
@@ -1830,6 +1868,14 @@ that is a map changing at 4am with nothing to say why. A hand-written source
1830
1868
  carries no `importedFrom`, is never replaced, and is the place for anything the
1831
1869
  index could not have told us.
1832
1870
 
1871
+ ### Seeing them in the console
1872
+
1873
+ A stack's source list collapses behind a summary once there are more than
1874
+ five, with the count and the base named on the fold. A stack imported from a
1875
+ provider's list is several hundred rows and drawing them flat buries every
1876
+ other stack on the page under one of them; a stack of a base and a layer or
1877
+ two stays open, since folding that hides nothing worth a click.
1878
+
1833
1879
  ## Finding a stack
1834
1880
 
1835
1881
  A stack has no infohash and appears in no feed, so nothing about it is
@@ -2175,6 +2221,135 @@ Debounced, and it should not follow the map past the stack's `maxzoom`: above
2175
2221
  that the client overzooms and there is nothing new to see, but the tiles are
2176
2222
  still requested and still composited.
2177
2223
 
2224
+ ## Merging vector sources
2225
+
2226
+ **Not built.** This is the design, written down before anything is started so
2227
+ that the hard part is on record rather than discovered halfway through it.
2228
+
2229
+ Most of the machinery is indifferent to what a tile contains. Source
2230
+ resolution, per-source `minzoom`, `maxzoom` and `bounds`, painting order,
2231
+ nested stacks, the cache, the ETag, `unionOfTileIds` and the bake all treat a
2232
+ tile as an opaque thing, and the byte-for-byte passthrough already serves a
2233
+ vector archive today. What a vector space adds is a merge, and the merge is
2234
+ where the analogy with the two pixel spaces breaks.
2235
+
2236
+ ### What has no vector analogue
2237
+
2238
+ `opacity`, `blend`, `feather`, `featherMetres`, `gaussianBlurSigma`,
2239
+ `output.encoding`, `output.tileSize`, `maskValues`, `maskRange` and
2240
+ `maskColors`. Every one of them is an operation on a pixel, and there are no
2241
+ pixels. A recipe naming any of them under a vector space should be **refused by
2242
+ validation**, not accepted and ignored: a stack that says `feather: 50` and
2243
+ draws a hard edge is a worse answer than one that will not save.
2244
+
2245
+ The idea of a mask survives in three other forms — clip to a **shape**, keep or
2246
+ drop whole **layers** by name, and filter **features** by a property. Those
2247
+ three are the vector mask, and none of them is a mask in the sense the
2248
+ elevation space means.
2249
+
2250
+ ### Overlap is the whole problem
2251
+
2252
+ The pixel spaces work because the unit is a pixel, every source has one
2253
+ everywhere, and "the topmost covered source wins" is a complete rule. Vector's
2254
+ unit is a feature. Two sources that both carry a `water` layer over the same
2255
+ ground hold **two polygons of the same lake**, and emitting both overwrites
2256
+ neither. The tile carries both, the style draws both, and it shows as doubled
2257
+ outlines, darker semi-transparent fills, duplicated labels, and z-fighting
2258
+ between them.
2259
+
2260
+ So the first thing a vector space has to define is what winning means. Three
2261
+ answers, and the third is not a default:
2262
+
2263
+ **Per-layer replacement.** For each layer _name_, the topmost source carrying it
2264
+ supplies it whole, and the same-named layer of every source below is dropped for
2265
+ that tile. No geometry work at all: read the layer list, choose, re-emit. It is
2266
+ the right answer for a base plus an overlay of layers the base does not have —
2267
+ an OSM base under your own `buildings`, or under contours generated from the
2268
+ terrain these stacks already merge — and it is correct for a base plus a patch
2269
+ wherever the tile falls wholly inside the patch.
2270
+
2271
+ **Clip and union.** Each source is clipped to its own coverage, and the source
2272
+ below is additionally clipped _against_ the coverage of the one above it, so the
2273
+ two cannot both carry the same lake. This is the true counterpart of the pixel
2274
+ merge, and the only thing that works on a tile the boundary of a patch runs
2275
+ through. A **rectangle** clip in tile coordinates is cheap and exact —
2276
+ Sutherland-Hodgman for polygons, Liang-Barsky for lines — and a rectangle is
2277
+ what an imported file list states for every patch in it. An arbitrary
2278
+ **cutline** is a different matter and wants a real polygon-clipping library.
2279
+
2280
+ **Renaming.** The patch's `water` becomes `patch_water`, and the style decides
2281
+ what to do with it. No conflict and no geometry work, at the cost of a style
2282
+ that has to know. Worth having as a way out; not worth having as the rule.
2283
+
2284
+ The recommendation is per-layer replacement as the default, rectangle clipping
2285
+ for the base-and-patch case, and a **recipe warning** wherever two sources carry
2286
+ the same layer with no clip between them: _sources 0 and 1 both provide `water`,
2287
+ and tiles where they meet will carry both_. Silent doubling is the failure that
2288
+ gets found six months later by somebody squinting at a coastline.
2289
+
2290
+ ### A seam cannot be feathered
2291
+
2292
+ Where two sources disagree geometrically at a boundary, a road jogs and a
2293
+ coastline steps, and there is nothing to fade. [Feathering a
2294
+ seam](#feathering-a-seam) works because a pixel can be a mixture of two values.
2295
+ A feature cannot be a mixture of two features.
2296
+
2297
+ Vector merging is therefore clean only where the sources were cut to agree — the
2298
+ same condition that makes a provider's raster patches merge cleanly, minus the
2299
+ fade that hides the cases where it is not quite true. That belongs in the
2300
+ console rather than on the map.
2301
+
2302
+ ### Overzooming a parent
2303
+
2304
+ Possible, and needed: a base to z12 under a patch to z14 has to climb the base
2305
+ at z13, exactly as [resampling from a parent](#resampling-from-a-parent-tile)
2306
+ does for pixels. Scale the parent's coordinates by the zoom difference, subtract
2307
+ the quadrant's offset, clip to the extent plus the buffer. Exact for geometry,
2308
+ and it invents nothing: the generalisation and the label placement were chosen
2309
+ for the parent's zoom, so an overzoomed tile is thinner than a real one at that
2310
+ zoom and looks it.
2311
+
2312
+ ### Extent, buffer and schema
2313
+
2314
+ Sources at different **extents** — 4096 and 512 are both common — have to be
2315
+ normalised to one before anything is combined. A power-of-two ratio is exact;
2316
+ anything else quantises coordinates.
2317
+
2318
+ **Buffers** have to be re-clipped after any coordinate change, or features leak
2319
+ past the edge of the tile and draw twice at the join.
2320
+
2321
+ **Schemas** are the quiet one. Two sources whose `water` layers use `class` and
2322
+ `kind` merge into a layer the style half understands, and nothing in the merge
2323
+ can reconcile them. Worth comparing key sets on the first merged tile and
2324
+ reporting the disagreement as a problem on the stack, the way an unresolved
2325
+ source is reported.
2326
+
2327
+ **Feature ids** collide across sources. Renumbering settles the collision and
2328
+ breaks any client-side feature state keyed on them, so it is a choice to make
2329
+ out loud rather than a detail to settle inside the encoder.
2330
+
2331
+ ### The libraries
2332
+
2333
+ `@mapbox/vector-tile` and `pbf` to read, `vt-pbf` to write. Small, pure
2334
+ JavaScript, and ordinary dependencies rather than a probe: the optional
2335
+ treatment [the codec gets](#the-codec-problem) exists because a native build
2336
+ genuinely fails to install on some platforms, and none of that applies here.
2337
+
2338
+ Not written here either, unlike the PMTiles writer and the S3 signer. A protobuf
2339
+ round-trip has more edge cases than either — geometry command encoding, unknown
2340
+ value types, key and value pools shared across features — and getting one wrong
2341
+ produces a tile that parses and is subtly wrong.
2342
+
2343
+ ### Staging, when it is started
2344
+
2345
+ 1. **Layer operations only.** `space: "vector"`, with keep, drop and rename by
2346
+ layer name, over the per-source zoom and bounds skipping that already exists.
2347
+ Covers base-plus-overlay, and needs no geometry code whatsoever.
2348
+ 2. **Overzoom and rectangle clipping.** Covers base-plus-patch, which is the
2349
+ shape [importing a list of URLs](#importing-a-list-of-urls) makes easy to
2350
+ build.
2351
+ 3. **Cutline clipping and property filters.**
2352
+
2178
2353
  ## Staging
2179
2354
 
2180
2355
  1. ~~**Recipe and resolution.**~~ Done. `data/stacks.json`, load and validate,
@@ -2206,10 +2381,11 @@ handling whatsoever.
2206
2381
 
2207
2382
  ## Open questions
2208
2383
 
2209
- - **Vector tiles.** Merging MVT layers from two archives is a real want and a
2210
- completely different operation — decode protobuf, merge layer by layer,
2211
- re-encode, with feature ID collisions to settle. It should be its own feature
2212
- and should not be smuggled in under `space`.
2384
+ - **Vector tiles.** Designed rather than open now: see [merging vector
2385
+ sources](#merging-vector-sources). It is its own space and its own merge,
2386
+ and what stays open is whether the overlap rule proposed there — layer
2387
+ replacement by default, clipping for a patch — holds up against a real
2388
+ pair of archives.
2213
2389
  - **Does the recipe travel?** A stack is a small JSON document and the feed
2214
2390
  already distributes documents. Publishing recipes so a subscriber gets the
2215
2391
  stack along with its sources is attractive, and raises an obvious question
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "pmtiles-swarm",
3
- "version": "0.90.0",
3
+ "version": "0.92.0",
4
4
  "description": "BitTorrent distribution for PMTiles map archives: create torrents, watch folders, publish and subscribe to RSS feeds, and seed through qBittorrent or an embedded client",
5
5
  "type": "module",
6
6
  "main": "src/index.js",
package/src/api.js CHANGED
@@ -50,15 +50,7 @@ import {
50
50
  stackCoverage,
51
51
  stackEtag,
52
52
  } from './stacks.js';
53
-
54
- /**
55
- * How many sources a stack's TileJSON names before it starts counting.
56
- *
57
- * Generous for a hand-written recipe -- those run to a handful -- and a hard
58
- * stop for one imported from a provider's index, where the list is hundreds
59
- * long and every entry is an address no client reads.
60
- */
61
- const TILEJSON_SOURCE_LIMIT = 25;
53
+ import { bakeRevision } from './bake.js';
62
54
 
63
55
  const here = path.dirname(fileURLToPath(import.meta.url));
64
56
 
@@ -3400,45 +3392,64 @@ export function createApp({
3400
3392
  // The order the recipe lists them: lowest priority first, last
3401
3393
  // wins. The console shows this order as it stands rather than
3402
3394
  // inverting it, so the screen and the file never disagree.
3403
- sources: resolved.sources.map((source) => ({
3404
- index: source.index,
3405
- name: source.name,
3406
- pinned: source.pinned,
3407
- required: source.required,
3408
- // What kind of source this is, said outright rather than left to
3409
- // be inferred from which fields happen to be null. A URL source
3410
- // has no infohash and no catalog entry, so a reader guessing from
3411
- // those calls it an unresolved category -- which is two wrong
3412
- // things about a source that is working perfectly well.
3413
- kind: source.remote
3414
- ? 'url'
3415
- : source.nested
3416
- ? 'stack'
3417
- : source.pinned
3418
- ? 'archive'
3419
- : 'category',
3420
- // A URL source resolves by definition: there is nothing to look
3421
- // up, the address is the answer. Whether it can actually be read
3422
- // is a question for the first tile, the same as for an archive
3423
- // this node holds but cannot open.
3424
- resolved: Boolean(source.entry) || Boolean(source.remote),
3425
- url: source.remote ?? null,
3426
- infohash: source.entry?.infoHash ?? null,
3427
- archiveName: source.entry?.name ?? null,
3428
- // The recipe's own, where it states them. For a URL source that is
3429
- // the only place they are known without opening the file, and it
3430
- // is what the merge itself uses to decide whether to.
3431
- minzoom:
3432
- source.entry?.pmtiles?.minZoom ?? source.source?.minzoom ?? null,
3433
- maxzoom:
3434
- source.entry?.pmtiles?.maxZoom ?? source.source?.maxzoom ?? null,
3435
- bounds:
3436
- source.entry?.pmtiles?.bounds ??
3437
- (Array.isArray(source.source?.bounds)
3438
- ? source.source.bounds
3439
- : null),
3440
- format: source.entry?.pmtiles?.format ?? null,
3441
- })),
3395
+ sources: resolved.sources.map((source) => {
3396
+ // A nested stack answers for the ground its own sources cover,
3397
+ // and knows it without opening anything -- so the row can say
3398
+ // what it reaches rather than a dash.
3399
+ const inner = source.nested ? stackCoverage(source.nested) : null;
3400
+ return {
3401
+ index: source.index,
3402
+ name: source.name,
3403
+ pinned: source.pinned,
3404
+ required: source.required,
3405
+ // What kind of source this is, said outright rather than left to
3406
+ // be inferred from which fields happen to be null. A URL source
3407
+ // has no infohash and no catalog entry, so a reader guessing from
3408
+ // those calls it an unresolved category -- which is two wrong
3409
+ // things about a source that is working perfectly well.
3410
+ kind: source.remote
3411
+ ? 'url'
3412
+ : source.nested
3413
+ ? 'stack'
3414
+ : source.pinned
3415
+ ? 'archive'
3416
+ : 'category',
3417
+ // A URL source resolves by definition: there is nothing to look
3418
+ // up, the address is the answer. Whether it can actually be read
3419
+ // is a question for the first tile, the same as for an archive
3420
+ // this node holds but cannot open.
3421
+ resolved:
3422
+ Boolean(source.entry) ||
3423
+ Boolean(source.remote) ||
3424
+ Boolean(source.nested),
3425
+ // How many sources the nested recipe has, which is the only
3426
+ // useful thing to say in a column that names an archive file for
3427
+ // every other kind: a stack resolves to a recipe, not to bytes.
3428
+ nested: source.nested ? source.nested.sources.length : null,
3429
+ url: source.remote ?? null,
3430
+ infohash: source.entry?.infoHash ?? null,
3431
+ archiveName: source.entry?.name ?? null,
3432
+ // The recipe's own, where it states them. For a URL source that is
3433
+ // the only place they are known without opening the file, and it
3434
+ // is what the merge itself uses to decide whether to.
3435
+ minzoom:
3436
+ source.entry?.pmtiles?.minZoom ??
3437
+ source.source?.minzoom ??
3438
+ inner?.minzoom ??
3439
+ null,
3440
+ maxzoom:
3441
+ source.entry?.pmtiles?.maxZoom ??
3442
+ source.source?.maxzoom ??
3443
+ inner?.maxzoom ??
3444
+ null,
3445
+ bounds:
3446
+ source.entry?.pmtiles?.bounds ??
3447
+ (Array.isArray(source.source?.bounds)
3448
+ ? source.source.bounds
3449
+ : (inner?.bounds ?? null)),
3450
+ format: source.entry?.pmtiles?.format ?? null,
3451
+ };
3452
+ }),
3442
3453
  };
3443
3454
  });
3444
3455
  // Reported so the console can say "install sharp" beside the stacks
@@ -3781,34 +3792,27 @@ export function createApp({
3781
3792
  minzoom: coverage.minzoom,
3782
3793
  maxzoom: coverage.maxzoom,
3783
3794
  bounds: coverage.bounds,
3784
- // Names what it resolved to, so a consumer can tell one resolution
3785
- // from the next without diffing tile URLs — the same reason the
3786
- // `/latest/` document carries a `latest` block.
3795
+ // Says that it resolved, and whether the resolution has moved, so a
3796
+ // consumer can tell one from the next without diffing tile URLs --
3797
+ // the same reason the `/latest/` document carries a `latest` block.
3787
3798
  //
3788
- // Capped. A stack imported from a provider's file index has hundreds
3789
- // of sources — Mapterhorn's is 458 — and listing every one puts tens
3790
- // of kilobytes of addresses in a document every map load fetches, to
3791
- // say something no client does anything with. What the block is for is
3792
- // telling one resolution from the next, and what moves between
3793
- // resolutions is the infohashes, so those are what it keeps.
3799
+ // A count and a fingerprint rather than the sources themselves. They
3800
+ // were listed here once, and it was wrong twice over. A stack's
3801
+ // sources are not a client's to join: they are the ingredients of one
3802
+ // endpoint, and a document listing them invites somebody to fetch
3803
+ // those instead of the tiles. And a source may be an address -- a URL,
3804
+ // or a bucket and a key -- which this document has no business
3805
+ // publishing: the tiles are public where the archives behind them are
3806
+ // read with credentials nobody else has.
3807
+ //
3808
+ // The fingerprint covers the recipe and what every source resolved
3809
+ // to, so it moves when a category resolves to a new build. That was
3810
+ // the only thing the list was good for.
3794
3811
  stack: {
3795
3812
  id: resolved.stack.id,
3796
3813
  space: resolved.stack.space ?? 'elevation',
3797
- sources: resolved.sources
3798
- .slice(0, TILEJSON_SOURCE_LIMIT)
3799
- .map((source) => ({
3800
- name: source.name,
3801
- infohash: source.entry?.infoHash ?? null,
3802
- archive: source.entry?.name ?? null,
3803
- })),
3804
- // Said rather than silently truncated: a consumer comparing two of
3805
- // these has to know it is looking at part of a list.
3806
- ...(resolved.sources.length > TILEJSON_SOURCE_LIMIT
3807
- ? {
3808
- more: resolved.sources.length - TILEJSON_SOURCE_LIMIT,
3809
- total: resolved.sources.length,
3810
- }
3811
- : {}),
3814
+ sources: resolved.sources.length,
3815
+ revision: bakeRevision(resolved),
3812
3816
  },
3813
3817
  };
3814
3818
  // A stack says how its tiles are encoded, so a style pointing at it does