pmtiles-swarm 0.89.0 → 0.91.0

package/CHANGELOG.md

@@ -7,6 +7,138 @@
 ### 🐞 Bug fixes
 - _...Add new stuff here..._
 
+## 0.91.0
+### ✨ Features and improvements
+- **A source at a URL can be added by hand, not only imported.** **Add source → an address you
+  type…** in the stack editor, writing the same source the importer does. The card asks for the
+  address and for the zoom range the archive holds — a catalog archive states its range in its own
+  header and this has no header anybody has read, so leaving it empty means every tile asks it.
+  Anything published as a plain HTTPS download works, an S3 object or presigned URL included, and so
+  does `s3://bucket/key.pmtiles` for a private bucket — see below.
+
+- **A stack's source list folds up past five sources.** An imported list is several hundred rows,
+  which buried every other stack on the page under one of them. The fold says how many there are and
+  names the base; a stack of a base and a layer or two stays open, since folding that hides nothing
+  worth a click.
+
+
+- **A stack source may be an object in a private S3 bucket.**
+  `s3://bucket/terrain.pmtiles`, read with a signed request per byte range. Only for a bucket that is
+  not public — a public object or a presigned URL is an ordinary HTTPS address and always worked.
+  `endpoint` is what makes it S3-compatible rather than S3: MinIO, Ceph, Garage, R2, Wasabi and B2
+  all answer the same protocol at their own address, and path-style addressing is the default because
+  it is what they speak. Credentials go under **Settings → Feeds → S3 buckets**, per bucket or once
+  for an account; with none configured the standard `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`,
+  `AWS_REGION` and `AWS_S3_ENDPOINT` variables are read, so a machine already set up to reach a
+  bucket needs nothing typed in.
+
+  Signed here rather than by an SDK: SigV4 for a GET is a hash, four HMACs and a string, all of which
+  `node:crypto` has. An AWS client would be a large dependency for one signature, and an optional one
+  would make reading a bucket work on some installs and not on others for no reason the operator
+  could see — so it is neither optional nor a dependency. The signer is checked against AWS's own
+  published test vectors rather than against itself.
+
+
+### 🐞 Bug fixes
+- **A stack of remote sources could be served but not exported.** The bake refused any stack whose
+  sources resolved to no archive on this node, which is every stack read from URLs — refusing exactly
+  the export worth having, since baking is how terrain that lives somewhere else becomes something
+  this node holds and can seed. It also had no way to walk a remote archive's directories, which is
+  how a bake finds which tiles exist at all.
+
+- **An archive whose deepest tiles repeat said it stopped a zoom short.** Identical tiles collapse
+  into one directory entry covering a range of ids, so the last entry's own id can sit at a shallower
+  zoom than the tiles it addresses — and the header was read off that id. A client asks for nothing
+  past a header's `maxZoom`, so the deepest zoom of such an archive was unreachable. Found baking a
+  stack over flat terrain, where every tile of the last zoom was the same tile.
+
+- **A stack used as a source of another stack was reported as unresolved.** It resolves to a recipe
+  rather than to bytes, and the console's reader looked only for a catalog entry or an address — so a
+  working nested stack showed "does not resolve", with no zoom range and no box, and put a "sources
+  missing" badge on the stack above it. It now says what it stands for and how deep it reaches,
+  worked out from the sources that stack would serve.
+
+- **A stack's advertised box ignored any source that stated none.** The union was taken over
+  whichever sources had a box, so a stack of a global base plus regional patches — an imported list,
+  exactly — advertised the patches as its extent and left the base out of it. A source with no box
+  is not one covering nothing, so it is now every box or none, and none means the world.
+
+## 0.90.0
+### ✨ Features and improvements
+- **A stack source may name a URL instead of a category, archive or stack.** `{ "url": "https://…" }`,
+  read straight over HTTP with no torrent involved — for an archive published as a plain download,
+  like Mapterhorn's terrain: a global base plus hundreds of regional patches, 11.8 TiB in total and
+  never meant to be downloaded whole. `FetchSource`, from the `pmtiles` package this project already
+  depends on, asks for byte ranges the way a swarm-backed source does; a tile costs the header once,
+  the directory once, and the tile itself.
+
+  Parent climbing works the same way it does for a category source — a shallow archive upscales for a
+  deeper request through the same code, since reading one is now only a question of which store method
+  answers. New per-source `minzoom` and `maxzoom` fields, checked before anything is opened, are what
+  make hundreds of these practical: a tile request outside a source's stated box or zoom range skips it
+  without a request leaving this node, which for a stack built from a provider's whole file list is
+  most of them, on every tile. A URL source always decodes rather than passing through raw, having no
+  infohash to answer that question with, and is not seeded, retired or rebuilt — none of the mechanisms
+  built for an archive this node actually holds apply to one it does not.
+
+- **A provider's list of PMTiles URLs can be imported as sources.** **Stacks → Import URL list…**,
+  or `POST /api/stacks/<id>/import`. Mapterhorn's `download_urls.json` names 458 files with the box
+  and zoom range of each; naming those by hand is not work anybody should do once, let alone again
+  when the provider adds one. An index with an `items` list is read, and so is a plain list of
+  addresses for a provider that publishes no index — which shape it is is worked out from the
+  document, since somebody pasting an address has no reason to know.
+
+  The global file becomes `sources[0]` and `required`: a stack is painted bottom-first, so the thing
+  covering everywhere has to sit under everything patching it, and the index does not list it first.
+  Every other entry keeps its box and zoom range, which is what lets a tile outside one skip it
+  without a request leaving the node. The encoding is asked for rather than read — an index rarely
+  states it, Mapterhorn's files are all terrarium and its JSON never says so, and a terrain source
+  read with the wrong one is a cliff face.
+
+  The list is fetched **by the node**, not the browser: the console is often on a different network,
+  and what matters is that the machine which will read the archives can reach them. **Check** shows
+  what an import would write before it writes anything.
+
+  In the editor an imported batch is one row rather than several hundred cards, with **Re-import**
+  and **Remove all**. A re-import replaces only what came from that same address — anything typed by
+  hand is left alone — and puts the batch back **where it already was** rather than on the end:
+  painting order is the whole meaning of a stack, and a batch that moved each time would quietly bury
+  a local override, a day later, on a schedule, with nothing to say why the map had changed.
+
+
+- **A stack can follow a provider's file list on a timer.** A stack feed row with **Into stack** set
+  reads a URL list rather than another node's recipes, and keeps the named stack level with it.
+  Mapterhorn's index has grown through several versions, and a node that imported it once is a node
+  serving whenever that was. The row says which of the two it is by whether **Into stack** is filled
+  in — a feed of recipes names its own stacks and an index cannot, being a list of files with no
+  opinion about what they are for, so the row supplies the stack and the encoding as well.
+
+  It reconciles exactly as **Re-import** does: hand-written sources untouched, the batch back where it
+  was, a withdrawn file dropped rather than left to 404 per tile. A poll where nothing changed writes
+  nothing at all — not the same recipe again, which would move its revision and with it every tile
+  cached against it. A stack that does not exist yet is created; one that does keeps everything it had.
+
+
+### 🐞 Bug fixes
+- **A stack reported no encoding unless its recipe stated one outright.** Reading `output.encoding`
+  alone reports null for the ordinary recipe — the one that re-encodes to nothing and writes whatever
+  its base source is written in, which is every imported stack, since an import sets the encoding per
+  source. Three places read it that way and all three were wrong about terrain: the console offered no
+  terrain preview, the TileJSON told clients nothing so terrarium heights rendered through the mapbox
+  formula, and a baked archive was written with no encoding metadata at all — an export that had to be
+  corrected by hand afterwards to be readable.
+
+- **A stack's TileJSON listed every source.** For a stack imported from a provider's index that is 458
+  addresses and 55 KB, in a document every map load fetches, describing files a client cannot use and
+  will never ask for — it reads tiles from the XYZ endpoint. It now names 25 with a count of the rest.
+  The extension it advertises is the one the endpoint actually serves, too: a stack whose sources state
+  no format has no coverage format to read, and the document said `.bin` for tiles answered as webp.
+
+- **Cancel asked you to fill the form in first.** A `<button>` in a form submits, and a submit is
+  checked against the form's required fields before it goes anywhere — so cancelling the import or
+  warm dialog answered "Please fill out this field" about a field that was about to be discarded.
+
+
 ## 0.89.0
 ### ✨ Features and improvements
 - **Both feeds are linked at the foot of the public page.** `RSS feed` is `archive RSS feed` now and

package/README.md

@@ -807,6 +807,7 @@ which the endpoint answers 501.
 | `GET`                  | `/api/torrents/:infoHash/stacks`                            | Which stacks would break if this archive were removed, and how                                                                                                                                                                                                                                                                             |
 | `GET`, `PUT`, `DELETE` | `/api/stacks/:id/raw`, `/api/stacks/:id`                    | Read a recipe as written, save one, or remove it. `/raw` is the recipe; `/api/stacks` is what it resolved to                                                                                                                                                                                                                               |
 
+| `POST` | `/api/stacks/:id/import` | Import a provider's list of PMTiles URLs as sources on this stack. An index with an `items` list, or a plain list of addresses; `dryRun` says what it would write without writing it |
 | `POST`, `DELETE` | `/api/stacks/:id/bake` | Run the stack over its sources and write the result as an archive, or stop one that is running. Answers as soon as the job starts; watch it on `/api/stacks` |
 | `GET` | `/stacks/:id/preview` | A map of a stack, for looking at it — **public** |
 | `GET` | `/stacks/:id/tiles.json` | TileJSON for a stack — **public**. `maxzoom` is the maximum over its sources, not the minimum |

package/docs/configuration.md

@@ -20,6 +20,7 @@ Some settings only take effect on restart. Those are marked **restart**.
 - [Scheduled sources](#scheduled-sources)
 - [Subscriptions](#subscriptions)
 - [Serving tiles](#serving-tiles)
+- [S3 buckets](#s3-buckets)
 - [Mutable publishing](#mutable-publishing)
 - [Authentication](#authentication)
 - [Statistics](#statistics)
@@ -776,6 +777,96 @@ connection, a piece already in flight. Ten attempts later it is one piece at the
 far end of an archive nobody has finished, and asking every fifteen seconds
 achieves nothing but log lines.
 
+## S3 buckets
+
+A stack source may be an address rather than an archive this node holds, and
+that address may name an object in a bucket:
+
+```json
+{
+  "sources": [{ "url": "s3://terrain/planet.pmtiles", "encoding": "terrarium" }]
+}
+```
+
+Only for a bucket that is **not public**. A public object, or a presigned URL,
+is an ordinary HTTPS address: write it as one and none of this applies.
+
+```json
+{
+  "s3": [
+    {
+      "bucket": "terrain",
+      "endpoint": "https://minio.lan:9000",
+      "region": "us-east-1",
+      "accessKeyId": "…",
+      "secretAccessKey": "…"
+    }
+  ]
+}
+```
+
+| field             | default            |                                                      |
+| ----------------- | ------------------ | ---------------------------------------------------- |
+| `bucket`          | any                | which bucket this row is for; unset matches all      |
+| `endpoint`        | AWS for the region | what makes it S3-compatible rather than S3           |
+| `region`          | `us-east-1`        | signed into every request, whether or not it is used |
+| `accessKeyId`     | —                  |                                                      |
+| `secretAccessKey` | —                  |                                                      |
+| `sessionToken`    | unset              | for temporary credentials                            |
+| `pathStyle`       | path, unless AWS   | `false` puts the bucket in the hostname              |
+
+A source is read through the row naming its bucket, or through whichever row
+names none: one set of credentials for a whole account is the ordinary case,
+and writing the same keys once per bucket is busywork with a copy-paste
+mistake in it.
+
+### The endpoint is the whole point
+
+MinIO, Ceph, Garage, Cloudflare R2, Wasabi and Backblaze B2 all answer the
+same protocol at their own address, so naming that address is the difference
+between "S3" and "S3-compatible". Path-style addressing is what they speak and
+is the default here; AWS withdrew it for buckets made after September 2020, so
+an `amazonaws.com` endpoint puts the bucket in the hostname instead. Set
+`pathStyle` only for a server that insists on the other one.
+
+### Credentials from the environment
+
+With no rows configured at all, the standard variables are read — the same
+ones the AWS CLI, rclone, go-pmtiles and tileserver-gl use, so a machine
+already set up to reach a bucket needs nothing written here:
+
+| variable                                     |                                 |
+| -------------------------------------------- | ------------------------------- |
+| `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY` | required for any of it to apply |
+| `AWS_SESSION_TOKEN`                          | temporary credentials           |
+| `AWS_REGION`, `AWS_DEFAULT_REGION`           | region                          |
+| `AWS_S3_ENDPOINT`, `AWS_ENDPOINT_URL_S3`     | endpoint                        |
+| `AWS_S3_FORCE_PATH_STYLE`                    | addressing                      |
+
+### Signed here, not by an SDK
+
+SigV4 for a GET is a hash, four HMACs and a string, all of which `node:crypto`
+already has. An AWS client would be a large dependency for one signature, and
+an optional one would make reading a bucket work on some installs and not on
+others for no reason the operator could see — so it is neither optional nor a
+dependency. The signer is checked against AWS's own published test vectors
+rather than against itself.
+
+Every request is signed as it goes out, including the range: the header, each
+directory and each tile. Nothing is presigned and nothing is cached but the
+bytes, so revoking a key stops reads at the next request rather than at the
+next restart. Changing the settings drops the open readers for the same
+reason.
+
+### What a bucket source is, and is not
+
+It is a URL source in every other respect — see
+[tile-stacks.md](tile-stacks.md), "A source read straight from a URL". It
+carries its own `minzoom`, `maxzoom` and `bounds`, it is never seeded or
+retired, and it can be baked: a stack over private buckets exports to an
+ordinary archive with an infohash, which is the path from "terrain we hold in
+a bucket" to "terrain we publish".
+
 ## Mutable publishing
 
 Announcing the current build of each category over the DHT (BEP 46).

package/docs/tile-stacks.md

@@ -53,6 +53,8 @@ of its parts.
 - [What a mask has to match](#what-a-mask-has-to-match)
 - [Feathering a seam](#feathering-a-seam)
 - [A stack as a source](#a-stack-as-a-source)
+- [A source read straight from a URL](#a-source-read-straight-from-a-url)
+- [Importing a list of URLs](#importing-a-list-of-urls)
   - [Exporting on a schedule](#exporting-on-a-schedule)
 - [Finding a stack](#finding-a-stack)
 - [Syncing a stack to another node](#syncing-a-stack-to-another-node)
@@ -255,20 +257,23 @@ it differs from the snake_case rio-rgbify-merge uses.
 
 ### Source fields
 
-| Field                  | Meaning                                                                                                      |
-| ---------------------- | ------------------------------------------------------------------------------------------------------------ |
-| `category`             | Resolve to the newest build in this category.                                                                |
-| `archive`              | Or pin one infohash. Exactly one of the two.                                                                 |
-| `required`             | A tile fails rather than being served without this source. Defaults true for the bottom-most, false above.   |
-| `encoding`             | `mapbox` or `terrarium`. Elevation space only.                                                               |
-| `baseVal` / `interval` | Mapbox decode offset and step. Default `-10000` and `0.1`.                                                   |
-| `maskValues`           | Decoded heights meaning "no data here". Elevation space only.                                                |
-| `maskRange`            | `[low, high]` in metres, or a list of them. Everything inside is nodata. Elevation space only.               |
-| `heightAdjustment`     | Metres, added **after** masking. Elevation space only.                                                       |
-| `feather`              | Pixels to fade in over wherever the source stops: a `cutline`, `bounds`, or the holes a mask leaves. Max 64. |
-| `featherMetres`        | The same fade written as metres of ground, worked out per tile. Usually the one to reach for.                |
-| `opacity`              | `0`–`1`, scales the source alpha. RGBA space only.                                                           |
-| `blend`                | `normal`, `multiply`, `screen`, `overlay`, `darken`, `lighten`. RGBA space only.                             |
+| Field                  | Meaning                                                                                                        |
+| ---------------------- | -------------------------------------------------------------------------------------------------------------- |
+| `category`             | Resolve to the newest build in this category.                                                                  |
+| `archive`              | Or pin one infohash.                                                                                           |
+| `stack`                | Or another stack, merged as heights. See "A stack as a source".                                                |
+| `url`                  | Or an archive read straight from an http(s) address, no torrent involved. Exactly one of the four.             |
+| `minzoom` / `maxzoom`  | Skip a `url` source outside this zoom range before opening it. Optional; the header answers anyway without it. |
+| `required`             | A tile fails rather than being served without this source. Defaults true for the bottom-most, false above.     |
+| `encoding`             | `mapbox` or `terrarium`. Elevation space only.                                                                 |
+| `baseVal` / `interval` | Mapbox decode offset and step. Default `-10000` and `0.1`.                                                     |
+| `maskValues`           | Decoded heights meaning "no data here". Elevation space only.                                                  |
+| `maskRange`            | `[low, high]` in metres, or a list of them. Everything inside is nodata. Elevation space only.                 |
+| `heightAdjustment`     | Metres, added **after** masking. Elevation space only.                                                         |
+| `feather`              | Pixels to fade in over wherever the source stops: a `cutline`, `bounds`, or the holes a mask leaves. Max 64.   |
+| `featherMetres`        | The same fade written as metres of ground, worked out per tile. Usually the one to reach for.                  |
+| `opacity`              | `0`–`1`, scales the source alpha. RGBA space only.                                                             |
+| `blend`                | `normal`, `multiply`, `screen`, `overlay`, `darken`, `lighten`. RGBA space only.                               |
 
 `attribution` is not optional in practice. A stack is a derived work of every
 source in it, and the thing that reliably gets lost when tiles are combined is
@@ -1631,6 +1636,224 @@ would wait a day before showing it had ever run.
 That is what a second node serving the same stacks wants: only one of them
 should be the node that bakes.
 
+## A source read straight from a URL
+
+A source may name a URL instead of a category, an archive or a stack:
+
+```json
+{
+  "id": "mapterhorn",
+  "sources": [
+    {
+      "url": "https://download.mapterhorn.com/planet.pmtiles",
+      "encoding": "terrarium",
+      "minzoom": 0,
+      "maxzoom": 12
+    },
+    {
+      "url": "https://download.mapterhorn.com/6-32-31.pmtiles",
+      "encoding": "terrarium",
+      "bounds": [0, 0, 5.625, 5.625],
+      "minzoom": 13,
+      "maxzoom": 13
+    }
+  ]
+}
+```
+
+Mapterhorn publishes terrain this way: a global base to about z12 and several
+hundred regional patches reaching higher, each one a plain HTTPS download and
+none of it in a swarm this node is part of. Downloading 11.8 TiB of it first,
+to get it into a category the ordinary way, is not a step anybody wants — the
+whole point is to read a tile at a time, from wherever it already is.
+
+### Read the same way a swarm archive is
+
+`FetchSource`, from the `pmtiles` package this project already depends on,
+asks for byte ranges the way `TorrentSource` does — a request for a tile costs
+the header once, the directory once, and the tile itself, not the file. A URL
+source keeps its own small cache of open readers, separate from the one
+catalog archives share, so a stack naming hundreds of them is not competing
+with the archives this node actually seeds for the same budget.
+
+Parent climbing works the same way it does for a category source: a shallow
+archive — Mapterhorn's own base stops at z12 — is upscaled for a deeper
+request exactly as GEBCO is, through the same code, because reading one is now
+a question of which store method answers and nothing else.
+
+### Adding one by hand
+
+**Add source → an address you type…** in the stack editor, which is the same
+source the importer writes and edits the same way. The card asks for two
+things nothing else can know: the address, and the zoom range the archive
+holds. A catalog archive states its range in its own header and a URL source
+has no header this node has read, so an unstated range means every tile asks
+it — correct, and the thing worth avoiding once there are several.
+
+Anything published as a plain HTTPS download works, which includes an S3
+bucket that serves range requests: an object URL, or a presigned one, is an
+address like any other, and this asks for byte ranges the way any PMTiles
+reader does. What it cannot do is a private bucket named `s3://…`, which is
+not a URL a browser or a fetch can follow — that needs a signed request and
+somewhere to keep the credentials, and neither exists here yet.
+
+### Cheap to have hundreds of
+
+The one thing that makes hundreds of these practical rather than merely
+possible: `bounds` and the two new per-source fields, `minzoom` and `maxzoom`,
+are checked **before** anything is opened. A tile request outside a source's
+stated box, or at a zoom no climb from here could reach into its stated range,
+skips that source without a request leaving this node. For a stack built from
+Mapterhorn's own file list — one tile mostly intersects a global base and one
+or two regional patches — that is most of several hundred sources, on every
+tile, settled from the recipe alone.
+
+Without a stated zoom range this still works, just later: `FetchSource`'s own
+header read answers "no tile" cheaply once opened, the same way a catalog
+archive's does. The static check is what avoids opening most of them at all,
+not what makes opening one safe.
+
+### What it does not do
+
+**Never passed through raw.** A URL source has no infohash to answer the
+byte-for-byte question with, so a stack containing one always decodes —
+`needsCodec` says so from the recipe, the same answer a nested stack gets and
+for the same reason.
+
+**Not seeded, not retired, not rebuilt.** This node does not hold a copy, so
+none of the mechanisms that apply to an archive apply here. The URL is not
+content-addressed either — nothing stops whoever publishes it editing the file
+in place — so it is treated as unstable for caching, the same as a category
+and for a related reason: a category is unstable because the build under it
+moves, a URL because the operator does not control what is at the other end
+of it.
+
+**Feathering a mask edge only partly.** A URL source that both masks and fades
+needs its parents read to measure the ramp, exactly as a catalog source does —
+and that path is wired up. What is not: a parent read that fails is caught and
+skipped silently, the same forgiving rule `readMaskEdges` already applies to a
+catalog source whose parent cannot be reached, so a patchy host degrades the
+ramp rather than failing the tile.
+
+## Importing a list of URLs
+
+Naming 458 sources by hand is not work anybody should do once, let alone again
+when the provider adds a file. A provider that publishes terrain as many
+separate archives generally publishes an index of them as well, and that index
+already carries exactly what the merge needs in order to skip a source without
+opening it.
+
+**Stacks → Import URL list…** takes the address of one. Mapterhorn's is
+`https://download.mapterhorn.com/download_urls.json`.
+
+### Two shapes, detected rather than declared
+
+An **index**: an object with an `items` list, or a bare array of the same
+entries. Each entry needs a `url` and the six numbers Mapterhorn's carries —
+`min_lon`, `min_lat`, `max_lon`, `max_lat`, `min_zoom`, `max_zoom` — which
+become the source's `bounds` and its `minzoom`/`maxzoom`.
+
+A **plain list** of addresses, one per line or as a JSON array of strings, for
+a provider that publishes no index. Blank lines and `#` comments are ignored.
+Nothing states a box or a zoom range, so every source is opened for every tile
+it might cover — which is the cost of having no index, and the reason the index
+path is worth preferring.
+
+Which one it is is worked out from the document, because somebody pasting an
+address has no reason to know which their provider answers with, and guessing
+wrong would be a silent import of nothing rather than an error.
+
+### The global file becomes the base
+
+An entry whose box covers the world — Mapterhorn's `planet.pmtiles` says -180
+to 180 and ±85.0511 — is made `sources[0]` and marked `required`. A stack is
+painted bottom-first, so the thing covering everywhere has to sit underneath
+everything patching it, and the index does not list it first.
+
+It is given no `bounds`: a clip excluding nothing is a rasterise on every
+partial tile for an answer that was never in doubt. Where several entries claim
+to be global, the one reaching deepest wins and the rest become ordinary
+layers.
+
+### The encoding is the importer's to state
+
+An index rarely says. Mapterhorn's files are all terrarium and its JSON never
+mentions it, so the dialog asks and sets it on every imported source — a
+terrain source read with the wrong encoding is a cliff face rather than a
+mistake anybody has to guess at.
+
+### Re-importing
+
+Every imported source carries `importedFrom`, naming the list it came from.
+That is what lets the editor draw one row for several hundred sources, and what
+a re-import uses to know which are its to replace. A source typed by hand
+carries none and is never touched.
+
+The batch also goes back **where it already was**, rather than on the end.
+Painting order is the whole meaning of a stack: a batch that moved on every
+re-import would quietly bury a local override somebody had deliberately placed
+above it — a day later, on a schedule, with nothing to say why the map changed.
+
+### Fetched by the node, not the browser
+
+The console is often on a different network from the node, and a list is only
+useful if the machine that will actually read those archives can reach them. A
+458-entry index is also not something a browser should be parsing and posting
+back a megabyte of.
+
+`POST /api/stacks/<id>/import` takes `{ url, encoding }`. With `dryRun` it
+answers what it would write without writing it, which is what **Check** in the
+dialog shows and what the editor's **Re-import** uses to update an unsaved
+draft. A draft can send its own `sources` along, so the batch comes back merged
+into the order the operator is holding rather than into the stored one.
+
+### Following the list instead of importing it once
+
+Mapterhorn's index has grown through several versions, and a node that imported
+it in March is a node serving March. The same address goes in **Settings →
+Feeds → Stack feeds** with **Into stack** naming the stack to keep level, and
+the poll that already checks other nodes for recipes checks this for files.
+
+The row says which of the two it is by whether **Into stack** is filled in. A
+feed of recipes names its own stacks and this one cannot: an index is a list of
+files with no opinion about what they are for, so the row supplies the stack
+and the encoding the index does not state.
+
+Reconciliation is the same one **Re-import** does, for the same reasons and
+with the same guarantees: hand-written sources are left alone, the batch stays
+where it was put, and a file the provider withdrew stops being asked for. A
+poll where nothing changed writes nothing at all — not the same recipe again,
+which would move its revision and with it every tile cached against it.
+
+A stack named here that does not exist yet is created. One that does keeps
+everything it had; following a list never amounts to taking a recipe over.
+
+### What a bulk import cannot say
+
+Every imported source gets a box and a zoom range, because the index states
+them. It gets no **mask** — no per-source clip beyond its bounding box, no
+feathered edge — because there is nowhere in an index for one to come from.
+
+That is fine where the provider has already merged for you. Mapterhorn's files
+are cut so that a patch and the planet agree at the seam, so a rectangular clip
+is the right clip. It is not fine where two sources genuinely overlap at
+different quality and the edge between them matters, and there the answer is to
+import the batch and then write the overlapping source by hand, above it.
+
+Note what happens if you edit an imported source instead: a mask typed onto one
+is wiped by the next re-import, which replaces the batch entire. On a schedule
+that is a map changing at 4am with nothing to say why. A hand-written source
+carries no `importedFrom`, is never replaced, and is the place for anything the
+index could not have told us.
+
+### Seeing them in the console
+
+A stack's source list collapses behind a summary once there are more than
+five, with the count and the base named on the fold. A stack imported from a
+provider's list is several hundred rows and drawing them flat buries every
+other stack on the page under one of them; a stack of a base and a layer or
+two stays open, since folding that hides nothing worth a click.
+
 ## Finding a stack
 
 A stack has no infohash and appears in no feed, so nothing about it is

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "pmtiles-swarm",
-  "version": "0.89.0",
+  "version": "0.91.0",
   "description": "BitTorrent distribution for PMTiles map archives: create torrents, watch folders, publish and subscribe to RSS feeds, and seed through qBittorrent or an embedded client",
   "type": "module",
   "main": "src/index.js",