pmtiles-swarm 0.79.1 → 0.89.0

package/CHANGELOG.md

@@ -7,6 +7,276 @@
 ### 🐞 Bug fixes
 - _...Add new stuff here..._
 
+## 0.89.0
+### ✨ Features and improvements
+- **Both feeds are linked at the foot of the public page.** `RSS feed` is `archive RSS feed` now and
+  `stack RSS feed` is beside it. One name was unambiguous while there was one feed; with a second, a
+  reader following the old name would have got archives when they wanted recipes and had nothing on
+  the page to tell them otherwise.
+
+
+### 🐞 Bug fixes
+- **A feed carrying every category.** `/categories.xml`, beside `/feed.xml` and `/stacks.xml` — the
+  set was inconsistent, and following all of a node's categories meant adding each by hand and
+  remembering to add the next one, which is not following its categories at all. It is what a
+  subscriber wants when they want everything this node files: including categories added later.
+
+  The items are archives, because a category has no bytes of its own and the build it resolves to is
+  what a subscriber would join — which also means every existing consumer already knows how to read
+  it, magnets and enclosures included. That is what separates it from the whole catalogue: `/feed.xml`
+  carries every build a node holds, and this carries the current one of each category. On a node
+  keeping four builds apiece the difference is fourfold.
+
+  In the footer of the public page it replaces `categories JSON`, which was the odd one out among
+  three feeds. `/latest/` is untouched and remains the JSON index.
+
+
+## 0.88.0
+### ✨ Features and improvements
+- **An archive card is built like a category's.** Copy buttons for the addresses that belong
+  somewhere else — TileJSON, magnet, and a **source URL** of its own, the TileJSON URL with the
+  `.torrent` and magnet in its fragment, pinned to that build rather than following the category.
+  The preview, `.torrent` and download stay links, because a page is followed and a file is saved.
+  The printed TileJSON row went with the button that replaced it: an address in full beside a button
+  that copies it is the same fact twice, and it was the longest line on the card.
+
+
+### 🐞 Bug fixes
+- **A terrain stack was offered no hillshade preview on the public page.** The listing reported its
+  encoding as whatever `output.encoding` restated, which for a recipe saying "same as the sources" —
+  the ordinary case — is nothing. So the page could not tell a terrain stack from an imagery one and
+  offered neither the raw preview nor the terrain one. It reports what the stack actually serves now,
+  and reports nothing for imagery, where a guess would have offered a hillshade of a photograph.
+
+- **A stack whose sources disagreed about their encoding wrote tiles in more than one of them.** With
+  no `output.encoding`, the merge took the encoding of whichever source answered first — and which
+  source answers varies by tile, since the base is sparse in one place and the layer above covers in
+  another. One stack wrote one tile as mapbox and the next as terrarium, with the TileJSON in front
+  describing neither: correct in one place, a cliff face in another, for no reason the recipe showed.
+  It follows the base source now, which is a property of the recipe rather than of the tile, and the
+  listing and the merge are held to the same answer by a test.
+
+
+## 0.87.0
+### ✨ Features and improvements
+- **A feed per stack, and an address to copy for it.** `/stacks/<id>.xml` carries one recipe, for
+  following a single map out of somebody's several rather than everything they publish. An **RSS**
+  button on the stack's row in the console and beside TileJSON and XYZ on the public page. Copied
+  rather than followed: the address is for another node's settings, and a browser shown an RSS
+  document mostly offers to download it. A stack this node adopted has no button and its feed 404s —
+  somebody else's recipe is theirs to publish.
+
+
+### 🐞 Bug fixes
+- **The copied XYZ address came back percent-encoded.** `http://…/stacks/test/%7Bz%7D/%7Bx%7D/%7By%7D.png`
+  rather than `{z}/{x}/{y}`, which no client will take. The public page resolves a relative address
+  against the page's own URL to make it absolute, and `new URL` percent-encodes braces because they
+  are not legal in a path — and an XYZ template is very little but braces. The TileJSON link was
+  unaffected, having none. Resolving still happens; the two sequences it introduces are put back.
+
+
+## 0.86.0
+### ✨ Features and improvements
+- _...Add new stuff here..._
+
+- **Stack recipes travel between nodes.** `GET /stacks.xml` is this node's own stacks as a feed;
+  **Settings → Feeds → Stack feeds** follows another node's and adopts what it carries. Archives
+  already travelled by category — a builder feeding two tile servers gave them the same archives —
+  and this is how the recipes that combine them travel with them, so a stack is written once and
+  corrected once.
+
+  This section of the docs used to argue against a feed, on the grounds that a stack is a mutable
+  document and syncing one is about conflicts. That holds where two nodes both edit a recipe and is
+  not the arrangement anybody runs: one node authors, the rest follow. What survives from the
+  objection is handled directly — **a stack made on this node is never overwritten by one arriving
+  under the same name**, which is refused and said.
+
+  A recipe is adopted under the publisher's own id, so `planet-terrain` answers at the same URL on
+  the builder and on every replica; namespacing it would have given three nodes three URLs and
+  defeated the point. A recipe naming a source this node has not got is adopted anyway and reports
+  the missing source until it arrives — refusing it would mean a replica could not be set up until
+  every archive had finished downloading, which is backwards.
+
+  What happens when a feed stops carrying a stack is the feed's own setting: keep it and say so, or
+  remove it here too.
+
+### 🐞 Bug fixes
+- _...Add new stuff here..._
+
+## 0.85.1
+### ✨ Features and improvements
+- _...Add new stuff here..._
+
+- **An export no longer pretends a stack has categories.** Both doors fell back to
+  `stack.categories` when none were given — a field the typedef never had, validation never checked,
+  the editor has no box for and nothing ever wrote. A category is what an *archive* is filed under,
+  and a stack has no bytes and no infohash for that to be about.
+
+  Left empty the archive is unfiled: held and seeded, in no category and no feed. The scheduled
+  export row says `unfiled` and the dialog says so in full, rather than offering a default that
+  could not exist.
+
+### 🐞 Bug fixes
+- _...Add new stuff here..._
+
+## 0.85.0
+### ✨ Features and improvements
+- _...Add new stuff here..._
+
+- **A scheduled export can be served by the node that baked it.** The **Local file** choice, the same
+  one every other row under Feeds offers: http, http + web seed, http + catalog. It is worth more
+  here than anywhere else — a baked archive is already on this disk, so serving it costs a route
+  rather than a download, and without it a scheduled export produces something only peers can reach.
+  A nightly build behind a URL that is always current is usually the point of scheduling one.
+
+  `publishDir` and `webSeedBase` remain the other half of the same question, for a directory
+  something else already serves. The two are not exclusive: publishing to a served directory *and*
+  offering this node as a web seed gives a client two places to get the same bytes.
+
+### 🐞 Bug fixes
+- _...Add new stuff here..._
+
+## 0.84.0
+### ✨ Features and improvements
+- _...Add new stuff here..._
+
+- **Scheduled exports are rows, like every other automation beside them.** `stackExports` in the
+  config, edited under Settings → Feeds with the same row editor the monitored folders and the
+  scheduled sources use: add a row, choose the stack from a dropdown, say when, and fill in the rest
+  — categories, builds to keep, keep for how many days, save location, publish directory, web seed
+  base, archive name, attribution, description.
+
+  **Several rows may name one stack.** That is what the two earlier shapes could not say: an `export`
+  block on the recipe, and then a table of one row per stack, both hold exactly one schedule — and a
+  nightly build to the fast disk beside a weekly one published elsewhere is an ordinary thing to
+  want. It also puts the schedule where the other automations already are: a watched folder says what
+  *this machine* does, not what a map is, and keeping it out of the recipe means a recipe copied to
+  another node does not quietly start baking there.
+
+  The export dialog no longer offers to repeat — a stack may have several schedules and a dialog
+  opened on the stack cannot say which it would edit. It exports once and points at the settings tab.
+  A stack's row in the Stacks view still shows the schedules aimed at it.
+
+### 🐞 Bug fixes
+- _...Add new stuff here..._
+
+## 0.83.0
+### ✨ Features and improvements
+- _...Add new stuff here..._
+
+- **A scheduled export is set up like the other automations beside it.** The row under Settings →
+  Feeds is the whole of one now, not only the timer: categories, builds to keep, keep for how many
+  days, archive name, attribution, description, where the data lands, a publish directory and a web
+  seed base. Those are the manual export dialog's fields plus the two retention rules every other
+  automation on that tab already had, which is the point — a scheduled export is set up in one place
+  rather than half in a settings tab and half in a dialog.
+
+  Retirement is the part that needed something new underneath. `keep` and `keepDays` are applied by
+  the same code a watched folder uses, but retiring needs a *family* — which archives are builds of
+  the same map — and a bake marked nothing, so there was nothing to compare. An archive now records
+  `source.stack`, and the family is every archive that names this stack. Without it a nightly export
+  is a disk that fills at one archive a night.
+
+### 🐞 Bug fixes
+- _...Add new stuff here..._
+
+## 0.82.0
+### ✨ Features and improvements
+- _...Add new stuff here..._
+
+- **Export schedules are set under Settings → Feeds.** Not only the two global settings that landed
+  there in 0.81.1 — the schedules themselves. **Scheduled exports** is a row per stack: never, every
+  day at a time, or every so many hours, saved onto that stack's recipe. Feeds is where somebody goes
+  to say when a thing runs, so it is where they are set; the export dialog's **Repeat** control still
+  writes the same schedule, for setting one while you are already there.
+
+  Turning a stack to _never_ pauses it with `enabled: false` rather than deleting the block. Where it
+  lands, what it is called and which categories it is filed under live in the same place, and those
+  should survive being paused.
+
+### 🐞 Bug fixes
+- _...Add new stuff here..._
+
+## 0.81.1
+### ✨ Features and improvements
+- _...Add new stuff here..._
+
+- **The scheduled-export settings are under Settings → Feeds.** `stacks.scheduledExports` and
+  `stacks.exportIntervalHours` had no schema entry, so they were config-file-only and invisible in
+  the console. Feeds rather than a group of their own, because that tab is already where the
+  automations that bring a file in on a timer sit — a scheduled source watching an upstream
+  directory, a subscription following someone else's feed. A scheduled export is the same kind of
+  thing; it just produces the file here instead of fetching it, and it lands in a category and goes
+  out over the feed exactly as a fetched one does.
+
+### 🐞 Bug fixes
+- _...Add new stuff here..._
+
+## 0.81.0
+### ✨ Features and improvements
+- _...Add new stuff here..._
+
+- **A stack can export itself on a schedule.** An archive is a snapshot: a stack over categories
+  follows a rebuild and a file baked from it does not, so it goes stale the moment its sources move
+  and somebody has to notice. An `export` block on the recipe says when — `at` for a time of day in
+  UTC, `everyHours` or `everyMinutes` for an interval, the same shape a scheduled source uses and
+  read by the same code — along with everything the export dialog collects. In the console, **Repeat**
+  turns the Export button into **Save schedule**, and the stack's row shows when it runs.
+
+  The hard part is remembering across a restart. The source poller keeps last-run times in memory,
+  which is fine when a missed poll costs one poll; here it would cost the whole bake, every restart,
+  for hours. So it is written to `stack-exports.json`, and written *before* the bake finishes — a
+  restart mid-export must not start it from the top, since the checkpoint is what carries it on.
+
+  A run whose sources have not moved is skipped: `bakeRevision` is recorded beside the time, and an
+  identical archive is the same map under a new infohash that then has to be seeded beside the one it
+  duplicates. It will not run two at once, will not start one over a bake already running, and does
+  not record a refusal as a run — a location that is full is something somebody fixes, and a schedule
+  that gave up quietly would hide that it ever ran. `stacks.scheduledExports: false` turns it off,
+  which is what a second node serving the same recipes wants.
+
+### 🐞 Bug fixes
+- _...Add new stuff here..._
+
+## 0.80.0
+### ✨ Features and improvements
+- **A stack can be a source in another stack.** `{ "stack": "jaxa-with-gebco" }` beside `category`
+  and `archive`. A base worked out once — terrain over bathymetry, masked at the coast and faded
+  across it — is a thing to reuse rather than retype, and a recipe that names it follows every later
+  correction to it, exactly as a source over a category follows a rebuild.
+
+  It is merged as **heights**: the inner stack is evaluated for the tile and handed straight to the
+  merge above, with no encode and decode in between. That saves two conversions per tile and, more
+  to the point, does not round a value on its way from one merge into the next. So a nested source
+  may say anything that acts on heights — `maskValues`, `maskRange`, `heightAdjustment`, `cutline`,
+  `bounds`, the fade, `opacity`, `blend` — and nothing that describes stored bytes. `encoding` and
+  its parameters are refused, and so is `maskColors`, which compares channels as an archive stored
+  them and has none to compare here.
+
+  A loop is refused by name on the way down, so a stack naming itself and a ring of three are the
+  same case and neither needs a depth counter to stop. The depth limit is separate and is four:
+  every level is a full merge of everything under it, so a tile's cost multiplies rather than adds.
+
+  Coverage folds in one level down, and the ETag carries the inner stack's own ETag rather than its
+  id — without that, editing the inner recipe would leave the outer one serving from a cache that
+  still believed in it, and propagation is the whole point of naming a stack instead of copying it.
+  The console offers held stacks in the source picker, alongside categories and archives.
+
+- **An export can set its attribution, and starts with the right one.** An archive travels without
+  the style that loaded it — seeded, mirrored, opened by people who never saw the stack it came from
+  — so its own metadata is the only place the credit survives. The export dialog has an
+  **Attribution** field, filled in from the stack: its own where the recipe states one, otherwise
+  every source's joined. Editable, because an export may be published under terms the recipe does
+  not know; filled in rather than blank, because unlike the description it is not something only the
+  person exporting knows.
+
+  Joined with ` | ` rather than `, `, in the TileJSON as well as in the archive. These strings are
+  almost always HTML links and a comma between two anchors renders as part of the last one's text,
+  which is why MapLibre, Mapbox and OpenLayers all separate them this way.
+
+### 🐞 Bug fixes
+- _...Add new stuff here..._
+
 ## 0.79.1
 ### ✨ Features and improvements
 - _...Add new stuff here..._

package/README.md

@@ -66,7 +66,7 @@ been sending and receiving, kept in SQLite so it survives a restart; an indicato
 peers can reach this node at all; and a settings screen covering monitored folders, watched web
 locations, remote nodes, save locations, access tokens and the external-program hooks.
 
-**Publishes RSS.** `/feed.xml`, and `/feed/<category>.xml` per category. Plain RSS 2.0 with
+**Publishes RSS.** `/feed.xml` for every archive, `/categories.xml` for the build each category currently resolves to, `/stacks.xml` for the stack recipes, and `/feed/<category>.xml` per category. Plain RSS 2.0 with
 torrent enclosures, so **qBittorrent's built-in RSS auto-downloader can subscribe today** with no
 new software. Items also carry a namespaced description of the map — format, zoom range, bounds,
 tile count — so a subscriber can decide whether it wants a 72 GiB download before starting one.
@@ -813,6 +813,8 @@ which the endpoint answers 501.
 | `GET` | `/stacks/:id/:size/:z/:x/:y.:ext` | The same tile at 256 or 512 px. A tile's coordinates are an extent rather than a pixel count, so this is the same ground on a finer or coarser grid |
 | `GET` | `/stacks/:id/:z/:x/:y.:ext` | One tile of a stack — **public**. Answered by the topmost source holding it; `X-Stack-Sources` says which were asked and what each said |
 | `GET` | `/feed.xml`, `/feed/:category.xml`, `/latest/:category.xml` | RSS — **public** |
+| `GET` | `/categories.xml` | Every category, as the build each resolves to — **public** |
+| `GET` | `/stacks.xml`, `/stacks/:id.xml` | Stack recipes, for another node to follow — **public** |
 
 Everything under `/api/` is guarded once a credential is configured; tiles, TileJSON and the feeds
 never are. A `peer` token may read but not change, and may be narrowed to some categories. See