pmtiles-swarm 0.79.1 → 0.80.0

package/CHANGELOG.md

@@ -7,6 +7,45 @@
 ### 🐞 Bug fixes
 - _...Add new stuff here..._
 
+## 0.80.0
+### ✨ Features and improvements
+- **A stack can be a source in another stack.** `{ "stack": "jaxa-with-gebco" }` beside `category`
+  and `archive`. A base worked out once — terrain over bathymetry, masked at the coast and faded
+  across it — is a thing to reuse rather than retype, and a recipe that names it follows every later
+  correction to it, exactly as a source over a category follows a rebuild.
+
+  It is merged as **heights**: the inner stack is evaluated for the tile and handed straight to the
+  merge above, with no encode and decode in between. That saves two conversions per tile and, more
+  to the point, does not round a value on its way from one merge into the next. So a nested source
+  may say anything that acts on heights — `maskValues`, `maskRange`, `heightAdjustment`, `cutline`,
+  `bounds`, the fade, `opacity`, `blend` — and nothing that describes stored bytes. `encoding` and
+  its parameters are refused, and so is `maskColors`, which compares channels as an archive stored
+  them and has none to compare here.
+
+  A loop is refused by name on the way down, so a stack naming itself and a ring of three are the
+  same case and neither needs a depth counter to stop. The depth limit is separate and is four:
+  every level is a full merge of everything under it, so a tile's cost multiplies rather than adds.
+
+  Coverage folds in one level down, and the ETag carries the inner stack's own ETag rather than its
+  id — without that, editing the inner recipe would leave the outer one serving from a cache that
+  still believed in it, and propagation is the whole point of naming a stack instead of copying it.
+  The console offers held stacks in the source picker, alongside categories and archives.
+
+- **An export can set its attribution, and starts with the right one.** An archive travels without
+  the style that loaded it — seeded, mirrored, opened by people who never saw the stack it came from
+  — so its own metadata is the only place the credit survives. The export dialog has an
+  **Attribution** field, filled in from the stack: its own where the recipe states one, otherwise
+  every source's joined. Editable, because an export may be published under terms the recipe does
+  not know; filled in rather than blank, because unlike the description it is not something only the
+  person exporting knows.
+
+  Joined with ` | ` rather than `, `, in the TileJSON as well as in the archive. These strings are
+  almost always HTML links and a comma between two anchors renders as part of the last one's text,
+  which is why MapLibre, Mapbox and OpenLayers all separate them this way.
+
+### 🐞 Bug fixes
+- _...Add new stuff here..._
+
 ## 0.79.1
 ### ✨ Features and improvements
 - _...Add new stuff here..._

package/docs/tile-stacks.md

@@ -52,6 +52,7 @@ of its parts.
 - [Clipping a source to a shape](#clipping-a-source-to-a-shape)
 - [What a mask has to match](#what-a-mask-has-to-match)
 - [Feathering a seam](#feathering-a-seam)
+- [A stack as a source](#a-stack-as-a-source)
 - [Finding a stack](#finding-a-stack)
 - [Syncing a stack to another node](#syncing-a-stack-to-another-node)
 - [What the offline merge got wrong](#what-the-offline-merge-got-wrong)
@@ -231,7 +232,7 @@ it differs from the snake_case rio-rgbify-merge uses.
 | `boundsSource`        | Index into `sources` whose bounds become the stack's. Omit for the union.                                  |
 | `bounds`              | Explicit `[w, s, e, n]`. Wins over `boundsSource`.                                                         |
 | `minzoom` / `maxzoom` | Clamps. Default to the min and **max** over the sources.                                                   |
-| `attribution`         | Falls back to every source's, concatenated.                                                                |
+| `attribution`         | Falls back to every source's, joined with `\|`.                                                            |
 
 ### Source fields
 
@@ -252,8 +253,14 @@ it differs from the snake_case rio-rgbify-merge uses.
 
 `attribution` is not optional in practice. A stack is a derived work of every
 source in it, and the thing that reliably gets lost when tiles are combined is
-who the data belongs to. If it is omitted, the implementation should concatenate
-the sources' own TileJSON `attribution` strings rather than emit nothing.
+who the data belongs to. If it is omitted, every source's own TileJSON
+`attribution` is joined rather than nothing being emitted.
+
+Joined with `|`, not with a comma. These strings are almost always HTML links,
+and a comma between two anchors renders as part of the last one's text — which
+is why MapLibre, Mapbox and OpenLayers all separate them this way. The same
+string is what an export writes into the archive's metadata, so a file and the
+endpoint it was baked from credit their sources identically.
 
 ## Translating a rio-rgbify-merge config
 
@@ -812,6 +819,16 @@ that render as sea.
 image of nothing in particular, and `custom` carries its four factors or is not
 worth writing at all.
 
+`attribution` is the third, and the one an export cannot afford to leave out. An
+archive travels without the style that loaded it — it is seeded, mirrored and
+opened by people who never saw the stack it came from — so its own metadata is
+the only place the credit survives. The dialog is filled in from the stack: its
+own `attribution` where the recipe states one, and otherwise every source's
+joined with `|`, since a stack is a derived work of all of them. It is
+editable, because an export may be published under terms the recipe does not
+know about; it is filled in rather than blank, because unlike the description it
+is not something only the person exporting knows.
+
 ### Starting one, and watching it
 
 **Export to archive**, on the stack, beside Edit and Delete. `POST
@@ -1354,6 +1371,85 @@ visible of the three and the cheapest to detect, since whether a neighbour exist
 is a directory lookup rather than a decode. It ramps down toward an absent
 neighbour, and nothing else has to happen.
 
+## A stack as a source
+
+A source may name a stack instead of a category or an archive:
+
+```json
+{
+  "id": "hillshade-ready",
+  "sources": [
+    { "stack": "jaxa-with-gebco" },
+    { "category": "swissalti", "featherMetres": 50 }
+  ]
+}
+```
+
+The reason to want it is that a base worked out once — terrain over bathymetry,
+masked at the coast and faded across it — is a thing to reuse rather than
+retype. A recipe that names it follows every later correction to it, exactly as
+a source over a category follows a rebuild.
+
+### It is merged as heights
+
+The inner stack is evaluated for the tile being built and its heights are handed
+straight to the merge above, with no encode and decode in between. That is not
+only a saving of two conversions per tile: an encoding is lossy about what it
+cannot represent, and a value that survived the inner merge should not be
+rounded on its way into the outer one. The inner stack's `output` block still
+applies where it is served on its own URL — it just has no part in this.
+
+So a nested source may say anything that acts on heights, and nothing that
+describes stored bytes:
+
+| Field                                               | On a nested stack                |
+| --------------------------------------------------- | -------------------------------- |
+| `maskValues`, `maskRange`, `heightAdjustment`       | Yes, on the heights it produced  |
+| `cutline`, `bounds`, `feather`, `featherMetres`     | Yes                              |
+| `opacity`, `blend`                                  | Yes                              |
+| `encoding`, `baseVal`, `interval`, the four factors | Refused — nothing was stored     |
+| `maskColors`                                        | Refused — no channels to compare |
+
+`maskColors` is the one that reads like an omission and is not. It compares the
+three channels as the archive stored them, which is what makes it exact where a
+height mask has to round; a stack that was never stored has no such channels to
+compare, and masking the heights it decoded to would be a different operation
+wearing the same name.
+
+### Nothing passes through
+
+The short-circuit that hands back a source's own bytes cannot apply: there are
+no bytes. A stack with a nested source always decodes, merges and encodes, and
+always needs a codec — `needsCodec` says so from the recipe rather than at the
+first tile.
+
+### Loops, and depth
+
+A loop is refused by name on the way down: the resolver carries the chain of ids
+it has walked, and a source naming one already in it resolves to nothing rather
+than being followed. That covers a stack naming itself and a ring of three
+equally, and it needs no depth counter to terminate.
+
+The depth limit is a separate thing, for the chain that does not loop and is
+still nobody's intention. **Four**, because every level is a full merge of
+everything under it: the cost of one tile multiplies rather than adds, and a
+five-deep chain over three sources each is a request nobody meant to make.
+
+### What the outer stack inherits
+
+Coverage folds in, one level down: a nested stack answers for the ground its own
+sources cover, with the same minzoom, maxzoom and bounds it would advertise on
+its own, and its attribution joins the outer stack's.
+
+The ETag includes the inner stack's own ETag rather than only its id. Without
+that, editing the inner recipe would leave every outer tile being served from a
+cache that still believed in it — and the whole point of naming a stack rather
+than copying it is that a correction propagates.
+
+`isPinned` asks the same question one level down. A nested stack is only as
+pinned as what is underneath it, which is what decides whether the outer stack's
+tiles are safe to cache hard.
+
 ## Finding a stack
 
 A stack has no infohash and appears in no feed, so nothing about it is
@@ -1669,10 +1765,6 @@ handling whatsoever.
 
 ## Open questions
 
-- **Should a stack be able to stack another stack?** Composable and obviously
-  tempting; also an unbounded fan-out of swarm reads behind one request. If
-  allowed, the depth needs a hard limit and the resolution hash needs to include
-  the whole tree.
 - **Vector tiles.** Merging MVT layers from two archives is a real want and a
   completely different operation — decode protobuf, merge layer by layer,
   re-encode, with feature ID collisions to settle. It should be its own feature

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "pmtiles-swarm",
-  "version": "0.79.1",
+  "version": "0.80.0",
   "description": "BitTorrent distribution for PMTiles map archives: create torrents, watch folders, publish and subscribe to RSS feeds, and seed through qBittorrent or an embedded client",
   "type": "module",
   "main": "src/index.js",

package/src/api.js

@@ -3202,6 +3202,9 @@ export function createApp({
     resolveStack(stack, {
       archive: (hash) => catalog.get(hash) ?? null,
       category: (name) => newestIn(name, req),
+      // The recipe, so the resolver can walk it. Loops and depth are its
+      // business, not this one's.
+      stack: (id) => stacks?.get(id) ?? null,
     });
 
   /**
@@ -3308,6 +3311,7 @@ export function createApp({
           savePath: req.body?.savePath,
           name: req.body?.name,
           description: req.body?.description,
+          attribution: req.body?.attribution,
         });
         return res.status(202).json({ bake: job });
       } catch (error) {

package/src/bake-jobs.js

@@ -14,6 +14,7 @@ import {
   wasStopped,
 } from './bake.js';
 import { PixelWorker } from './pixels.js';
+import { stackCoverage } from './stacks.js';
 import { outputFormat } from './stack-tile.js';
 
 /**
@@ -409,7 +410,9 @@ export class BakeManager {
     // presses the button rather than an hour in.
     assertBakeable(resolved, codec);
 
-    const unresolved = resolved.sources.filter((source) => !source.entry);
+    const unresolved = resolved.sources.filter(
+      (source) => !source.entry && !source.nested,
+    );
     if (unresolved.length === resolved.sources.length) {
       const error = new Error(
         "none of this stack's sources resolved to an archive on this node",
@@ -601,6 +604,7 @@ export class BakeManager {
         filename: job.name,
         publishDir: job.publishDir ?? null,
         description: options.description ?? null,
+        attribution: options.attribution ?? null,
         categories: options.categories ?? null,
       },
       metadata: {
@@ -609,7 +613,11 @@ export class BakeManager {
         // description would fill in a field the dialog showed as empty, which
         // is a worse surprise than having no description at all.
         description: options.description,
-        attribution: resolved.stack.attribution,
+        // What the dialog was shown, which is the stack's own where it has one
+        // and every source's joined where it has not. An archive travels
+        // without the style that loaded it, so this is the only place the
+        // credit survives -- and a stack is a derived work of all of them.
+        attribution: options.attribution ?? stackCoverage(resolved).attribution,
         encoding: resolved.stack.output?.encoding,
         encodingFactors: resolved.stack.output,
         sparse: resolved.stack.sparse,

package/src/elevation.js

@@ -702,16 +702,24 @@ export function paintHeights(layers, weights = []) {
 export function mergeElevation(contributions, options) {
   const { size } = options;
   const layers = contributions.map((contribution) => {
-    if (!contribution?.raster) return null;
+    if (!contribution?.raster && !contribution?.heights) return null;
     const source = contribution.source ?? {};
 
-    let heights = decodeHeights(contribution.raster, source);
+    // A nested stack arrives as the numbers it produced rather than as pixels
+    // somebody has to decode: it was never stored, so there is nothing to
+    // decode and no channels to compare a colour against. Copied because the
+    // rest of this works in place and the caller's array is not ours.
+    let heights = contribution.heights
+      ? Float32Array.from(contribution.heights)
+      : decodeHeights(contribution.raster, source);
     // Both masks say the same thing -- nothing here -- and both have to run
     // before the height adjustment, which would otherwise shift the values
     // out from under the comparison.
     //
     maskHeights(heights, source.maskValues);
-    maskColors(heights, contribution.raster, source.maskColors);
+    if (contribution.raster) {
+      maskColors(heights, contribution.raster, source.maskColors);
+    }
     // A band as well as, not instead of: a source may have a sentinel it names
     // exactly and a range of ground it does not want either.
     maskRanges(heights, source.maskRange);
@@ -724,13 +732,9 @@ export function mergeElevation(contributions, options) {
     // On the output's grid before anything else: a source whose tiles are a
     // different size covers the same extent, only sampled more or less
     // finely, so this is a plain scale rather than a realignment.
-    if (contribution.raster.width !== size) {
-      heights = resampleToSize(
-        heights,
-        contribution.raster.width,
-        size,
-        options.resampling,
-      );
+    const width = contribution.width ?? contribution.raster.width;
+    if (width !== size) {
+      heights = resampleToSize(heights, width, size, options.resampling);
     }
 
     const parentZ = contribution.parentZ ?? options.z;

package/src/index.js

@@ -494,6 +494,7 @@ PMTILES_SWARM_PUBLIC_URL
         return resolveStack(stack, {
           archive: (hash) => catalog.get(hash),
           category: (name) => catalog.byCategory(name)[0] ?? null,
+          stack: (id) => stacks?.get(id) ?? null,
         });
       })
       .catch((error) => console.warn(`[bake] resume failed: ${error.message}`));

package/src/stack-tile.js

@@ -12,6 +12,7 @@ import { paddedKnown, parentsFor } from './mask-edge.js';
 import {
   INSIDE,
   OUTSIDE,
+  PARTIAL,
   classifyTile,
   cropMask,
   featherMask,
@@ -65,7 +66,11 @@ const PARENT_LIMIT = 6;
 export function parentLimitFor(resolved) {
   const { maxzoom } = stackCoverage(resolved);
   const shallowest = resolved.sources
-    .map((source) => source.entry?.pmtiles?.maxZoom)
+    .map((source) =>
+      source.nested
+        ? stackCoverage(source.nested).maxzoom
+        : source.entry?.pmtiles?.maxZoom,
+    )
     .filter((zoom) => Number.isFinite(zoom));
   if (!shallowest.length || !Number.isFinite(maxzoom)) return PARENT_LIMIT;
 
@@ -304,6 +309,10 @@ export function passThroughRead({
   // decoding the tile to find out how wide it is.
   if (size) return null;
 
+  // A nested stack has no stored bytes at all, so there is nothing this could
+  // hand back even when it is the only thing answering.
+  if (reads.some((read) => read.nested)) return null;
+
   const answered = reads.filter((read) => read.found);
   // More than one and they have to be painted; none and there is nothing to
   // send. An error on any source is a decision this must not take, because the
@@ -362,6 +371,13 @@ async function readAll({ resolved, z, x, y, tiles, signal, clips }) {
   const limit = parentLimitFor(resolved);
   return Promise.all(
     resolved.sources.map(async (source, index) => {
+      // A stack in place of an archive. Nothing to fetch: it is evaluated
+      // rather than read, and `gather` is where that happens -- but the clip
+      // still decides whether it is worth evaluating at all.
+      if (source.nested) {
+        if (clips?.[index]?.where === OUTSIDE) return { source, found: null };
+        return { source, nested: true };
+      }
       if (!source.entry) return { source, found: null };
       // Nothing of this source is in the shape, so there is nothing to fetch.
       // Decided before the read, which is where the saving is: no swarm round
@@ -394,10 +410,63 @@ async function readAll({ resolved, z, x, y, tiles, signal, clips }) {
  * @param {object} options - The reads and what to decode them with.
  * @returns {Promise<object>} - `{contributors, contributions}` or `{error}`.
  */
-async function gather({ reads, z, x, y, tiles, codec, signal, size, rgba }) {
+async function gather({
+  reads,
+  z,
+  x,
+  y,
+  tiles,
+  codec,
+  signal,
+  size,
+  rgba,
+  cutlines,
+}) {
   const contributors = [];
   const contributions = [];
   for (const read of reads) {
+    if (read.nested) {
+      // Evaluated rather than read, and handed on as heights: encoding it here
+      // only for the merge above to decode it again would cost two conversions
+      // and lose whatever the inner encoding could not hold.
+      const inner = await stackHeights({
+        resolved: read.source.nested,
+        z,
+        x,
+        y,
+        tiles,
+        codec,
+        signal,
+        size,
+        cutlines,
+      });
+      if (inner?.error) {
+        if (read.source.required) {
+          return {
+            contributors,
+            error: {
+              status: inner.error.status,
+              message: `${read.source.name} is required: ${inner.error.message}`,
+            },
+          };
+        }
+        contributors.push(`${read.source.name}=error`);
+        contributions.push(null);
+        continue;
+      }
+      if (!inner) {
+        contributors.push(`${read.source.name}=absent`);
+        contributions.push(null);
+        continue;
+      }
+      contributors.push(`${read.source.name}=stack(${inner.contributors})`);
+      contributions.push({
+        source: read.source.source,
+        heights: inner.heights,
+        width: inner.width,
+      });
+      continue;
+    }
     if (read.error) {
       if (read.source.required) {
         return {
@@ -577,6 +646,98 @@ async function readMaskEdges({
   );
 }
 
+/**
+ * Turns each clip into the weight its source is painted with.
+ *
+ * Rasterised at the grid the layers are painted on, and only for the tiles the
+ * edge actually crosses -- everything else was settled by `classifyTile`
+ * without touching a pixel.
+ * @param {object} options - The contributions, the clips and the grid.
+ * @returns {void}
+ */
+function applyClips({ contributions, clips, z, x, y, grid }) {
+  for (const [index, contribution] of contributions.entries()) {
+    const clip = clips?.[index];
+    if (!contribution || !clip || clip.where !== PARTIAL) continue;
+    // Rasterised with a border where the edge is feathered, because the ramp
+    // is measured from the boundary and a boundary just outside the tile still
+    // decides what the pixels inside it weigh. The shape is known in full, so
+    // this costs the extra rows and nothing else -- unlike a mask read out of
+    // a source's own pixels, which would need its neighbours.
+    //
+    // Asked again rather than taken from the clip: `clipsFor` may have been
+    // given a grid the merge did not end up using, and a fade in metres is a
+    // different number of pixels on each of them.
+    const margin = featherFor(contribution.source, { z, y, size: grid });
+    const mask = rasterizeTile(clip.shape, z, x, y, grid, margin);
+    contribution.coverage = cropMask(
+      featherMask(mask, grid + margin * 2, margin),
+      grid,
+      margin,
+    );
+  }
+}
+
+/**
+ * One tile of a stack, as heights, for a stack that is using it as a source.
+ *
+ * The same reading and merging a served tile goes through, stopping before the
+ * encode. A nested stack is handed to the merge above it as the numbers it
+ * produced: encoding here only for that merge to decode it again would cost
+ * two conversions per tile and lose whatever the inner encoding could not
+ * hold. See docs/tile-stacks.md -- "A stack as a source".
+ * @param {object} options - The resolved inner stack and what to read with.
+ * @returns {Promise<object|null>} - `{heights, width, contributors}`, `{error}`,
+ *   or null where nothing covered this tile.
+ */
+export async function stackHeights({
+  resolved,
+  z,
+  x,
+  y,
+  tiles,
+  codec,
+  signal,
+  size,
+  cutlines,
+}) {
+  const clips = clipsFor(resolved, cutlines, z, x, y, size);
+  const reads = await readAll({ resolved, z, x, y, tiles, signal, clips });
+  const gathered = await gather({
+    reads,
+    z,
+    x,
+    y,
+    tiles,
+    codec,
+    signal,
+    size,
+    rgba: false,
+    cutlines,
+  });
+  if (gathered.error) return { error: gathered.error };
+
+  const present = gathered.contributions.filter(Boolean);
+  if (present.length === 0) return null;
+
+  const grid =
+    size ?? Math.max(...present.map((c) => c.width ?? c.raster.width));
+  applyClips({ contributions: gathered.contributions, clips, z, x, y, grid });
+
+  const heights = mergeElevation(gathered.contributions, {
+    z,
+    x,
+    y,
+    size: grid,
+    gaussianBlurSigma: resolved.stack.gaussianBlurSigma,
+    resampling: resolved.stack.resampling,
+  });
+  // Nodata is deliberately not filled in. A hole is what lets the stack above
+  // show through, and a slab of -10000 is ground rather than a hole.
+  if (!heights) return null;
+  return { heights, width: grid, contributors: gathered.contributors.join() };
+}
+
 /**
  * Merges what the sources gave into one encoded tile.
  * @param {object} options - Everything the merge needs.
@@ -603,32 +764,12 @@ async function merge({
   // source brought, so the finest one is not thrown away.
   const grid =
     size ??
-    Math.max(...contributions.filter(Boolean).map((c) => c.raster.width));
+    Math.max(
+      ...contributions.filter(Boolean).map((c) => c.width ?? c.raster.width),
+    );
   const output = resolved.stack.output ?? {};
 
-  // Rasterised at the grid the layers are painted on, and only for the tiles
-  // the edge actually crosses. Everything else was settled by `classifyTile`
-  // without touching a pixel.
-  for (const [index, contribution] of contributions.entries()) {
-    const clip = clips?.[index];
-    if (!contribution || !clip || clip.where !== 'partial') continue;
-    // Rasterised with a border where the edge is feathered, because the ramp
-    // is measured from the boundary and a boundary just outside the tile still
-    // decides what the pixels inside it weigh. The shape is known in full, so
-    // this costs the extra rows and nothing else -- unlike a mask read out of
-    // a source's own pixels, which would need its neighbours.
-    //
-    // Asked again rather than taken from the clip: `clipsFor` may have been
-    // given a grid the merge did not end up using, and a fade in metres is a
-    // different number of pixels on each of them.
-    const margin = featherFor(contribution.source, { z, y, size: grid });
-    const mask = rasterizeTile(clip.shape, z, x, y, grid, margin);
-    contribution.coverage = cropMask(
-      featherMask(mask, grid + margin * 2, margin),
-      grid,
-      margin,
-    );
-  }
+  applyClips({ contributions, clips, z, x, y, grid });
 
   const merging = {
     z,
@@ -816,6 +957,7 @@ export async function answerStackTile(options) {
       signal,
       size,
       rgba,
+      cutlines,
     });
     if (gathered.error) {
       return {

package/src/stacks.js

@@ -74,6 +74,35 @@ const SPACES = new Set(['elevation', 'rgba']);
 /** An id has to survive being a path segment without being escaped. */
 const ID = /^[a-z0-9][a-z0-9._-]*$/i;
 
+/**
+ * How many stacks deep a recipe may reach.
+ *
+ * A loop is caught by name and refused outright; this is for the chain that
+ * does not loop but is still nobody's intention -- every level is a full merge
+ * of everything under it, so the cost of a tile multiplies rather than adds.
+ */
+const MAX_STACK_DEPTH = 4;
+
+/**
+ * Fields that describe stored bytes, which a nested stack does not have.
+ *
+ * It is merged as heights -- evaluated and handed straight to the merge above
+ * it, with no encode and decode in between -- so an encoding is a question
+ * about nothing, and `maskColors` compares channels as they were stored where
+ * nothing was ever stored. Everything else a source may say still applies:
+ * masks by height, the fade, a cutline, an adjustment, opacity and blend.
+ */
+const NOT_ON_A_NESTED_SOURCE = Object.freeze([
+  'encoding',
+  'baseVal',
+  'interval',
+  'redFactor',
+  'greenFactor',
+  'blueFactor',
+  'baseShift',
+  'maskColors',
+]);
+
 /**
  * Checks a stack definition, returning what is wrong with it.
  *
@@ -99,11 +128,37 @@ export function validateStack(stack) {
   }
 
   stack.sources.forEach((source, index) => {
-    const named = [source?.category, source?.archive].filter(Boolean).length;
+    const named = [source?.category, source?.archive, source?.stack].filter(
+      Boolean,
+    ).length;
     // Both is worse than neither: it looks deliberate, and whichever one the
     // implementation happened to prefer would be a silent choice.
     if (named !== 1) {
-      problems.push(`sources[${index}] needs exactly one of category, archive`);
+      problems.push(
+        `sources[${index}] needs exactly one of category, archive, stack`,
+      );
+    }
+    if (source?.stack) {
+      if (typeof source.stack !== 'string' || !ID.test(source.stack)) {
+        problems.push(`sources[${index}].stack must be the id of a stack`);
+      }
+      if (source.stack === stack.id) {
+        // Caught by the resolver as well, which has to handle a loop through
+        // three stacks anyway. Named here because a recipe naming itself is a
+        // mistake worth reading in the editor rather than in the tile.
+        problems.push(`sources[${index}].stack is this stack`);
+      }
+      // A nested stack is merged as heights: it never becomes bytes, so there
+      // is nothing for these to describe. Refused rather than ignored, since a
+      // field that quietly does nothing is the failure this is most prone to.
+      for (const key of NOT_ON_A_NESTED_SOURCE) {
+        if (source[key] !== undefined) {
+          problems.push(
+            `sources[${index}].${key} does not apply to a stack: it is ` +
+              'merged as heights and never stored as pixels',
+          );
+        }
+      }
     }
     if (source?.opacity !== undefined) {
       const value = Number(source.opacity);
@@ -291,6 +346,9 @@ export function validateStack(stack) {
  */
 export function needsCodec(stack) {
   for (const [index, source] of (stack.sources ?? []).entries()) {
+    // Always: a nested stack is evaluated rather than read, and evaluating it
+    // means decoding everything underneath.
+    if (source.stack) return `sources[${index}].stack`;
     if (source.maskValues?.length) return `sources[${index}].maskValues`;
     if (source.maskColors?.length) return `sources[${index}].maskColors`;
     if (source.maskRange?.length) return `sources[${index}].maskRange`;
@@ -346,27 +404,59 @@ export function stackRevision(stack) {
  * resolves depends on who is asking — the same rule `/latest/<category>/`
  * applies, which takes the request's credential into account.
  * @param {Stack} stack - The definition.
- * @param {object} resolvers - `archive(hash)` and `category(name)` lookups.
+ * @param {object} resolvers - `archive(hash)` and `category(name)` lookups,
+ *   and `stack(id)` returning a recipe for a source that names one.
+ * @param {string[]} [chain] - The stack ids already being resolved, so a
+ *   recipe naming one of them stops rather than looping.
  * @returns {object} - The stack, its sources bottom-first, and any failures.
  */
-export function resolveStack(stack, resolvers) {
+export function resolveStack(stack, resolvers, chain = []) {
+  const here = [...chain, stack.id];
   const sources = (stack.sources ?? []).map((source, index) => {
-    const entry = source.category
-      ? resolvers.category(source.category)
-      : resolvers.archive(source.archive);
-    return {
+    const common = {
       index,
       source,
-      entry: entry ?? null,
       // The bottom-most source is the base, and a stack without its base is
       // holes rather than a map. Above it, absence is survivable.
       required: source.required ?? index === 0,
+    };
+
+    if (source.stack) {
+      // A stack in place of an archive. Resolved here rather than at the tile,
+      // so a recipe that cannot work says so once instead of at every request.
+      const inner = resolvers.stack?.(source.stack) ?? null;
+      const looping = Boolean(inner) && here.includes(inner.id);
+      const deep = here.length >= MAX_STACK_DEPTH;
+      return {
+        ...common,
+        entry: null,
+        name: source.stack,
+        // A stack of categories moves when they are rebuilt, exactly as one of
+        // them would on its own.
+        pinned: false,
+        nested:
+          inner && !looping && !deep
+            ? resolveStack(inner, resolvers, here)
+            : null,
+        // Told apart, because they call for different things: a loop is a
+        // recipe to fix, a name that resolves to nothing is a stack to add.
+        looping,
+        deep,
+      };
+    }
+
+    const entry = source.category
+      ? resolvers.category(source.category)
+      : resolvers.archive(source.archive);
+    return {
+      ...common,
+      entry: entry ?? null,
       name: source.category ?? source.archive,
       pinned: Boolean(source.archive),
     };
   });
 
-  const missing = sources.filter((s) => !s.entry && s.required);
+  const missing = sources.filter((s) => s.required && !s.entry && !s.nested);
   return { stack, sources, missing };
 }
 
@@ -440,7 +530,21 @@ export function stacksUsing(stacks, entry, categoryInfo) {
  */
 export function stackCoverage(resolved) {
   const present = resolved.sources.filter((s) => s.entry?.pmtiles);
-  const summaries = present.map((s) => s.entry.pmtiles);
+  // A nested stack answers for the ground its own sources cover, so it stands
+  // in for one here -- worked out the same way, one level down.
+  const nested = resolved.sources
+    .filter((s) => s.nested)
+    .map((s) => stackCoverage(s.nested));
+  const summaries = [
+    ...present.map((s) => s.entry.pmtiles),
+    ...nested.map((cover) => ({
+      minZoom: cover.minzoom,
+      maxZoom: cover.maxzoom,
+      bounds: cover.bounds,
+      format: cover.format,
+      attribution: cover.attribution,
+    })),
+  ];
 
   const minzoom =
     resolved.stack.minzoom ??
@@ -453,8 +557,12 @@ export function stackCoverage(resolved) {
 
   let bounds = resolved.stack.bounds;
   if (!bounds && resolved.stack.boundsSource !== undefined) {
-    bounds = present.find((s) => s.index === resolved.stack.boundsSource)?.entry
-      ?.pmtiles?.bounds;
+    const at = resolved.sources.find(
+      (s) => s.index === resolved.stack.boundsSource,
+    );
+    bounds =
+      at?.entry?.pmtiles?.bounds ??
+      (at?.nested ? stackCoverage(at.nested).bounds : undefined);
   }
   if (!bounds && summaries.length) {
     const boxes = summaries.map((s) => s.bounds).filter(Array.isArray);
@@ -478,10 +586,13 @@ export function stackCoverage(resolved) {
   // A stack is a derived work of every source in it, and attribution is the
   // thing that reliably goes missing when tiles are combined. Joined rather
   // than dropped when the recipe does not state one.
+  // Pipes, not commas. These are almost always HTML links, and a comma
+  // between two anchors reads as part of the last one's text -- which is how
+  // MapLibre, Mapbox and OpenLayers all write a multi-source attribution.
   const attribution =
     resolved.stack.attribution ??
     [...new Set(summaries.map((s) => s.attribution).filter(Boolean))].join(
-      ', ',
+      ' | ',
     ) ??
     undefined;
 
@@ -512,7 +623,12 @@ export function stackEtag(resolved, z, x, y) {
   const parts = [
     resolved.stack.id,
     stackRevision(resolved.stack),
-    ...resolved.sources.map((s) => s.entry?.infoHash ?? '-'),
+    // A nested stack contributes its own revision and its own sources, or an
+    // edit one level down would serve from a cache the outer stack thinks is
+    // still good.
+    ...resolved.sources.map((s) =>
+      s.nested ? stackEtag(s.nested, z, x, y) : (s.entry?.infoHash ?? '-'),
+    ),
     z,
     x,
     y,
@@ -531,7 +647,9 @@ export function stackEtag(resolved, z, x, y) {
  * @returns {boolean} - True when nothing here can move.
  */
 export function isPinned(resolved) {
-  return resolved.sources.every((s) => s.pinned);
+  return resolved.sources.every((s) =>
+    s.nested ? isPinned(s.nested) : s.pinned,
+  );
 }
 
 /**

package/src/web/index.html

@@ -686,6 +686,20 @@
           <div class="sub" id="bake-filename"></div>
         </div>
 
+        <div class="field">
+          <label for="bake-attribution">Attribution</label>
+          <input id="bake-attribution" placeholder="" />
+          <div class="sub">
+            Written into the archive's own metadata, so the credit travels with
+            the file rather than living in whatever style happened to load it.
+            Filled in from every source's attribution joined with
+            <code>|</code>, since a stack is a derived work of all of them and
+            this is the thing that reliably goes missing when tiles are
+            combined. The stack's own <code>attribution</code> wins where it
+            has one.
+          </div>
+        </div>
+
         <div class="field">
           <label for="bake-description">Description</label>
           <input id="bake-description" placeholder="" />
@@ -7532,7 +7546,7 @@ Every piece is hashed against the ` +
         const select = $('stack-add-source');
         const taken = new Set(
           (stackDraft?.sources ?? []).flatMap((source) =>
-            [source.category, source.archive].filter(Boolean),
+            [source.category, source.archive, source.stack].filter(Boolean),
           ),
         );
         const groups = [];
@@ -7617,6 +7631,38 @@ Every piece is hashed against the ` +
           );
         }
 
+        try {
+          // A stack in place of an archive. The reason to want it is that a
+          // base worked out once -- terrain over bathymetry, masked and faded
+          // -- is a thing to reuse rather than retype, and a recipe that names
+          // it follows every later correction to it.
+          const { stacks: held } = await api('/api/stacks');
+          const free = (held ?? []).filter(
+            (one) =>
+              !taken.has(one.id) &&
+              one.id !== $('stack-id').value.trim() &&
+              one.space === (stackDraft?.space ?? 'elevation'),
+          );
+          if (free.length) {
+            groups.push(
+              `<optgroup label="Stacks — a recipe used as one layer">${free
+                .map((one) => {
+                  const sources = one.sources?.length ?? 0;
+                  return `<option value="stk:${escapeHtml(one.id)}">${escapeHtml(
+                    one.title ?? one.id,
+                  )} — ${sources} source${sources === 1 ? '' : 's'}</option>`;
+                })
+                .join('')}</optgroup>`,
+            );
+          }
+        } catch (error) {
+          groups.push(
+            `<optgroup label="Stacks — could not load: ${escapeHtml(
+              error.message,
+            )}"></optgroup>`,
+          );
+        }
+
         select.innerHTML = groups.length
           ? groups.join('')
           : '<option value="">nothing left to add</option>';
@@ -7696,6 +7742,11 @@ Every piece is hashed against the ` +
                 ? '<span class="pill">wins</span>'
                 : '';
 
+        // A nested stack is merged as heights: it was never stored, so there
+        // is no encoding to read it with and no channels to compare a colour
+        // against. Those fields are left out rather than shown and refused on
+        // save; everything that acts on heights stays.
+        const nested = Boolean(source.stack);
         const perSpace = rgba
           ? `<label class="choice">Opacity
                <input type="number" min="0" max="1" step="0.05" style="width:5rem"
@@ -7714,7 +7765,14 @@ Every piece is hashed against the ` +
                    .join('')}
                </select>
              </label>`
-          : `<label class="choice">Encoding
+          : `${
+               nested
+                 ? `<div class="sub" style="max-width:34rem">Merged as heights:
+                      <code>${escapeHtml(source.stack)}</code> is evaluated for
+                      this tile and handed straight to this merge, so there is
+                      no encoding to read it with. Its own output settings
+                      apply where it is served on its own.</div>`
+                 : `<label class="choice">Encoding
                <select data-stack-field="encoding" data-stack-index="${index}">
                  <option value="mapbox"${
                    (source.encoding ?? 'mapbox') === 'mapbox' ? ' selected' : ''
@@ -7768,6 +7826,7 @@ Every piece is hashed against the ` +
                       <code>6553.6, 25.6, 0.1, 10000</code> and terrarium is
                       <code>256, 1, 0.00390625, 32768</code>.</div>`
                  : ''
+             }`
              }
              <label class="choice">Shift by
                <input type="number" step="any" style="width:6rem"
@@ -7804,13 +7863,17 @@ Every piece is hashed against the ` +
             <div class="row">
               <span class="muted">${index}</span>
               <code>${escapeHtml(
-                source.category ?? shortHash(source.archive) ?? '',
+                source.stack ?? source.category ?? shortHash(source.archive) ?? '',
               )}</code>
               <span class="sub" title="${
                 source.archive
                   ? 'Pinned to this build. It will not follow a rebuild, and breaks if the archive is removed.'
-                  : 'Follows whichever build in this category is newest.'
-              }">${source.archive ? 'pinned' : 'category'}</span>
+                  : source.stack
+                    ? 'Another stack, merged as one layer. It follows every later change to that recipe, and to whatever it resolves to.'
+                    : 'Follows whichever build in this category is newest.'
+              }">${
+                source.archive ? 'pinned' : source.stack ? 'stack' : 'category'
+              }</span>
               ${role}
               <span class="bar-gap"></span>
               <button type="button" data-stack-move="${index}" data-stack-dir="-1"
@@ -7821,11 +7884,15 @@ Every piece is hashed against the ` +
             </div>
             <div class="bar">
               ${perSpace}
-              <label class="choice">Mask colours
-                <input style="width:10rem" placeholder="#000000"
-                       data-stack-field="maskColors" data-stack-index="${index}"
-                       value="${escapeHtml((source.maskColors ?? []).join(', '))}" />
-              </label>
+              ${
+                nested
+                  ? ''
+                  : `<label class="choice">Mask colours
+                       <input style="width:10rem" placeholder="#000000"
+                              data-stack-field="maskColors" data-stack-index="${index}"
+                              value="${escapeHtml((source.maskColors ?? []).join(', '))}" />
+                     </label>`
+              }
               <label class="choice"
                      title="Clip this source to a shape, so only what is inside it contributes. Most archives are already clipped when they are built — this is for one you did not build and cannot rebuild.">
                 Clip to
@@ -8032,7 +8099,11 @@ Every piece is hashed against the ` +
         // Appended, so the newest source is the one that covers the rest --
         // which is what somebody adding a more detailed layer means.
         stackDraft.sources.push(
-          kind === 'arc' ? { archive: name } : { category: name },
+          kind === 'arc'
+            ? { archive: name }
+            : kind === 'stk'
+              ? { stack: name }
+              : { category: name },
         );
         renderStackDraft();
         fillStackSourceChoices();
@@ -8177,6 +8248,10 @@ Every piece is hashed against the ` +
         // how tiles are combined; an archive describes what it is, and only
         // the person exporting it knows that.
         $('bake-description').value = '';
+        // Unlike the description, this is filled in: it is not a thing only
+        // the person exporting knows, it is a fact about the sources, and an
+        // archive published without it is the failure worth preventing.
+        $('bake-attribution').value = stack?.attribution ?? '';
         $('bake-categories').value = (stack?.categories ?? []).join(', ');
         $('bake-output').textContent = bakeOutputSummary(stack);
         showBakeFilename();
@@ -8268,6 +8343,7 @@ Every piece is hashed against the ` +
         const name = $('bake-name').value.trim();
         const filename = $('bake-file').value.trim();
         const description = $('bake-description').value.trim();
+        const attribution = $('bake-attribution').value.trim();
 
         try {
           await api(`/api/stacks/${encodeURIComponent(id)}/bake`, {
@@ -8278,6 +8354,7 @@ Every piece is hashed against the ` +
               ...(name ? { name } : {}),
               ...(filename ? { filename } : {}),
               ...(description ? { description } : {}),
+              ...(attribution ? { attribution } : {}),
             },
           });
         } catch (error) {