pmtiles-swarm 0.98.1 → 0.98.3

package/CHANGELOG.md

@@ -7,6 +7,37 @@
 ### 🐞 Bug fixes
 - _...Add new stuff here..._
 
+## 0.98.3
+### ✨ Features and improvements
+
+### 🐞 Bug fixes
+- **A stack could not be pointed at a completed MBTiles archive.** The source picker offered only
+  PMTiles, so an MBTiles the node was perfectly able to serve tiles from was simply absent from the
+  menu. It offers a complete one now, by the same rule the tile route enforces — and still hides one
+  that is arriving, since a stack naming that would answer 503 for every tile until the download
+  finished.
+- **A planet dump was offered as a stack source.** The same filter read an archive with no recorded
+  kind as PMTiles, which is not a harmless default: a mirrored `.osm.pbf` appeared in the menu, and
+  naming it would have produced a recipe that failed at its first tile. The kind now falls back to
+  the filename before anything is assumed, which is how `prewarm.js` resolves it after the identical
+  mistake sent every mirrored dump round the header-read backoff for ever. Such an archive is also
+  labelled `unknown · not servable` in the list, where it previously said nothing at all.
+
+## 0.98.2
+### ✨ Features and improvements
+
+### 🐞 Bug fixes
+- **The console called a complete MBTiles archive "not servable".** It is, and has been since the
+  tile store learned to open one: PMTiles always, MBTiles from a complete local copy. The console
+  never followed. It tested `kind === 'pmtiles'` in two places, so an MBTiles that had finished
+  hashing was labelled `mbtiles · not servable` in the list and had its TileJSON, preview and tile
+  endpoint hidden in the panel — all of which the node was answering perfectly well.
+
+  One rule now, matching what the tile route actually enforces, and an archive still arriving reads
+  as `servable once complete` rather than as one that never will be. Warming stays PMTiles-only: it
+  fetches the pieces a region's tiles live in, and an MBTiles is read whole or not at all, so there
+  is never a region of one to warm.
+
 ## 0.98.1
 ### ✨ Features and improvements
 - **The two archive readers no longer import each other.** Summarising an MBTiles archive reached

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "pmtiles-swarm",
-  "version": "0.98.1",
+  "version": "0.98.3",
   "description": "BitTorrent distribution for PMTiles map archives: create torrents, watch folders, publish and subscribe to RSS feeds, and seed through qBittorrent or an embedded client",
   "type": "module",
   "main": "src/index.js",

package/src/web/index.html

@@ -1823,6 +1823,63 @@
         }
       };
 
+      /**
+       * Whether this archive has a tile endpoint at all.
+       *
+       * The same rule the tile route applies, rather than a second opinion
+       * about it: PMTiles always, MBTiles once the file is whole. MBTiles is
+       * SQLite, so it cannot be read a byte range at a time out of the swarm —
+       * but a complete one on local disk is an ordinary database holding the
+       * same tiles, and refusing to serve it is refusing something that works.
+       * See docs/internals.md — "Serving an MBTiles archive".
+       * @param {object} entry - A catalog entry.
+       * @returns {boolean} - Whether to offer the tile endpoints.
+       */
+      const isServable = (entry) => {
+        const kind = kindOf(entry);
+        if (kind === 'pmtiles') return true;
+        return kind === 'mbtiles' && entry.complete === true;
+      };
+
+      /**
+       * What kind of archive this is, from the record or failing that the name.
+       *
+       * An entry from before the kind was recorded carries none, and reading
+       * that as PMTiles is not a harmless default: it is how a planet dump ends
+       * up labelled servable and offered as a stack source, where naming it
+       * would produce a recipe that fails at its first tile. `prewarm.js`
+       * resolves it exactly this way, after the same mistake sent every
+       * mirrored .osm.pbf round the header-read backoff for ever.
+       * @param {object} entry - A catalog entry.
+       * @returns {string} - 'pmtiles', 'mbtiles', or 'unknown'.
+       */
+      const kindOf = (entry) => {
+        if (entry.kind) return entry.kind;
+        const name = entry.name ?? '';
+        if (/\.pmtiles$/i.test(name)) return 'pmtiles';
+        if (/\.mbtiles$/i.test(name)) return 'mbtiles';
+        return 'unknown';
+      };
+
+      /**
+       * What to say under the size about an archive that is not PMTiles.
+       *
+       * An MBTiles that is here in full is servable, and saying "not servable"
+       * beside a finished file was simply wrong. One that is still arriving is
+       * not servable *yet*, which is a different thing to tell somebody: it
+       * becomes one by waiting, where an .osm.pbf never will.
+       * @param {object} entry - A catalog entry.
+       * @returns {string} - The note, or empty for a PMTiles archive.
+       */
+      const kindNote = (entry) => {
+        const kind = kindOf(entry);
+        if (kind === 'pmtiles') return '';
+        if (kind !== 'mbtiles') return `${escapeHtml(kind)} · not servable`;
+        return entry.complete === true
+          ? escapeHtml(kind)
+          : `${escapeHtml(kind)} · servable once complete`;
+      };
+
       /**
        * Closes the details, panel and row together.
        *
@@ -1891,8 +1948,8 @@
                 .join(' · ')}</div>
             </td>
             <td>${bytes(entry.size)}${
-              (entry.kind ?? 'pmtiles') !== 'pmtiles'
-                ? `<div class="sub">${escapeHtml(entry.kind)} · not servable</div>`
+              kindNote(entry)
+                ? `<div class="sub">${kindNote(entry)}</div>`
                 : ''
             }</td>
             <td>${added(entry.createdAt)}</td>
@@ -2518,10 +2575,15 @@
 
         const base = location.origin;
         const tileJson = `${base}/archives/${entry.infoHash}/tiles.json`;
-        // Only PMTiles can be a tile endpoint. Offering the URL for anything
-        // else hands out a link that will never work and reads as a fault in
-        // the server rather than a fact about the archive.
-        const servable = (entry.kind ?? 'pmtiles') === 'pmtiles';
+        // Offering the URL for something that cannot serve tiles hands out a
+        // link that will never work and reads as a fault in the server rather
+        // than a fact about the archive.
+        const servable = isServable(entry);
+        // Warming pulls the pieces a region's tiles live in, so it only means
+        // anything for an archive read out of the swarm. An MBTiles is read
+        // whole or not at all — once it is here there is nothing left to
+        // fetch, and before that there is no tile to warm.
+        const warmable = kindOf(entry) === 'pmtiles';
         const torrentUrl = `${base}/archives/${entry.infoHash}/archive.torrent`;
         const summary = entry.pmtiles ?? {};
         const mode = entry.mode ?? 'mirror';
@@ -2606,11 +2668,18 @@
           ${
             servable
               ? `<div class="sub" style="margin-bottom:1rem"><code>${escapeHtml(tileJson)}</code></div>`
-              : `<div class="sub" style="margin-bottom:1rem">
-                   This is ${escapeHtml(entry.kind ?? 'not a PMTiles')} archive, so it has no tile
-                   endpoint — only PMTiles can be read a byte range at a time, which is what
-                   serving tiles from a swarm needs. It is still distributed and seeded normally.
-                 </div>`
+              : kindOf(entry) === 'mbtiles'
+                ? `<div class="sub" style="margin-bottom:1rem">
+                     This MBTiles archive is still arriving, so it has no tile endpoint yet. It is
+                     SQLite, which cannot be read a byte range at a time out of the swarm the way
+                     PMTiles can — so it becomes servable when the download finishes rather than as
+                     pieces land. It is distributed and seeded normally in the meantime.
+                   </div>`
+                : `<div class="sub" style="margin-bottom:1rem">
+                     This is ${escapeHtml(entry.kind ?? 'not a PMTiles')} archive, so it has no tile
+                     endpoint — only PMTiles and MBTiles can be served as tiles. It is still
+                     distributed and seeded normally.
+                   </div>`
           }
 
           <h2>Completion hook</h2>
@@ -2665,7 +2734,7 @@
 
           <h2>Actions</h2>
           <div class="actions">
-            ${servable ? '<button id="warm">Warm region</button>' : ''}
+            ${warmable ? '<button id="warm">Warm region</button>' : ''}
             <button id="set-location">Set location…</button>
             <button id="clear-cache" ${mode === 'cache' ? '' : 'disabled title="only cache-mode archives have a cache to clear"'}>Clear cache</button>
             <button id="recheck" title="Hash the files on disk again and believe the result. For an archive that reads 0% next to a file that is plainly there, or one that claims to be complete and is not — every other figure comes from something written down earlier, and this is the only thing that goes and looks.">Recheck files</button>
@@ -2895,7 +2964,7 @@
         }
         $('copy-hash').onclick = () => copy(entry.infoHash, 'Infohash');
 
-        if (servable) {
+        if (warmable) {
           $('warm').onclick = () => openWarmDialog(entry);
         }
 
@@ -8126,10 +8195,12 @@ Every piece is hashed against the ` +
           // missing from a menu the operator is looking at. This is the same
           // endpoint the Archives view reads, and it answers a bare array.
           const held = await api('/api/torrents');
+          // The same rule the tile route enforces, because a stack source is
+          // read through exactly that path: a complete MBTiles belongs here,
+          // and one still arriving does not -- a stack naming it would answer
+          // 503 for every tile until the download finished.
           const free = (held ?? []).filter(
-            (entry) =>
-              !taken.has(entry.infoHash) &&
-              (entry.kind ?? 'pmtiles') === 'pmtiles',
+            (entry) => !taken.has(entry.infoHash) && isServable(entry),
           );
           if (free.length) {
             groups.push(