pmtiles-swarm 0.68.2 → 0.68.3

package/CHANGELOG.md

@@ -7,6 +7,25 @@
 ### 🐞 Bug fixes
 - _...Add new stuff here..._
 
+## 0.68.3
+### 🐞 Bug fixes
+- **Exporting a stack failed on its first merged tile.** "Cannot transfer object of unsupported
+  type", and the export stopped — reported on the stack, with its checkpoint kept, but stopped.
+
+  The pixel worker hands a decoded tile over rather than copying it, because a raster is most of a
+  megabyte per source and copying each one is work on the very thread the worker exists to keep
+  free. It only did that where the buffer looked like it owned its memory: byte offset zero, and a
+  backing buffer exactly its own length.
+
+  A tile decoded by `sharp` passes that check and cannot be transferred anyway. Its memory comes
+  from libvips, so the backing store is externally allocated and Node refuses it — while looking,
+  by every property there is to look at, exactly like one it would accept. There is nothing to
+  test for, so the copy is now unconditional.
+
+  The tests never caught it because every one of them built its rasters with `Buffer.alloc`, whose
+  memory Node owns and will happily hand over. There is a test now that decodes a tile through the
+  codec first, and it fails against the old code.
+
 ## 0.68.2
 ### 🐞 Bug fixes
 - **Archives were listed under a heading that said Stacks.** The catalogue page's Archives

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "pmtiles-swarm",
-  "version": "0.68.2",
+  "version": "0.68.3",
   "description": "BitTorrent distribution for PMTiles map archives: create torrents, watch folders, publish and subscribe to RSS feeds, and seed through qBittorrent or an embedded client",
   "type": "module",
   "main": "src/index.js",

package/src/pixels.js

@@ -75,28 +75,25 @@ export class PixelWorker {
     const id = this.#next;
     this.#next += 1;
 
-    // Rasters are handed over rather than cloned, and where possible without
-    // being copied first either. A decoded tile is most of a megabyte per
-    // source, and copying every one of them is work on the very thread this
-    // exists to keep free -- enough of it to cost more than it saves.
+    // Copied into a buffer this thread allocated, then transferred.
     //
-    // Copied only when the buffer does not own its memory. Node pools
-    // allocations under 4 KiB, and transferring a pooled buffer would hand
-    // over the whole pool with everything else in it; a raster is far larger
-    // than that and so is always its own allocation, but the check is what
-    // makes that a fact rather than an assumption.
+    // An earlier version handed the raster over as it stood where it looked
+    // like it owned its memory -- byteOffset zero and a buffer exactly its own
+    // length -- to save copying most of a megabyte per source. A decoded tile
+    // passes that check and cannot be transferred anyway: `sharp` gets its
+    // memory from libvips, so the backing store is externally allocated, and
+    // Node refuses it with "Cannot transfer object of unsupported type". There
+    // is no property that tells the two apart, so there is nothing to check
+    // for -- the copy is the only thing that is always right.
     //
-    // Either way the caller gives up these rasters. Nothing reads them after a
-    // merge, which is what makes that safe.
+    // The caller gives up these rasters either way. Nothing reads a
+    // contribution after its merge, which is what makes that safe.
     const transfers = [];
-    const owned = (bytes) =>
-      bytes.byteOffset === 0 && bytes.buffer.byteLength === bytes.byteLength;
     const contributions = job.contributions.map((contribution) => {
       if (!contribution?.raster?.data) return contribution;
       const source = contribution.raster.data;
-      const bytes = owned(source)
-        ? source
-        : Uint8Array.prototype.slice.call(source);
+      const bytes = new Uint8Array(source.byteLength);
+      bytes.set(source);
       transfers.push(bytes.buffer);
       return {
         ...contribution,