format-png 0.2.0 → 0.3.0

package/CHANGELOG.md

@@ -1,5 +1,93 @@
 # Changelog
 
+## 0.4.0
+
+Worker pools now encode one large image on all their workers, built on the
+format-png crate's new segment API.
+
+### Faster
+
+- A pool, and the async functions, compress a large image's data in 1 MiB
+  segments across all the workers. A 2500x3800 photo-like image at level 6,
+  in Node on a 20-core machine: 7.8 s on one worker, 4.3 s on two, 2.7 s on
+  four, 2.0 s on eight (3.8x). Images with 1 MiB of filtered data or less are
+  encoded whole, as before.
+- Jobs take turns: while a large image is split, other jobs still start as
+  soon as a worker is free, in the order they were submitted.
+
+### Changed
+
+- A pool's `encode` and `encodeRgba8`, and `encodeAsync` and
+  `encodeRgba8Async`, default to `threads: "auto"`. For large images the file
+  is a little larger (about 0.1% for that photo) and its bytes differ from the
+  sync `encode`'s. Pass `threads: "single"` for the sync function's bytes.
+- Cancelling a running job no longer stops its worker if that worker holds
+  an image another job is splitting: the task finishes, and its result is
+  dropped. The job still rejects at once.
+
+### Added
+
+- `threads` encode option: "single" (the default for the sync API) or
+  "auto", as the crate's `Threads`. In WebAssembly, "auto" isn't faster on its
+  own; it gives the same bytes as a pool.
+
+### Other
+
+- Built against the format-png crate 0.2.0 (from 0.1.1).
+- `npm run bench:pool` times a pool splitting one image, at several sizes.
+
+## 0.3.0
+
+An async API that runs format-png in Web Workers, or worker_threads in Node,
+so large images don't block the main thread. The sync API is unchanged.
+
+### Added
+
+- Async versions of the functions: `decodeAsync`, `decodeRgba8Async`,
+  `encodeAsync`, `encodeRgba8Async`, `readChunksAsync`, `readHeaderAsync` and
+  `parseTextAsync`. They need no setup: they share one worker pool, created on
+  first use. `terminateDefaultWorkerPool()` frees it, and
+  `defaultWorkerPool()` returns it.
+- `createWorkerPool(options?)` returns a pool with async versions of `decode`,
+  `decodeRgba8`, `encode`, `encodeRgba8`, `readChunks`, `readHeader` and
+  `parseText`. They take the same arguments and options, resolve to the same
+  results, and reject with the same `PngError`.
+- Pools need no `init()`: each worker loads the embedded module. By default a
+  pool has one worker per CPU core, at most 8.
+- Workers start as jobs need them and stay alive until `terminate()`, each
+  reusing its decoders and encoders. Jobs queue when every worker is busy and
+  start in the order they were submitted.
+- Inputs are copied to the worker by default. With `transfer: true` they're
+  moved instead, which detaches the caller's buffer. Results always come back
+  without a copy.
+- Pass `signal` to cancel a queued or running job.
+- `WorkerPoolError` with `code` "terminated" (rejected by `terminate()`, and
+  by calls after it) or "worker-crashed" (the pool replaces the worker).
+- `createWorker` option, for bundlers that don't bundle
+  `new Worker(new URL(..., import.meta.url))`, such as esbuild and Rollup.
+  Vite and webpack 5 need no setup.
+
+### Faster
+
+The WebAssembly module is now optimized for speed rather than size
+(`opt-level = 3`). Outputs are byte for byte the same. Measured with
+`npm run bench` in Node 22, on 2500x3800 RGBA images:
+
+| | Photo-like | Smooth |
+|---|---|---|
+| `decodeRgba8` | 304 → 195 ms (36% faster) | 137 → 72 ms (48% faster) |
+| Encode, level 6 | 7.3 → 7.1 s (3% faster) | 1.01 → 0.86 s (15% faster) |
+| Encode, level 9 | 16.5 → 16.0 s (3% faster) | 15.6 → 14.9 s (5% faster) |
+| Encode, filter "none" | 1.50 → 1.43 s (5% faster) | unchanged |
+
+The module is 261 kB, up from 229 kB, and the package is about 170 kB packed.
+
+### Other
+
+- A `PngError` now has the underlying WebAssembly error as its `cause`.
+- No new runtime dependencies, and no `SharedArrayBuffer` or WebAssembly
+  threads, so no COOP/COEP headers are needed.
+
 ## 0.2.0
 
 The WebAssembly module is now embedded in the package, and `init()` with no
@@ -21,6 +109,11 @@ fetched: the package makes no network requests.
   `optimizeDeps: { exclude: ["format-png"] }` is no longer needed.
 - If you passed a URL to `init`, call `init()` instead.
 
+### Other
+
+- The WebAssembly module is a third smaller (229 kB, down from 335 kB): it no
+  longer carries symbol names and debug info. Speed is unchanged.
+
 ## 0.1.2
 
 - Built against the `format-png` crate 0.1.1. The API and behavior are unchanged.

package/README.md

@@ -11,7 +11,8 @@ A PNG decoder and encoder for the browser and Node, compiled from Rust to WebAss
 - **Reads every chunk** without decompressing the image: header, palette, transparency, `gAMA`, `cHRM`, `sRGB`, `iCCP`, `cICP`, `pHYs`, `tIME`, `eXIf`, `tEXt`, `zTXt`, `iTXt`, plus private and unknown chunks raw.
 - **Writes metadata** back: color space, ICC profiles, Exif, text and your own chunks.
 - **Makes files smaller:** converts images with up to 256 colors to a palette, and strips the chunks you don't need, without changing a pixel.
-- **No dependencies.** About 130 kB packed, mostly the WebAssembly module. Ships with TypeScript types.
+- **Runs off the main thread** with a built-in worker pool, so large images don't freeze the page.
+- **No dependencies.** About 170 kB packed, mostly the WebAssembly module. Ships with TypeScript types.
 
 ## Install
 
@@ -52,7 +53,7 @@ canvas.getContext("2d").putImageData(toImageData(image), 0, 0);
 import { encodeRgba8 } from "format-png";
 
 const imageData = context.getImageData(0, 0, width, height);
-const png = encodeRgba8(imageData, { compression: 9 });
+const png = encodeRgba8(imageData, { compression: 6 }); // 6 is the default
 
 const blob = new Blob([png], { type: "image/png" });
 ```
@@ -95,7 +96,7 @@ import { decode, encode } from "format-png";
 const image = decode(bytes, { preserveMetadata: true, preserveChunks: true });
 const smaller = encode(
     { ...image, header: { ...image.header, interlaced: false } },
-    { compression: 9, palette: "auto", strip: "safe" },
+    { palette: "auto", strip: "safe" },
 );
 ```
 
@@ -137,12 +138,120 @@ const png = encodeRgba8({
 ```js
 import { PngEncoder } from "format-png";
 
-const encoder = new PngEncoder({ compression: 9, palette: "auto" });
+const encoder = new PngEncoder({ palette: "auto" });
 const pngs = frames.map((frame) => encoder.encodeRgba8(frame));
 encoder.free();
 ```
 
-Each instance runs on the thread that created it. To encode in parallel, use one per Web Worker, each with its own `init()`.
+Each instance runs on the thread that created it. To decode or encode in parallel, use the async functions or a worker pool.
+
+## Off the main thread
+
+WebAssembly runs on the thread that calls it, so decoding or encoding a large PNG with the functions above blocks the page until it's done. Each function has an async version that runs it in a Web Worker instead (worker_threads in Node):
+
+```js
+import { decodeRgba8Async, encodeRgba8Async } from "format-png";
+
+decodeRgba8Async(bytes).then((image) => {
+    // the same RgbaImage as decodeRgba8(bytes)
+});
+
+const png = await encodeRgba8Async(imageData, { palette: "auto" });
+```
+
+`decodeAsync`, `decodeRgba8Async`, `encodeAsync`, `encodeRgba8Async`, `readChunksAsync`, `readHeaderAsync` and `parseTextAsync` take the same arguments and options as the functions they're named after, resolve to the same results, and reject with the same `PngError`.
+
+You don't need to call `init()` or set anything up. The functions share one worker pool, created on first use with the default size. Its workers start only as jobs need them, so one call starts one worker. To free the workers and their memory, call `terminateDefaultWorkerPool()`; the next call starts a new pool. `defaultWorkerPool()` returns that pool.
+
+### Worker pools
+
+For control over the workers, such as their number, or a pool per part of your app that you terminate separately, create a pool:
+
+```js
+import { createWorkerPool } from "format-png";
+
+const pool = createWorkerPool(); // or { size: 4 }
+
+const image = await pool.decodeRgba8(bytes);
+const png = await pool.encodeRgba8(image, { palette: "auto" });
+const pngs = await Promise.all(frames.map((frame) => pool.encodeRgba8(frame)));
+
+await pool.terminate();
+```
+
+- **The same API, async.** A pool's `decode`, `decodeRgba8`, `encode`, `encodeRgba8`, `readChunks`, `readHeader` and `parseText` methods work like the async functions.
+- **No `init()` needed.** Each worker loads the embedded WebAssembly module itself.
+- **Size.** By default, one worker per CPU core (`navigator.hardwareConcurrency`, or `os.availableParallelism()` in Node), at most 8.
+- **Lazy workers.** Creating a pool starts nothing. Workers start as jobs need them, up to `size`, and then stay alive between jobs, each keeping its own decoders and encoders, so a batch keeps their buffer reuse.
+- **A queue.** When every worker is busy, jobs wait and start in the order they were submitted. A large image split across the workers takes turns with the jobs after it, so it doesn't hold them up.
+- **Node.** An idle pool doesn't keep the process alive, but call `terminate()` to free the workers' memory.
+
+### One large image, across all the workers
+
+Encoding a large image is mostly compression. A pool spreads one image's compression across its workers: one worker filters the image and splits the result into 1 MiB segments, all the workers compress segments, and the first worker joins them into the PNG. Images with more than 1 MiB of filtered data are split, which is about 512×512 RGBA and up; smaller ones are encoded whole on one worker.
+
+On a 2500×3800 photo-like image at the default level, in Node on a 20-core machine:
+
+| Workers | Time | Speedup |
+|---|---|---|
+| 1 | 7.8 s | 1.0× |
+| 2 | 4.3 s | 1.8× |
+| 4 | 2.7 s | 2.8× |
+| 8 | 2.0 s | 3.8× |
+
+Filtering and joining run on one worker, so the speedup levels off. Run `npm run bench:pool` in the repository to measure your machine.
+
+The segments are compressed independently, so the file is a little larger than one compressed as a single stream, about 0.1% for that photo. A split image's bytes are the same as `encode(image, { threads: "auto" })` gives on any thread, with any number of workers. For the smallest file, pass `threads: "single"`: the pool then encodes on one worker, with the same bytes as the sync `encode`.
+
+### When to use which
+
+Use the async functions or a pool for large images, for batches that can run in parallel, and whenever the page has to stay responsive. Use the sync functions for small images such as icons: posting the bytes to a worker and back costs more than decoding them, and the sync call returns in well under a frame.
+
+WebAssembly memory never shrinks, so each worker keeps as much as the largest image it handled until the pool is terminated. For very large images, use a smaller pool, or a short-lived one, or call `terminateDefaultWorkerPool()` when you're done.
+
+### Copying and transferring buffers
+
+By default the input bytes (or `image.data` for `encode` and `encodeRgba8`) are **copied** to the worker, so you can keep using them. Results always come back without a copy.
+
+With `transfer: true`, the input's `ArrayBuffer` is **moved** to the worker instead, which saves the copy:
+
+```js
+const image = await decodeRgba8Async(bytes, { transfer: true });
+bytes.byteLength; // 0: the buffer is detached as soon as decodeRgba8 is called
+```
+
+> [!WARNING]
+> Transferring detaches the **whole** `ArrayBuffer`, including every other view of it, as soon as the method is called. It can't be a `SharedArrayBuffer`. In Node, don't transfer a small `Buffer` made by `Buffer.from` or `Buffer.allocUnsafe`: it may share Node's internal buffer pool.
+
+`readChunks` returns each chunk's `data` as a view into its input, just as the sync function does: into your `bytes` by default, or into the buffer moved back from the worker with `transfer: true`.
+
+### Cancelling and errors
+
+Pass a `signal` to cancel a job. It rejects with `signal.reason` at once. A queued job leaves the queue. WebAssembly can't be interrupted, so a running job's worker is stopped and replaced, unless it holds an image another job is splitting: then the task finishes first, and its result is dropped.
+
+```js
+const controller = new AbortController();
+const job = encodeRgba8Async(image, { signal: controller.signal });
+controller.abort(); // job rejects with an AbortError
+```
+
+Failures that come from the pool rather than the PNG reject with a `WorkerPoolError`:
+- `code: "terminated"`: the pool was terminated. This applies to queued and running jobs, and to every call after `terminate()`.
+- `code: "worker-crashed"`: the worker stopped unexpectedly. The pool starts a new one for the next job.
+
+### Bundlers
+
+Pools start their workers with `new Worker(new URL("./worker.js", import.meta.url), { type: "module" })`.
+- **Vite and webpack 5** find and bundle that worker with no setup. With Vite, set `worker: { format: "es" }`.
+- **esbuild and Rollup** need it built as a second entry, `format-png/worker.js` (the file `node_modules/format-png/dist/worker.js`), given to the pool with `createWorker`:
+
+```js
+const pool = createWorkerPool({
+    createWorker: () => new Worker(new URL("./worker.js", import.meta.url), { type: "module" }),
+});
+```
+
+`createWorker` also lets you start workers your own way. In Node it may return a `worker_threads` `Worker`.
 
 ## API
 
@@ -159,6 +268,9 @@ Each instance runs on the thread that created it. To encode in parallel, use one
 | `toImageData(image)` | Wraps decoded RGBA for `putImageData`. Browser only. |
 | `pixelsPerInch(dimensions)` | Converts `pHYs` to pixels per inch. |
 | `PngDecoder`, `PngEncoder` | Reusable decoder and encoder. |
+| `decodeAsync`, `decodeRgba8Async`, `encodeAsync`, `encodeRgba8Async`, `readChunksAsync`, `readHeaderAsync`, `parseTextAsync` | The functions above, in a Web Worker or worker_thread, returning promises. |
+| `createWorkerPool(options?)` | A pool of workers with the same async methods, for control over the workers. |
+| `defaultWorkerPool()`, `terminateDefaultWorkerPool()` | The pool behind the async functions, and a way to free it. |
 
 **Decode options:**
 - `validateCrc` (default `true`): check each chunk's checksum.
@@ -176,6 +288,9 @@ Each instance runs on the thread that created it. To encode in parallel, use one
 | `palette` | `"keep"`, `"auto"` | `"keep"` |
 | `strip` | `"keep"`, `"safe"`, `"all"` | `"keep"` |
 | `keepUnsafeChunks` | Also write raw chunks whose data depends on the pixels, such as `bKGD` and `sBIT`. Only when the pixels are unchanged. | `false` |
+| `threads` | `"single"`: compress the image data as one stream. `"auto"`: in independent 1 MiB segments, which a worker pool compresses in parallel; the file is a little larger. | `"single"`; `"auto"` in worker pools and the async functions |
+
+On photos, levels 7 to 9 are much slower than the default for very little gain (level 9 takes about twice as long for a file about 1% smaller), and levels 4 and 5 are several times faster for files 4 to 7% larger, which suits large photos. Smooth images and graphics gain more from higher levels, but take longer still: level 9 can be over 10 times slower.
 
 Every type is exported, and documented in the bundled `.d.ts` file.
 
@@ -195,7 +310,7 @@ try {
 
 ## Browser support
 
-The package is ES2022 with WebAssembly: Chrome and Edge 85, Firefox 90, Safari 14.1 and later.
+The package is ES2022 with WebAssembly: Chrome and Edge 85, Firefox 90, Safari 14.1 and later. Worker pools use module workers: Firefox 114 and Safari 15 or later, and Safari 15.4 for `transfer: true`. A pool can also be created inside a Web Worker.
 
 ## License
 

package/dist/async.d.ts

@@ -0,0 +1,28 @@
+import type { DecodeOptions, EncodeOptions, PngChunks, PngHeader, PngImage, PngText, RawImage, ReadChunksOptions, RgbaImage, RgbaImageInput } from "./core.js";
+import { type JobOptions, type WorkerPool } from "./pool.js";
+/**
+ * The pool behind the async functions, created on first use with the default
+ * size. Its workers start as jobs need them, and in Node an idle pool doesn't
+ * keep the process alive.
+ */
+export declare function defaultWorkerPool(): WorkerPool;
+/**
+ * Terminates the shared pool, freeing its workers and their memory. Its
+ * pending jobs reject with a `WorkerPoolError`; the next async call starts a
+ * new pool.
+ */
+export declare function terminateDefaultWorkerPool(): Promise<void>;
+/** `decode`, in a worker. */
+export declare function decodeAsync(bytes: Uint8Array, options?: DecodeOptions & JobOptions): Promise<RawImage>;
+/** `decodeRgba8`, in a worker. */
+export declare function decodeRgba8Async(bytes: Uint8Array, options?: DecodeOptions & JobOptions): Promise<RgbaImage>;
+/** `readChunks`, in a worker. */
+export declare function readChunksAsync(bytes: Uint8Array, options?: ReadChunksOptions & JobOptions): Promise<PngChunks>;
+/** `readHeader`, in a worker. */
+export declare function readHeaderAsync(bytes: Uint8Array, options?: JobOptions): Promise<PngHeader>;
+/** `parseText`, in a worker. */
+export declare function parseTextAsync(type: PngText["chunkType"], data: Uint8Array, options?: JobOptions): Promise<PngText>;
+/** `encode`, in a worker. */
+export declare function encodeAsync(image: PngImage, options?: EncodeOptions & JobOptions): Promise<Uint8Array>;
+/** `encodeRgba8`, in a worker. */
+export declare function encodeRgba8Async(image: RgbaImageInput, options?: EncodeOptions & JobOptions): Promise<Uint8Array>;

package/dist/async.js

@@ -0,0 +1,48 @@
+import { createWorkerPool } from "./pool.js";
+let shared;
+/**
+ * The pool behind the async functions, created on first use with the default
+ * size. Its workers start as jobs need them, and in Node an idle pool doesn't
+ * keep the process alive.
+ */
+export function defaultWorkerPool() {
+    return (shared ??= createWorkerPool());
+}
+/**
+ * Terminates the shared pool, freeing its workers and their memory. Its
+ * pending jobs reject with a `WorkerPoolError`; the next async call starts a
+ * new pool.
+ */
+export async function terminateDefaultWorkerPool() {
+    const pool = shared;
+    shared = undefined;
+    await pool?.terminate();
+}
+/** `decode`, in a worker. */
+export function decodeAsync(bytes, options) {
+    return defaultWorkerPool().decode(bytes, options);
+}
+/** `decodeRgba8`, in a worker. */
+export function decodeRgba8Async(bytes, options) {
+    return defaultWorkerPool().decodeRgba8(bytes, options);
+}
+/** `readChunks`, in a worker. */
+export function readChunksAsync(bytes, options) {
+    return defaultWorkerPool().readChunks(bytes, options);
+}
+/** `readHeader`, in a worker. */
+export function readHeaderAsync(bytes, options) {
+    return defaultWorkerPool().readHeader(bytes, options);
+}
+/** `parseText`, in a worker. */
+export function parseTextAsync(type, data, options) {
+    return defaultWorkerPool().parseText(type, data, options);
+}
+/** `encode`, in a worker. */
+export function encodeAsync(image, options) {
+    return defaultWorkerPool().encode(image, options);
+}
+/** `encodeRgba8`, in a worker. */
+export function encodeRgba8Async(image, options) {
+    return defaultWorkerPool().encodeRgba8(image, options);
+}