rapier-jxl 2.1.0 → 2.2.0

package/AGENTS.md

@@ -26,3 +26,8 @@
 
 Nothing here reads files, fetches or touches the DOM: it runs in a worker, Node or Deno alike. Publish the version
 you took and its licence; do not vendor a minified copy under another name.
+
+For lossless acceleration with unchanged bytes, use `rapier-jxl/wasm` (or its single-file `wasm.min.mjs`)
+in place of the effort door. It falls back to JavaScript when SIMD is missing or blocked. `KERNELS.md`
+describes the 3 MiB private arena, controls and source build. Do not combine separate minified bundles
+to share kernel controls: use the one-file accelerated door.

package/ENCODER-COMPARISON.md

@@ -5,10 +5,10 @@ A bounded survey, not proof of a global minimum; the encoders differ in what the
 
 | Encoder / entry | Version | Minified JS + WASM bytes | gzip bytes | Included capability |
 | --- | --- | ---: | ---: | --- |
-| Rapier JXL core | 2.1.0 | 19,539 | 8,511 | 8-bit lossless RGBA, lossy modular with exact alpha |
-| Rapier JXL effort door | 2.1.0 | 30,008 | 12,320 | The core, and the weighted predictor searched for lossless |
-| Rapier JXL JPEG door | 2.1.0 | 32,218 | 13,439 | JPEG coefficients, orientation; no JPEG reconstruction |
-| Rapier JXL photo door | 2.1.0 | 31,288 | 12,872 | 8-bit photographic VarDCT, exact alpha; q100 lossless |
+| Rapier JXL core | 2.2.0 | 21,326 | 9,271 | 8-bit lossless RGBA, lossy modular with exact alpha |
+| Rapier JXL effort door | 2.2.0 | 33,584 | 13,793 | The core, and the weighted predictor searched for lossless |
+| Rapier JXL JPEG door | 2.2.0 | 32,354 | 13,505 | JPEG coefficients, orientation; no JPEG reconstruction |
+| Rapier JXL photo door | 2.2.0 | 33,011 | 13,616 | 8-bit photographic VarDCT, exact alpha; q100 lossless |
 | [jSquash](https://github.com/jamsinclair/jSquash/tree/main/packages/jxl) | 1.3.0 | 1,388,572 | 525,782 | 8-bit lossless/lossy RGBA |
 | [Discourse's jSquash package](https://www.npmjs.com/package/@discourse/jxl) | 1.3.0 | 1,388,572 | 525,782 | The same encoder bytes as jSquash |
 | [Lacinak's jSquash fork](https://github.com/kelaci/jSquash) | 1.3.0-kelaci.0 | 2,071,514 | 844,866 | High bit-depth input options |
@@ -117,7 +117,7 @@ and paint-0002 at q80 and q90.
 
 `transcode` against `cjxl --lossless_jpeg=1`. The contracts differ: Rapier carries the coefficients and the
 orientation and cannot rebuild the JPEG file; native also stores the reconstruction data and rebuilt every
-original exactly. So a smaller Rapier row is not a like-for-like win. Six inputs are tiny conformance fixtures;
+accepted original exactly. So a smaller Rapier row is not a like-for-like win. Seven inputs are tiny conformance fixtures;
 Grace is the one photograph.
 
 | JPEG input | Original bytes | Rapier carrier | Native reversible | JPEG rebuilt by native |
@@ -154,7 +154,7 @@ bundled Geist fonts and pinned `@napi-rs/canvas` 0.1.100. Grace is the public-do
 
 ```sh
 cjxl INPUT.pam OUTPUT.jxl --distance=D --effort=7 --num_threads=0 \
-  --alpha_distance=0 --keep_invisible=1 --premultiply=0 --container=0 \
+  --alpha_distance=0 --resampling=1 --ec_resampling=1 --keep_invisible=1 --premultiply=0 --container=0 \
   -x color_space=RGB_D65_SRG_Per_SRG --quiet
 djxl INPUT.jxl OUTPUT.pam --bits_per_sample=8 --color_space=RGB_D65_SRG_Per_SRG --num_threads=0 --quiet
 butteraugli_main REFERENCE.ppm DECODED.ppm --intensity_target 80

package/KERNELS.md

@@ -0,0 +1,58 @@
+# Optional integer kernels
+
+The small `index.mjs` and `effort.mjs` doors remain JavaScript. To opt in, import
+`encode` / `encodeSteps` from `rapier-jxl/wasm` instead of `rapier-jxl/effort`.
+This door keeps the effort API, automatically tries SIMD, and falls back to the
+reference JavaScript if WebAssembly or SIMD is absent or blocked by policy.
+It changes neither lossless bytes nor the hurry rule. Rapier's full image worker
+opts in automatically; the document-only worker still ships no encoder.
+
+The public repository also builds **`wasm.min.mjs`**, one self-contained module.
+The package marks these two entry modules as side-effectful so bundlers retain
+their automatic backend configuration.
+Do not combine `kernels.min.mjs` with a separately bundled core: each bundle would
+own different hooks. There is deliberately no separate minified control door.
+Readable modules can share controls through `rapier-jxl/kernels`:
+
+```js
+import {encode, configureKernels, kernelMode} from 'rapier-jxl/wasm';
+configureKernels('auto'); // 'off', 'scalar', or 'simd' are explicit alternatives
+const bytes = encode(rgba, width, height, {effort: 3, quality: 100});
+console.log(kernelMode()); // the actual backend, not merely the requested one
+```
+
+Configure once before an encode, not inside its progress callback. Controls are
+per JavaScript realm; workers have independent arenas. A second argument selects
+kernels for deterministic benchmarks, for example `{channel: true, weighted:
+false, fill: false}`. This only changes where arithmetic runs, never its result.
+
+## What ships
+
+`kernels-scalar.wat` implements prediction, weighted prediction and property
+extraction, token histograms and a bulk bit writer. `kernels-simd.wat` implements
+four-pixel average/gradient prediction and RGBA-to-planar conversion; rows and
+odd tails take an explicit scalar path. Weighted prediction stays scalar: its
+west-error state is sequential. Both modules use bounded integer arithmetic,
+including signed i64 floor division for the weighted average. No relaxed SIMD,
+floating-point WASM, imports other than private memory, network access, shared
+memory, threads, or runtime dependencies are used.
+
+The private arena is 3 MiB per enabled realm, reused synchronously. No views escape.
+Groups are at most 65,536 samples; Int16 planes, unit multipliers and bounded
+offsets are admitted. Other planes and custom writers keep the JavaScript path.
+Non-little-endian hosts keep JavaScript as well. The existing input and output
+limits remain in force. SIMD detection calls `WebAssembly.validate` on the tiny
+`kernels-probe.wat` module; failure does not disable the encoder.
+
+## Rebuild
+
+The checked-in `kernels-bytes.mjs` contains generated base64 bytes, not source to
+edit. In the Rapier tree run `node repo/images/build-jxl-kernels.mjs`; in a staged
+public repository run `node build-jxl-kernels.mjs`. Install **wabt 1.0.37** only as
+a build tool. `WABT_MODULE` may point to its `index.js` outside the source tree.
+Add `--check` to reject a generated file that differs. No compiler ships in the
+inline encoder. Source and build script are retained in the public package.
+
+The JX/3 handoff records the bootstrap compiler used in its network-restricted
+sandbox and the outstanding independent wabt rebuild. Do not describe that
+handoff as a wabt-certified build until `--check` has been run with wabt.

package/README.md

@@ -1,8 +1,23 @@
 # Rapier JXL
 
-A JPEG XL encoder in pure JavaScript: one file, 19.5 kB, 8.5 kB gzipped, no WebAssembly, no
-dependencies, MIT. The encoder inside [Rapier](https://rapier.website), published on its own. The smallest
-JavaScript or WebAssembly JPEG XL encoder among the payloads [we measured](ENCODER-COMPARISON.md).
+A JPEG XL encoder in pure JavaScript, for writing `.jxl` from canvas pixels or a JPEG in a browser, a worker, Node
+or Deno. The core needs no WebAssembly; an optional SIMD door accelerates the same arithmetic. No server.
+
+- The core is one file of 21.3 kB, 9.3 kB gzipped: the smallest JavaScript or WebAssembly JPEG XL
+  encoder among the payloads [we measured](ENCODER-COMPARISON.md).
+- Lossless and lossy in one `encode` call; alpha stays exact at every quality.
+- Photographs, JPEGs carried without decoding, and smaller exact files at more time are optional doors, added
+  only when imported.
+- The same input writes the same bytes in every JavaScript engine; quality is measured against libjxl 0.12.0.
+- No dependencies, MIT. An agent adding it to an app reads `AGENTS.md`.
+
+```sh
+npm install rapier-jxl
+```
+
+The encoder inside [Rapier](https://rapier.website), published on its own.
+
+## What it does
 
 - **Lossless.** Every pixel back as it went in: 8-bit grey, grey with alpha, RGB, RGBA.
 - **Lossy**, quality 1 to 99, for flat-colour rasters (screenshots, pixel art, scanned line art). Alpha stays exact.
@@ -19,6 +34,10 @@ JavaScript or WebAssembly JPEG XL encoder among the payloads [we measured](ENCOD
 - **sRGB or Display P3.** `{colorSpace: 'display-p3'}` declares a wide-gamut canvas's samples. A JPEG's profile is
   read by what it does, not what it says: sRGB and Display P3 are carried and declared.
 
+For the accelerated effort door, use `rapier-jxl/wasm` or the self-contained `wasm.min.mjs`.
+It falls back to JavaScript when SIMD is unavailable and writes identical lossless bytes.
+See [KERNELS.md](KERNELS.md) for controls, memory, source and rebuild instructions.
+
 ## Use it
 
 ```js
@@ -89,6 +108,10 @@ also retain the ordinary prefix searches, and photo effort 5 retains its quantis
 group buffer of 1,376,256 bytes plus histogram tables; encoding both candidates costs more CPU and may raise peak
 RSS. Hurry keeps a completed candidate even when it arrives at the final group's yield.
 
+Version 2.2.0 adds the `kernels` and `wasm` doors (integer WebAssembly under the lossless hot loops, JavaScript
+fallback), a worker pool for large exact pictures, exact hybrid-integer codes, and a colour-transform search at effort 4.
+Exact pictures stay lossless; 27 of the 154 test streams change bytes, none larger.
+
 Version 2.1.0 fits the photo door's existing quantisation constants across thumbnail and source-sized photos.
 Lossy photo bytes change deliberately; lossless and ordinary JPEG streams retain their hashes. Pin a package
 version when exact lossy output bytes matter. Quality numbers remain a setting, not a guarantee of equal
@@ -125,16 +148,17 @@ New: `rapier-jxl/effort`, each door's twin in steps with `hurry`, `colorSpace: '
 
 | file | bytes | gzip | Brotli | added to the core, gzip |
 | --- | ---: | ---: | ---: | ---: |
-| `rapier-jxl.min.mjs`, the core: `encode` | 19,539 | 8,511 | 7,515 | |
-| `effort.min.mjs`: `encode` with effort | 30,008 | 12,320 | 10,860 | 3,809 |
-| `jpeg.min.mjs`: `transcode` | 32,218 | 13,439 | 11,874 | 8,497 |
-| `photo.min.mjs`: `encodePhoto` | 31,288 | 12,872 | 11,370 | 6,257 |
-| `jpeg-ans.min.mjs`: optional ANS carrier | 35,281 | 14,506 | 12,804 | 9,564 |
-| `photo-ans.min.mjs`: optional ANS photo | 34,293 | 13,899 | 12,295 | 7,298 |
-| every door in one bundle | 61,101 | 24,091 | 21,147 | |
-| all readable modules | 174,894 | 53,445 | | |
-
-Exact bytes of release 2.1.0's files, measured by the script that stages this repository; `sizes.json`
+| `rapier-jxl.min.mjs`, the core: `encode` | 21,326 | 9,271 | 8,233 | |
+| `effort.min.mjs`: `encode` with effort | 33,584 | 13,793 | 12,173 | 4,522 |
+| `wasm.min.mjs`: accelerated effort, JS fallback | 43,535 | 18,533 | 16,239 | 9,262 |
+| `jpeg.min.mjs`: `transcode` | 32,354 | 13,505 | 11,946 | 8,524 |
+| `photo.min.mjs`: `encodePhoto` | 33,011 | 13,616 | 12,048 | 6,249 |
+| `jpeg-ans.min.mjs`: optional ANS carrier | 35,491 | 14,608 | 12,892 | 9,580 |
+| `photo-ans.min.mjs`: optional ANS photo | 35,994 | 14,650 | 12,975 | 7,315 |
+| every door in one bundle | 74,625 | 30,285 | 26,433 | |
+| all readable modules | 211,215 | 67,195 | | |
+
+Exact bytes of release 2.2.0's files, measured by the script that stages this repository; `sizes.json`
 carries their hashes and tools (terser 5.51.2, Node v22.22.2; gzip 9, Brotli 11). Each minified file stands alone
 and is proved at staging to write the same bytes as its readable source; the last column is what a door adds to a
 bundle that already holds the core. `effort`'s `encode` is the core's at effort 1, so it takes the core's place, in
@@ -143,8 +167,10 @@ that column and in the bundle of every door.
 What it writes: bare codestreams, 8-bit, prefix codes (or ANS in the optional doors), one frame, no preview, animation, ICC (sRGB or
 Display P3 is declared), XYB, chroma-from-luma or filters. Lossless in modular mode, a palette of up to 2,048
 colours weighed against direct coding by actual length, groups of 256, reversible YCoCg, a predictor chosen per
-channel; at effort 2 and 3 also the weighted predictor, its contexts split by its own error; at 4 local modelling of
-palette indices; at 6 local gradient-property splits. Effort 5 uses rung 4. Lossy through Squeeze
+channel and each channel's integer code (one of four hybrid configurations) chosen by exact cost; at effort 2 and 3
+also the weighted predictor, its contexts split by its own error; at 4 local modelling of palette indices and a
+sampled search of the 42 reversible colour transforms, kept only when its complete stream is smaller; at 6 local
+gradient-property splits. Effort 5 uses rung 4. Lossy through Squeeze
 with exact alpha. Carried JPEGs in VarDCT with the JPEG's own tables. The photo door
 writes DCT8 coefficients for the same writer.
 
@@ -162,7 +188,7 @@ headers, `npm run fuzz:scale -- --out fuzz-run --workers 4` repeats the fixed bu
 
 ## Why
 
-Rapier keeps pictures inside Markdown, and its standard says a raster picture in Markdown is JPEG XL: exact where
+Rapier keeps pictures inside Markdown, and its standard defaults to JPEG XL for raster pictures in Markdown: exact where
 it must be, small where it may be. Drawings stay SVG. An editor carrying an encoder offline in a small page needed
 one this size, and none existed. The standard is at [rapier.website](https://rapier.website); for an agent, see
 `AGENTS.md`.

package/bits.mjs

@@ -71,10 +71,13 @@ export class BitWriter {
 // makes a nested one's fractions the index-th of `count` equal shares of its caller's.
 export function complete(steps) { let step; while (!(step = steps.next()).done); return step.value; }
 export function* part(steps, index, count) {
-  let step, hurry;
-  while (!(step = steps.next(hurry)).done) hurry = yield (index + step.value) / count;
+  let step, reply;
+  while (!(step = steps.next(reply)).done) reply = yield scaled(step.value, done => (index + done) / count);
   return step.value;
 }
+// A step's fraction mapped by `f`. A pass handed to a pool (frame.mjs, groupPass) is one step carrying its fractions as
+// `at`, mapped alike; its reply passes back unchanged.
+export const scaled = (value, f) => typeof value === 'number' ? f(value) : {...value, at: g => f(value.at(g))};
 
 // Residuals travel unsigned: 0, -1, 1, -2, 2 ... become 0, 1, 2, 3, 4 ...
 export function packSigned(value) { return value >= 0 ? value * 2 : -value * 2 - 1; }

package/build-jxl-kernels.mjs

@@ -0,0 +1,29 @@
+// SPDX-License-Identifier: MIT
+// Build-time only. npm install --no-save wabt@1.0.37 (or set WABT_MODULE to its index.js).
+// node repo/images/build-jxl-kernels.mjs [--check]
+import {readFile, writeFile} from 'node:fs/promises';
+import {fileURLToPath, pathToFileURL} from 'node:url';
+import {resolve} from 'node:path';
+import {existsSync} from 'node:fs';
+const directory = new URL(existsSync(new URL('jxl/kernels-scalar.wat', import.meta.url)) ? 'jxl/' : './', import.meta.url);
+const specifier = process.env.WABT_MODULE ? pathToFileURL(resolve(process.env.WABT_MODULE)).href : 'wabt';
+let factory;
+try { ({default: factory} = await import(specifier)); }
+catch (cause) { throw new Error('The kernel build needs build-time wabt@1.0.37. Set WABT_MODULE to its index.js when installed outside this tree.', {cause}); }
+const wabt = await factory();
+let text = '// Generated from kernels-*.wat. MIT. Do not edit.\n';
+for (const [name, symbol] of [['scalar','SCALAR'], ['simd','SIMD'], ['probe','SIMD_PROBE']]) {
+  const source = new URL(`kernels-${name}.wat`, directory);
+  const wat = wabt.parseWat(fileURLToPath(source), await readFile(source, 'utf8'), {simd: true});
+  try {
+    wat.resolveNames(); wat.validate({simd: true});
+    const {buffer} = wat.toBinary({log: false, canonicalize_lebs: true, write_debug_names: false});
+    if (!WebAssembly.validate(buffer)) throw new Error(`Invalid ${name} module`);
+    text += `export const ${symbol} = ${JSON.stringify(Buffer.from(buffer).toString('base64'))};\n`;
+    console.log(`${name}: ${buffer.length} WebAssembly bytes`);
+  } finally { wat.destroy(); }
+}
+const target = new URL('kernels-bytes.mjs', directory);
+if (process.argv.includes('--check')) {
+  if (text !== await readFile(target, 'utf8')) throw new Error('kernels-bytes.mjs differs from the WAT build');
+} else await writeFile(target, text);

package/effort-job.mjs

@@ -0,0 +1,229 @@
+// SPDX-License-Identifier: MIT
+// The effort door's work (effort.mjs): effort 1's stream, the rungs above it, and the group work of the weighted search
+// that a pool of workers shares (pool.mjs). Every choice is integer arithmetic: the same input and options write the
+// same bytes everywhere, on one thread or many.
+import {BitWriter, part, scaled} from './bits.mjs';
+import {writeImageHeader, writeModularFrameHeader, groupLayout, groupRect, groupPass, addCounts, assembleCodestream, GROUP_DIM} from './frame.mjs';
+import {buildCode, writePrefixCode} from './prefix.mjs';
+import {AVERAGE_PREDICTOR, GRADIENT_PREDICTOR, ALPHABET, leaf, split, channelTree, writeTree, writeModularHeader, writeChannelHistograms, codeChannel} from './modular.mjs';
+import {inspectPixels, losslessSteps, planeFill} from './lossless.mjs';
+import {lossySteps} from './lossy.mjs';
+import {WEIGHTED_PREDICTOR, WEIGHTED_PROPERTY, WEIGHTED_CUTS, codeWeighted} from './weighted.mjs';
+import {fault, admitOptions, admitPixels} from './admit.mjs';
+import {localSteps} from './local.mjs';
+import {rctSearchSteps} from './rct-search.mjs';
+
+// The hurry in a step's reply: the caller's flag, or for a pooled pass a reply of null (ended) or {hurried}.
+const hurryOf = reply => reply === null || (typeof reply === 'object' ? reply.hurried : reply);
+
+// The door's options as every door reads them, and the effort, which this door alone reads (the core, one rung, does
+// not): a whole number from 1 to 9, 1 when absent. Admitted at once; the steps follow. With `pool` (pool.mjs drives
+// them), an exact picture of more than one group yields each pass over its groups as one request: the same bytes.
+export function effortJob(data, width, height, options, pool) {
+  const {quality, colorSpace} = admitOptions(options), effort = options?.effort === undefined ? 1 : options.effort;
+  if (!Number.isInteger(effort) || effort < 1 || effort > 9) throw fault('JXL_INPUT', 'Effort is a whole number from 1 to 9.');
+  admitPixels(data, width, height);
+  return effortSteps(data, width, height, quality, colorSpace, effort, pool && quality >= 100 && !groupLayout(width, height).single);
+}
+
+// Effort 1 is the core's work (index.mjs), step for step; the rungs run after it and keep a smaller stream if they
+// find one. A search that runs out of memory leaves the smallest completed stream standing.
+function* effortSteps(data, width, height, quality, colorSpace, effort, pooled) {
+  const shape = inspectPixels(data, width, height);
+  if (quality < 100) {
+    if (!shape.palette) return yield* lossySteps(data, width, height, {quality, shape, colorSpace});
+    const exact = yield* part(losslessSteps(data, width, height, {shape, colorSpace}), 0, 2);
+    let bytes;
+    try { bytes = yield* part(lossySteps(data, width, height, {quality, shape, colorSpace}), 1, 2); }
+    catch (error) { if (error.code !== 'JXL_SIZE' && (!(error instanceof RangeError) || error.code)) throw error; bytes = exact; }
+    return exact.length <= bytes.length ? exact : bytes;
+  }
+  if (effort < 2) return yield* losslessSteps(data, width, height, {shape, colorSpace, pooled});
+  let best = yield* part(losslessSteps(data, width, height, {shape, colorSpace, pooled}), 0, 2);
+  const direct = shape.palette ? {...shape, palette: null} : shape;
+  if (effort >= 4) {
+    const searches = [searchSteps(data, width, height, direct, colorSpace, 3, pooled)];
+    if (shape.palette) searches.push(localSteps(data, width, height, shape, colorSpace, 4, true, pooled));
+    if (effort >= 6) {
+      searches.push(localSteps(data, width, height, shape, colorSpace, 6, false, pooled));
+      if (shape.palette) searches.push(localSteps(data, width, height, shape, colorSpace, 6, true, pooled));
+    }
+    searches.push(rctSearchSteps(data, width, height, shape, colorSpace, pooled));
+    for (let i = 0; i < searches.length; i++) {
+      try {
+        let step, reply, hurried = false;
+        while (!(step = searches[i].next(reply)).done) hurried = hurryOf(reply = yield scaled(step.value, done => 0.5 + (i + done) / (2 * searches.length)));
+        if (step.value && step.value.length < best.length) best = step.value;
+        if (hurried) return best;
+      } catch (error) { if (error.code !== 'JXL_SIZE' && (!(error instanceof RangeError) || error.code)) throw error; }
+    }
+    return best;
+  }
+  try {
+    const bytes = yield* part(searchSteps(data, width, height, direct, colorSpace, effort, pooled), 1, 2);
+    if (bytes && bytes.length < best.length) best = bytes;
+  } catch (error) { if (error.code !== 'JXL_SIZE' && (!(error instanceof RangeError) || error.code)) throw error; }
+  return best;
+}
+
+// A token's raw bits: a residual token s above zero carries s - 1 of them, an LZ77 length token above 235 s - 236.
+const raw = s => Math.max(0, s < 224 ? s - 1 : s - 236);
+// A histogram's complete cost in bits: its prefix code's header and every token's code and raw bits.
+function cost(freqs) {
+  const code = buildCode(freqs), writer = new BitWriter(128);
+  writePrefixCode(writer, code);
+  let bits = writer.bitLength;
+  for (let s = 0; s < freqs.length; s++) if (freqs[s]) bits += freqs[s] * (code.lengths[s] + raw(s));
+  return bits;
+}
+const sum = histograms => { const out = new Uint32Array(ALPHABET); for (const h of histograms) for (let s = 0; s < ALPHABET; s++) out[s] += h[s]; return out; };
+
+const CONTEXTS = WEIGHTED_CUTS.length + 1, IDENTITY = Int32Array.from({length: CONTEXTS}, (_, k) => k);
+
+// A group of the search, here or in a pool's worker. Counting, every channel goes into `target` twice: under effort 1's
+// predictor (`first`, one histogram per channel) and under the weighted one, its tokens kept by error interval
+// (CONTEXTS histograms per channel after those). Writing, each channel through its plan (`plans`: the leaf, its codes
+// and the intervals' contexts) into `target`, or, when none, as the group's own section, whose bytes it returns.
+export function searchGroup(setup) {
+  const {channels, first, plans} = setup, fill = planeFill(setup), leaves = first?.map(p => leaf(p));
+  const planes = Array.from({length: channels}, () => new Int16Array(GROUP_DIM * GROUP_DIM));
+  return (rgba, stride, x0, y0, w, h, target) => {
+    fill(planes, rgba, stride, x0, y0, w, h);
+    if (!plans) {
+      for (let c = 0; c < channels; c++) {
+        codeChannel(null, target[c], planes[c], w, h, leaves[c]);
+        codeWeighted(null, target.slice(channels + c * CONTEXTS, channels + (c + 1) * CONTEXTS), planes[c], w, h, 0, IDENTITY);
+      }
+      return;
+    }
+    const section = target || new BitWriter(w * h * channels + 64);
+    if (!target) writeModularHeader(section, {useGlobalTree: true, transforms: []});
+    plans.forEach(({leaf: l, codes, contextOf}, c) => {
+      if (l.predictor === WEIGHTED_PREDICTOR) codeWeighted(section, codes, planes[c], w, h, 0, contextOf);
+      else codeChannel(section, codes[0], planes[c], w, h, l);
+    });
+    return target ? undefined : section.finish();
+  };
+}
+
+// The direct plan of lossless.mjs (YCoCg, 256-pixel groups), every channel counted over the whole picture under effort
+// 1's predictor (chosen by samples, as the core chooses it) and under the weighted one; each rung's plan takes every
+// channel's cheapest, and the plans are written the cheapest way. Nothing written (null) when no plan leaves effort 1.
+function* searchSteps(rgba, width, height, shape, colorSpace, effort, pooled) {
+  const {channels} = shape, layout = groupLayout(width, height), groups = layout.groupsX * layout.groupsY;
+  const setup = {channels, alpha: shape.alpha, palette: null}, fill = planeFill(setup);
+  const planes = Array.from({length: channels}, () => new Int16Array(32 * 32));
+  const rect = g => groupRect(layout, width, height, g);
+  // Effort 1's predictor per channel: three 32-pixel samples priced under gradient and average, as the core does.
+  const sw = Math.min(width, 32), sh = Math.min(height, 32);
+  const sampled = [GRADIENT_PREDICTOR, AVERAGE_PREDICTOR].map(predictor => ({predictor, freqs: planes.map(() => new Uint32Array(ALPHABET))}));
+  for (const fraction of [0, 0.5, 1]) {
+    fill(planes, rgba, width, Math.floor((width - sw) * fraction), Math.floor((height - sh) * fraction), sw, sh);
+    for (const candidate of sampled) for (let c = 0; c < channels; c++) codeChannel(null, candidate.freqs[c], planes[c], sw, sh, leaf(candidate.predictor));
+  }
+  const first = planes.map((_, c) => leaf(sampled[cost(sampled[1].freqs[c]) < cost(sampled[0].freqs[c]) ? 1 : 0].predictor));
+  // Every channel counted under effort 1's predictor and under the weighted one, its tokens kept by error interval.
+  // A step is a group of the counting pass (the first half of the search's fractions) or of a plan's writing (the
+  // second half, shared by the plans written), so the pace of the steps holds whether one plan is written or two.
+  const contexts = CONTEXTS;
+  const firstCounts = planes.map(() => new Uint32Array(ALPHABET)), intervals = planes.map(() => Array.from({length: contexts}, () => new Uint32Array(ALPHABET)));
+  const counts = [...firstCounts, ...intervals.flat()], counted = {...setup, first: first.map(l => l.predictor)}, count = searchGroup(counted);
+  if ((yield* groupPass({pooled, kind: 'search', setup: {...counted, sizes: counts.map(h => h.length)}, at: g => (g + 1) / (2 * groups), stop: () => true},
+    groups, g => count(rgba, width, ...rect(g), counts), (g, partial) => addCounts(counts, partial))) === null) return null;
+  // Rung 2: each channel's cheaper of effort 1's predictor and the weighted one in one context.
+  const rung2 = planes.map((_, c) => {
+    const plan = {cost: cost(firstCounts[c]), leaves: [first[c]], freqs: [firstCounts[c]], cuts: []};
+    const whole = sum(intervals[c]), single = cost(whole);
+    return single < plan.cost ? {cost: single, leaves: [leaf(WEIGHTED_PREDICTOR)], freqs: [whole], cuts: []} : plan;
+  });
+  // Rung 3: or its intervals grouped into runs of neighbours, the grouping of least cost found exactly by dynamic
+  // programming. best[j] is the least cost of intervals 0..j-1 grouped, `from[j]` where its last group begins; an
+  // interval no token reaches leaves the merged cost as it was.
+  const rung3 = effort < 3 ? rung2 : rung2.map((plan, c) => {
+    const filled = intervals[c].map(h => h.some(v => v));
+    const best = new Float64Array(contexts + 1).fill(Infinity), from = new Int32Array(contexts + 1);
+    best[0] = 0;
+    for (let j = 1; j <= contexts; j++) {
+      const merged = new Uint32Array(ALPHABET);
+      let bits = 0;
+      for (let i = j - 1; i >= 0; i--) {
+        if (filled[i]) { const h = intervals[c][i]; for (let s = 0; s < ALPHABET; s++) merged[s] += h[s]; bits = cost(merged); }
+        // Every group is a leaf of the tree and an entry of the context map: 24 bits more, so an interval no token
+        // reaches joins its neighbour.
+        const price = best[i] + bits + 24;
+        if (price < best[j]) { best[j] = price; from[j] = i; }
+      }
+    }
+    if (!(best[contexts] < plan.cost)) return plan;
+    const groupsOf = [];
+    for (let j = contexts; j > 0; j = from[j]) groupsOf.unshift([from[j], j]);
+    return {cost: best[contexts], leaves: groupsOf.map(() => leaf(WEIGHTED_PREDICTOR)), freqs: groupsOf.map(([i, j]) => sum(intervals[c].slice(i, j))), cuts: groupsOf.slice(0, -1).map(([, j]) => WEIGHTED_CUTS[j - 1]),
+      contextOf: Int32Array.from({length: contexts}, (_, k) => groupsOf.findIndex(([i, j]) => k >= i && k < j))};
+  });
+  const candidates = [rung2, rung3].filter((plans, k) => k ? plans.some((plan, c) => plan !== rung2[c]) : plans.some((plan, c) => plan.leaves[0] !== first[c]));
+  if (!candidates.length) return null;
+  // The tree: one subtree per channel on the channel property, and inside a split channel a balanced tree on the
+  // weighted predictor's property, values above a cut to the left.
+  const byError = (leaves, cuts) => {
+    const build = (lo, hi) => { if (lo === hi) return leaves[lo]; const mid = (lo + hi) >> 1; return split(WEIGHTED_PROPERTY, cuts[mid], build(mid + 1, hi), build(lo, mid)); };
+    return build(0, leaves.length - 1);
+  };
+  const assemble = plans => ({tree: channelTree(plans.map(plan => byError(plan.leaves, plan.cuts))), leaves: plans.flatMap(plan => plan.leaves), freqs: plans.flatMap(plan => plan.freqs)});
+  // A plan's price in bits: its tree and histograms as written, and every token's code and raw bits. Two plans' streams
+  // differ by their prices and at most 27 bits a section more (a section's padding, and its size's field in the table
+  // of contents, 12 to 32 bits) and a byte: a plan that far cheaper is written alone, closer plans both.
+  const price = plans => {
+    const {tree, leaves, freqs} = assemble(plans), w = new BitWriter(4096);
+    const histograms = writeChannelHistograms(w, writeTree(w, tree), freqs, l => leaves.indexOf(l));
+    let bits = w.bitLength;
+    freqs.forEach((f, i) => { const {lengths} = histograms[i + 1].code; for (let s = 0; s < f.length; s++) if (f[s]) bits += f[s] * (lengths[s] + raw(s)); });
+    return bits;
+  };
+  let chosen = candidates;
+  if (candidates.length > 1) {
+    const [a, b] = candidates.map(price), margin = 27 * (layout.single ? 1 : groups + 1) + 8;
+    if (b + margin <= a) chosen = [candidates[1]];
+    else if (a + margin <= b) chosen = [candidates[0]];
+    else if (b < a) chosen = [candidates[1], candidates[0]];
+  }
+  let written = 0, hurried = false;
+  let smallest = null;
+  for (const plans of chosen) {
+    const bytes = yield* write(plans);
+    if (!bytes) return smallest;
+    // Price the likely winner first, but retain rung 2's byte choice on an equal-length completed pair.
+    if (!smallest || bytes.length < smallest.length || (bytes.length === smallest.length && plans === candidates[0])) smallest = bytes;
+    if (hurried) return smallest;
+  }
+  return smallest;
+
+  // A plan's stream, a group per step; a hurry at its final group keeps the already-completed candidate.
+  function* write(plans) {
+    const {tree, leaves, freqs} = assemble(plans);
+    const transforms = channels >= 3 ? [{type: 'rct', beginC: 0, rctType: 6}] : [];
+    const header = new BitWriter(256);
+    writeImageHeader(header, width, height, shape.colour, shape.alpha, {colorSpace});
+    writeModularFrameHeader(header, {alpha: shape.alpha});
+    const global = new BitWriter(4096);
+    global.write(1, 1);  // default DC quantisation
+    global.write(1, 1);  // a global tree
+    const histograms = writeChannelHistograms(global, writeTree(global, tree), freqs, l => leaves.indexOf(l));
+    writeModularHeader(global, {useGlobalTree: true, transforms});
+    const coded = {...setup, plans: plans.map(plan => ({leaf: plan.leaves[0], codes: plan.leaves.map(l => histograms[leaves.indexOf(l) + 1].code), contextOf: plan.contextOf}))};
+    const group = searchGroup(coded), sections = [];
+    if (layout.single) {
+      group(rgba, width, 0, 0, width, height, global);
+      hurried = yield 0.5 + ++written / (2 * chosen.length * groups);
+      sections.push(global.finish());
+    } else {
+      sections.push(global.finish());
+      for (let i = 0; i < layout.dcGroupsX * layout.dcGroupsY + 1; i++) sections.push(new Uint8Array(0));
+      const at = written, base = sections.length, place = (g, section) => { sections[base + g] = section; };
+      hurried = yield* groupPass({pooled, kind: 'search', setup: coded, at: g => 0.5 + (at + g + 1) / (2 * chosen.length * groups), stop: g => g + 1 < groups},
+        groups, g => place(g, group(rgba, width, ...rect(g))), place);
+      if (hurried === null) return null;
+      written += groups;
+    }
+    return assembleCodestream(header, sections);
+  }
+}

package/effort.d.mts

@@ -4,7 +4,8 @@ export {LIMITS} from './index.mjs';
 export type {Pixels, Limits, EncodeOptions, ErrorCode, EncoderError, Job} from './index.mjs';
 /** The core's encode with a search above it for exact pictures: effort 1 (the default) writes the core's bytes; 2 tries
  * the weighted predictor on every channel, 3 splits its contexts by that predictor's error, 4 models palette indices
- * per group, and 6 also learns gradient-property splits. A level without another rung uses the preceding one.
+ * per group and tries the other reversible colour transforms, and 6 also learns gradient-property splits. A level
+ * without another rung uses the preceding one.
  * Never larger than the effort below; slower. */
 export function encode(data: Pixels, width: number, height: number, options?: EncodeOptions): Uint8Array;
 /** encode's work and bytes as steps. Above effort 1 an exact picture's first half (fractions up to 0.5) writes effort