html-minifier-setup 0.1.0 → 0.2.0

package/README.md

@@ -16,7 +16,7 @@ npx html-minifier-setup [inputs...] [options]
 
 (Like HTML Minifier, HTML Minifier Setup requires Node.js ≥22.13.)
 
-Inputs are HTML files, directories, or URLs (default: the current directory). From directories, up to 20 pages (`--sample`) are picked, spread across subdirectories to cover a site’s templates.
+Inputs are HTML files, directories, or URLs (default: the current directory). From directories, up to 20 pages (`--sample`) are picked, spread across a site’s sections first and then across the folders within them, to cover its templates. The same pages give the same sample, and the report says how many pages it was taken from.
 
 The render check, which catches most of what static checks can’t, uses Playwright’s Chromium, or else Chrome or Edge, if installed. Where none is found, HTML Minifier Setup offers to download Chromium.
 
@@ -43,7 +43,6 @@ npx html-minifier --input-dir ./dist --output-dir ./dist
 | `--keep <list>` | Options to recommend regardless of risks and gains |
 | `--skip <list>` | Options not to evaluate |
 | `--no-render` | Skip the render check, not recommending options that need it |
-| `--no-equivalence` | Skip HTML Minifier’s equivalence check |
 | `--no-presets` | Skip comparing with HTML Minifier’s presets |
 | `--viewports <list>` | Viewports to render at (default: `1280x800,390x844`) |
 | `--concurrency <count>` | Pages to render at once, across viewports and options (default: 4) |
@@ -85,7 +84,6 @@ Where full whitespace collapsing breaks a page, HTML Minifier Setup falls back t
 | --- | --- | --- |
 | Static | What scripts and style sheets of the page (including linked local or same-origin ones) rely on: comments that frameworks read (e.g., Knockout, Lit, hydration markers), empty elements scripts fill, classes scripts add after `removeUnusedCSS` removed their rules, attributes that style sheets or scripts select on, and CSS that keeps whitespace | Nothing |
 | Render | Anything that changes what the page looks like on load, or adds script errors | Chromium, Chrome, or Edge |
-| Equivalence | HTML Minifier bugs, as HTML Minifier’s own check finds output that means something else to browsers than its source—for the options that check models | An HTML Minifier checkout next to this one (for now) |
 
 In the report, options are **recommended**, **kept** (as asked), or not used because they **don’t fit** these pages, have **no gain** by the chosen metric, have **no effect**, or were **skipped**. With `--json`, statuses are `accepted`, `kept`, `rejected`, `no-gain`, `no-effect`, and `skipped`.
 
@@ -100,7 +98,7 @@ The checks can show that options change pages, not that they never will:
 * The render check sees pages as they load, not after interaction (e.g., a menu a script opens, using classes `removeUnusedCSS` removed).
 * Scripts from other origins can’t be read, and classes or attributes a script builds at runtime (e.g., `'is-' + state`) can’t be found.
 * Rendering must match pixel for pixel, so lossy changes (e.g., SVG coordinates rounded by `minifySVG`) count as differences.
-* A page sample may miss templates the rest of the site uses.
+* A page sample may miss templates the rest of the site uses—for sites with many different templates, raise `--sample`.
 * Options that have no effect can’t be judged: On pages that are minified already, most options are left out, as there’s nothing left for them to do. Run HTML Minifier Setup on pages before minification instead.
 
 Test the configuration on your site before deploying it.
@@ -109,4 +107,39 @@ Options that newer versions of HTML Minifier add are evaluated once HTML Minifie
 
 Options HTML Minifier can’t express in one configuration are not evaluated: `minifyURLs` depends on each page’s URL, and `removeTagWhitespace` produces invalid HTML.
 
-Large pages take longer, especially by `brotli`.
+Large pages take longer, especially by `brotli`.
+
+## Working on HTML Minifier Setup
+
+Note: This section assumes working with dependencies installed (`npm i`).
+
+### Tests
+
+```shell
+npm test
+```
+
+Runs the type check (`tsc`) and then the suites in test/. Render tests skip where no browser is installed (`npx playwright install chromium` installs one). `npm run lint` covers the rest.
+
+### Equivalence check
+
+With [an HTML Minifier](https://github.com/j9t/html-minifier) checkout next to this one (../html-minifier), runs also use HTML Minifier’s equivalence check (test/equivalence/ there), which finds output that means something else to browsers than its source—HTML Minifier bugs rather than what a site relies on. Options whose output it flags don’t fit, and the report names the check and the HTML Minifier versions involved. It covers the options the check models.
+
+The check runs on the main thread and adds noticeably to the time evaluating options takes; `--no-equivalence` skips it. Without a checkout, as for users, it doesn’t run, and test/equivalence.test.js skips.
+
+### Benchmarks
+
+```shell
+npm run benchmark
+```
+
+Times a run by phase as the CLI does it—including the equivalence check, where available, and the presets—on the test fixtures, or on files, directories, or URLs passed as arguments. Times are each phase’s fastest iteration.
+
+Parameters:
+
+* No argument: Runs and, if a baseline exists, shows deltas
+* `--save`: Saves the run as the baseline (benchmark/benchmark-baseline.json, e.g., on `main` before switching to a branch)
+* `--no-render`: Skips the render check, timing what doesn’t need a browser
+* `--iterations=N`: Sets the number of timed iterations (default: 3)
+
+Runs are only compared with a baseline taken with the same inputs, render setting, and iterations. To profile, run `node --cpu-prof benchmark/benchmark.js` and load the .cpuprofile file in Chrome DevTools or with `npx speedscope`.

package/cli.js

@@ -5,7 +5,7 @@ import path from 'node:path';
 import readline from 'node:readline/promises';
 import { pathToFileURL } from 'node:url';
 import { Command, Option } from 'commander';
-import { splitConfig } from './src/catalog.js';
+import { checkCandidates, splitConfig } from './src/catalog.js';
 import { paint, supportsColor } from './src/color.js';
 import { loadEquivalence } from './src/equivalence.js';
 import { METRICS } from './src/measure.js';
@@ -20,6 +20,8 @@ let progress = createProgress();
 
 const { version } = JSON.parse(fs.readFileSync(new URL('package.json', import.meta.url), 'utf8'));
 
+const INTRO = 'This is HTML Minifier Setup, identifying a recommended configuration for HTML Minifier.';
+
 /** @param {unknown} value */
 const list = (value) => (Array.isArray(value) ? value.map(String) : String(value).split(',')).map((item) => item.trim()).filter(Boolean);
 
@@ -47,11 +49,19 @@ const viewports = (value) => {
  */
 async function readConfig(file) {
   const resolved = path.resolve(file);
-  if (/\.[cm]?js$/.test(resolved)) {
+  const load = async () => {
     const module = await import(pathToFileURL(resolved).href);
-    return module.default ?? module;
+    return 'default' in module ? module.default : module;
+  };
+  if (/\.[cm]?js$/i.test(resolved)) return load();
+  const text = (await fs.promises.readFile(resolved, 'utf8')).replace(/^\uFEFF/, '');
+  // As HTML Minifier reads them: other files as JSON, or else as module
+  if (/\.json$/i.test(resolved)) return JSON.parse(text);
+  try {
+    return JSON.parse(text);
+  } catch {
+    return load();
   }
-  return JSON.parse(await fs.promises.readFile(resolved, 'utf8'));
 }
 
 const program = new Command()
@@ -68,7 +78,8 @@ const program = new Command()
   .option('--keep <list>', 'Options to recommend regardless of risks and gains', list, [])
   .option('--skip <list>', 'Options not to evaluate', list, [])
   .option('--no-render', 'Skip the render check (not recommending options that need it)')
-  .option('--no-equivalence', 'Skip HTML Minifier’s equivalence check')
+  // For development, where an HTML Minifier checkout sits next to this one
+  .addOption(new Option('--no-equivalence', 'Skip HTML Minifier’s equivalence check').hideHelp())
   .option('--no-presets', 'Skip comparing with HTML Minifier’s presets')
   .option('--viewports <list>', 'Viewports to render at', viewports, VIEWPORTS_DEFAULT)
   .option('--concurrency <count>', 'Pages to render at once', positiveInteger, CONCURRENCY_DEFAULT)
@@ -77,6 +88,7 @@ const program = new Command()
   .option('-v, --verbose', 'Show every page and viewport in details, progress line by line where STDERR isn’t a terminal, and more notes')
   .action(async (inputs, settings, command) => {
     if (settings.verbose) progress = createProgress({ verbose: true });
+    if (process.stderr.isTTY && !settings.json) progress.log(`${INTRO}\n`);
     const interactive = Boolean(process.stdin.isTTY && process.stdout.isTTY) && !settings.output && !settings.json;
     // Asked while pages are evaluated, for the offers at the end
     const latest = interactive && !process.env.CI && !process.env.NO_UPDATE_NOTIFIER ? fetchLatestVersion('html-minifier') : null;
@@ -87,6 +99,7 @@ const program = new Command()
       fileConfig = CONFIG_FILE_DEFAULT;
       progress.log(`Using config file ${CONFIG_FILE_DEFAULT}`);
     }
+    checkCandidates([...settings.keep, ...settings.skip]);
     const { base, values, preserved, ignored } = splitConfig(fileConfig ? await readConfig(fileConfig) : {});
     if (ignored.length && settings.verbose) progress.log(`Not carried from ${fileConfig}: ${ignored.join(', ')}`);
     /** @param {'fileExt' | 'ignoreDir'} name */
@@ -114,6 +127,7 @@ const program = new Command()
       const result = await customize(pages, {
         base,
         values,
+        pagesFound: pages.found,
         metric: settings.metric,
         keep: settings.keep,
         skip: settings.skip,

package/package.json

@@ -39,7 +39,8 @@
     "minification",
     "minifier",
     "optimization",
-    "setup"
+    "setup",
+    "wizard"
   ],
   "license": "MIT",
   "name": "html-minifier-setup",
@@ -56,5 +57,5 @@
     "test:watch": "node --test --watch \"test/**/*.test.js\""
   },
   "type": "module",
-  "version": "0.1.0"
+  "version": "0.2.0"
 }

package/src/catalog.js

@@ -4,6 +4,7 @@
 import fs from 'node:fs';
 import path from 'node:path';
 import { fileURLToPath } from 'node:url';
+import { presets as PRESETS } from 'html-minifier';
 
 /**
  * @typedef {object} Candidate
@@ -107,6 +108,17 @@ export const EXCLUDED = {
 
 const CLASSIFIED = new Set([...CANDIDATES.map((candidate) => candidate.name), ...MODIFIERS, ...CARRIED, ...PRESERVED, ...Object.keys(EXCLUDED)]);
 
+/**
+ * Fails on names that aren’t candidates, as options to keep or skip
+ * @param {string[]} names
+ */
+export function checkCandidates(names) {
+  const known = CANDIDATES.map((candidate) => candidate.name);
+  for (const name of names) {
+    if (!known.includes(name)) throw new Error(`Unknown option: ${name} (options that can be kept or skipped: ${known.join(', ')})`);
+  }
+}
+
 /**
  * Options without a place here
  * @param {string[]} options
@@ -143,6 +155,11 @@ export function splitConfig(config) {
   /** @type {string[]} */
   const ignored = [];
   const names = new Set(CANDIDATES.map((candidate) => candidate.name));
+  // A preset can set how input is read, too (e.g., `caseSensitive`)
+  const preset = typeof config.preset === 'string' ? /** @type {Record<string, Record<string, unknown>>} */ (PRESETS)[config.preset.toLowerCase()] : undefined;
+  for (const name of CARRIED) {
+    if (preset && name in preset) base[name] = preset[name];
+  }
   for (const [name, value] of Object.entries(config)) {
     if (CARRIED.includes(name)) {
       base[name] = value;

package/src/equivalence.js

@@ -1,5 +1,6 @@
-// HTML Minifier’s own check that output means the same to browsers as its source, which catches
-// minifier bugs rather than what a site relies on—used where available, never duplicated here
+// For development: HTML Minifier’s own check that output means the same to browsers
+// as its source, which catches minifier bugs rather than what a site relies on—
+// used where an HTML Minifier checkout sits next to this one, never duplicated here
 
 import fs from 'node:fs';
 import { fileURLToPath, pathToFileURL } from 'node:url';
@@ -11,11 +12,9 @@ import { fileURLToPath, pathToFileURL } from 'node:url';
  * @prop {(input: string, output: string, options: Record<string, unknown>) => string[]} compare Differences found, if any
  */
 
-// @@ Import `html-minifier/equivalence` alone once HTML Minifier exports its check
-const SPECIFIER_EXPORT = 'html-minifier/equivalence';
-const FILE_SIBLING = fileURLToPath(new URL('../../html-minifier/test/equivalence/compare.js', import.meta.url));
+const FILE_COMPARE = fileURLToPath(new URL('../../html-minifier/test/equivalence/compare.js', import.meta.url));
 // Lists the options the check is run with
-const FILE_SIBLING_RUNNER = fileURLToPath(new URL('../../html-minifier/test/equivalence/runner.js', import.meta.url));
+const FILE_RUNNER = fileURLToPath(new URL('../../html-minifier/test/equivalence/runner.js', import.meta.url));
 
 /**
  * @param {unknown} difference
@@ -40,29 +39,19 @@ function versionOf(url) {
 }
 
 /**
- * The installed HTML Minifier’s check, or that of an HTML Minifier checkout next to this
- * package; `null` where neither is available
+ * The check of an HTML Minifier checkout next to this one; `null` where there is none
  * @returns {Promise<Equivalence | null>}
  */
 export async function loadEquivalence() {
-  /** @type {[string, () => Promise<any>][]} */
-  const sources = [
-    [SPECIFIER_EXPORT, () => import(SPECIFIER_EXPORT)],
-    [FILE_SIBLING, async () => {
-      if (!fs.existsSync(FILE_SIBLING) || !fs.existsSync(FILE_SIBLING_RUNNER)) return null;
-      const [compare, runner] = await Promise.all([import(pathToFileURL(FILE_SIBLING).href), import(pathToFileURL(FILE_SIBLING_RUNNER).href)]);
-      return { ...runner, ...compare };
-    }]
-  ];
-  for (const [source, load] of sources) {
-    const module = await load().catch(() => null);
-    if (typeof module?.compareMarkup !== 'function' || !module.OPTIONS_STRUCTURAL || !module.OPTIONS_CODE) continue;
-    const supported = new Set([...Object.keys(module.OPTIONS_STRUCTURAL), ...Object.keys(module.OPTIONS_CODE)]);
-    return {
-      source: source === FILE_SIBLING ? `${source} (HTML Minifier ${versionOf(new URL('../../package.json', pathToFileURL(FILE_SIBLING)))}, minifying with installed ${versionOf(new URL(import.meta.resolve('html-minifier/package.json')))})` : source,
-      supports: (name) => supported.has(name),
-      compare: (input, output, options) => /** @type {unknown[]} */ (module.compareMarkup(input, output, options)).map(describe)
-    };
-  }
-  return null;
+  if (!fs.existsSync(FILE_COMPARE) || !fs.existsSync(FILE_RUNNER)) return null;
+  const module = await Promise.all([import(pathToFileURL(FILE_RUNNER).href), import(pathToFileURL(FILE_COMPARE).href)])
+    .then(([runner, compare]) => ({ ...runner, ...compare }))
+    .catch(() => null);
+  if (typeof module?.compareMarkup !== 'function' || !module.OPTIONS_STRUCTURAL || !module.OPTIONS_CODE) return null;
+  const supported = new Set([...Object.keys(module.OPTIONS_STRUCTURAL), ...Object.keys(module.OPTIONS_CODE)]);
+  return {
+    source: `${FILE_COMPARE} (HTML Minifier ${versionOf(new URL('../../package.json', pathToFileURL(FILE_COMPARE)))}, minifying with installed ${versionOf(new URL(import.meta.resolve('html-minifier/package.json')))})`,
+    supports: (name) => supported.has(name),
+    compare: (input, output, options) => /** @type {unknown[]} */ (module.compareMarkup(input, output, options)).map(describe)
+  };
 }

package/src/hazards.js

@@ -52,8 +52,8 @@ const COMMENTS_MEANINGFUL = [
   [/^\s*(?:Instance|Template)(?:Begin|End)\b|^\s*#(?:Begin|End)(?:Library|Template)/, 'template editor region']
 ];
 
-export const OPTIONS_ATTRIBUTES = new Set(['collapseAttributeWhitespace', 'collapseBooleanAttributes', 'removeDefaultTypeAttributes', 'removeEmptyAttributes', 'removeRedundantAttributes', 'sortClassNames']);
-export const OPTIONS_WHITESPACE = new Set(['collapseInlineTagWhitespace', 'collapseNoBreakSpaces', 'collapseWhitespace', 'trimCustomFragments']);
+const OPTIONS_ATTRIBUTES = new Set(['collapseAttributeWhitespace', 'collapseBooleanAttributes', 'removeDefaultTypeAttributes', 'removeEmptyAttributes', 'removeRedundantAttributes', 'sortClassNames']);
+const OPTIONS_WHITESPACE = new Set(['collapseInlineTagWhitespace', 'collapseNoBreakSpaces', 'collapseWhitespace', 'trimCustomFragments']);
 
 /** @param {string} text */
 const clip = (text, length = 60) => text.length > length ? `${text.slice(0, length - 1)}…` : text;

package/src/index.js

@@ -1,5 +1,4 @@
 export { CANDIDATES, CARRIED, EXCLUDED, FALLBACKS, MODIFIERS, PRESERVED, splitConfig } from './catalog.js';
-export { loadEquivalence } from './equivalence.js';
 export { analyzePage, findHazards } from './hazards.js';
 export { measure } from './measure.js';
 export { createPage, loadPages } from './pages.js';

package/src/measure.js

@@ -3,8 +3,8 @@ import zlib from 'node:zlib';
 
 // As in HTML Minifier’s benchmarks: common defaults for on-the-fly compression (quality 11, for
 // files precompressed at build time, takes seconds per large file)
-export const GZIP_LEVEL = 6;
-export const BROTLI_QUALITY = 6;
+const GZIP_LEVEL = 6;
+const BROTLI_QUALITY = 6;
 
 const brotliCompress = promisify(zlib.brotliCompress);
 const gzipCompress = promisify(zlib.gzip);
@@ -15,8 +15,18 @@ const gzipCompress = promisify(zlib.gzip);
 /** @type {Metric[]} */
 export const METRICS = ['raw', 'gzip', 'brotli'];
 
+// For prose
+/** @type {Record<Metric, string>} */
+export const NAMES_METRIC = { raw: 'uncompressed', gzip: 'Gzip', brotli: 'Brotli' };
+
+/**
+ * How much an option saves by the metric, in prose (e.g., “saves nothing with Gzip”)
+ * @param {Metric} metric
+ */
+export const byMetric = (metric) => metric === 'raw' ? 'uncompressed' : `with ${NAMES_METRIC[metric]}`;
+
 /** @type {Sizes} */
-export const SIZES_ZERO = Object.freeze({ raw: 0, gzip: 0, brotli: 0 });
+const SIZES_ZERO = Object.freeze({ raw: 0, gzip: 0, brotli: 0 });
 
 /**
  * Size by one metric as servers would send it, compressed off the main thread

package/src/minify-worker.js

@@ -0,0 +1,14 @@
+// Minifies what the pool in workers.js sends, answering with the output or the error message
+
+import { parentPort } from 'node:worker_threads';
+import { minify } from 'html-minifier';
+
+const port = /** @type {import('node:worker_threads').MessagePort} */ (parentPort);
+
+port.on('message', async (/** @type {{html: string, options: Record<string, unknown>}} */ { html, options }) => {
+  try {
+    port.postMessage({ output: await minify(html, { ...options, log: () => {} }) });
+  } catch (err) {
+    port.postMessage({ error: err instanceof Error ? err.message : String(err) });
+  }
+});

package/src/pages.js

@@ -102,37 +102,77 @@ async function fetchSameOrigin(pageUrl, ref) {
   return null;
 }
 
+// Golden ratio, whose multiples spread picks evenly over a range
+const SPREAD = 0.6180339887;
+
+/**
+ * Items in an order whose every start is spread across them, rather than their first ones
+ * @template T
+ * @param {T[]} items
+ * @returns {T[]}
+ */
+function spread(items) {
+  /** @type {Set<number>} */
+  const picked = new Set();
+  // Bounded, as the last gaps can take a while to hit
+  for (let step = 0; step < 4 * items.length && picked.size < items.length; step++) {
+    picked.add(Math.floor(((step * SPREAD) % 1) * items.length));
+  }
+  for (let index = 0; index < items.length; index++) picked.add(index);
+  return [...picked].map((index) => /** @type {T} */ (items[index]));
+}
+
+/**
+ * Takes from each list in turn, up to `limit` items
+ * @param {string[][]} lists
+ * @param {number} limit
+ */
+function interleave(lists, limit) {
+  /** @type {string[]} */
+  const items = [];
+  let active = lists.filter((list) => list.length);
+  for (let round = 0; active.length && items.length < limit; round++) {
+    for (const list of active) {
+      if (items.length < limit) items.push(/** @type {string} */ (list[round]));
+    }
+    active = active.filter((list) => list.length > round + 1);
+  }
+  return items;
+}
+
 /**
- * Picks pages round-robin across directories, so that a sample covers a site’s sections (and
- * so, likely, its templates) rather than the first directory’s pages
+ * Picks pages across a site’s sections first and then across the folders within them, so that
+ * a sample covers its templates rather than one section’s first pages; the same paths always
+ * give the same sample
  * @param {string[]} paths
  * @param {number} count
  * @returns {string[]}
  */
 export function samplePaths(paths, count) {
   if (paths.length <= count) return paths;
-  /** @type {Map<string, string[]>} */
-  const byDir = new Map();
-  for (const p of [...paths].sort()) {
-    const dir = path.dirname(p);
-    const queue = byDir.get(dir) ?? [];
-    queue.push(p);
-    byDir.set(dir, queue);
-  }
-  // Within a directory, a golden-ratio order spreads picks rather than taking its first pages
-  const queues = [...byDir.values()].map((queue) => queue
-    .map((item, index) => ({ item, key: (index * 0.6180339887) % 1 }))
-    .sort((a, b) => a.key - b.key)
-    .map(({ item }) => item));
-  /** @type {string[]} */
-  const sample = [];
-  for (let round = 0; sample.length < count; round++) {
-    for (const queue of queues) {
-      const item = queue[round];
-      if (item !== undefined && sample.length < count) sample.push(item);
+  /** @typedef {{files: string[], folders: Map<string, Folder>}} Folder */
+  /** @type {Folder} */
+  const root = { files: [], folders: new Map() };
+  for (const item of [...paths].sort()) {
+    const segments = item.split(/[\\/]/);
+    let folder = root;
+    for (const segment of segments.slice(0, -1)) {
+      let next = folder.folders.get(segment);
+      if (!next) {
+        next = { files: [], folders: new Map() };
+        folder.folders.set(segment, next);
+      }
+      folder = next;
     }
+    folder.files.push(item);
   }
-  return sample;
+  /**
+   * The first `count` pages of the folder, in the order they’re picked
+   * @param {Folder} folder
+   * @returns {string[]}
+   */
+  const order = (folder) => interleave([spread(folder.files), ...spread([...folder.folders.values()]).map(order)], count);
+  return order(root);
 }
 
 /**
@@ -185,10 +225,10 @@ export async function findOutsideLinks(root, ignore = ['node_modules']) {
 }
 
 /**
- * Reads pages from files, directories, and URLs
+ * Reads pages from files, directories, and URLs, sampled where there are more
  * @param {string[]} inputs
  * @param {{extensions?: string[], ignore?: string[], sample?: number}} [settings]
- * @returns {Promise<Page[]>}
+ * @returns {Promise<Page[] & {found: number}>} `found`: pages found before sampling
  */
 export async function loadPages(inputs, { extensions = EXTENSIONS_DEFAULT, ignore = ['node_modules'], sample = 20 } = {}) {
   // As HTML Minifier reads them, with `*` for all files
@@ -213,7 +253,7 @@ export async function loadPages(inputs, { extensions = EXTENSIONS_DEFAULT, ignor
   }
 
   const keys = samplePaths([...files.map(({ file }) => file), ...urls], sample);
-  return Promise.all(keys.map(async (key) => {
+  const pages = await Promise.all(keys.map(async (key) => {
     if (RE_URL_INPUT.test(key)) {
       const response = await fetch(key, { signal: AbortSignal.timeout(30000) });
       if (!response.ok) throw new Error(`Couldn’t fetch ${key}: ${response.status}`);
@@ -227,4 +267,5 @@ export async function loadPages(inputs, { extensions = EXTENSIONS_DEFAULT, ignor
       root
     });
   }));
+  return Object.assign(pages, { found: files.length + urls.length });
 }

package/src/render.js

@@ -30,6 +30,7 @@ import { isInside } from './pages.js';
 /** @typedef {{width: number, height: number}} Viewport */
 /** @typedef {{shot: Buffer, text: string, errors: string[], blocked: string[]}} Capture */
 
+// @@ Probe a higher or core-based default, weighing speed against memory and inconclusive renders
 export const CONCURRENCY_DEFAULT = 4;
 
 /** @type {Viewport[]} */
@@ -151,7 +152,7 @@ export const combineOriginals = (first, second) => ({
 // Playwright’s Chromium, or else browsers people tend to have, as original and minified pages are
 // compared in the same browser
 /** @type {Browser[]} */
-export const BROWSERS = [{ name: 'Chromium' }, { name: 'Chrome', channel: 'chrome' }, { name: 'Edge', channel: 'msedge' }];
+const BROWSERS = [{ name: 'Chromium' }, { name: 'Chrome', channel: 'chrome' }, { name: 'Edge', channel: 'msedge' }];
 
 /** The path of Playwright’s command line interface, if installed */
 export function playwrightCli() {

package/src/report.js

@@ -1,18 +1,18 @@
 import { presets as PRESETS } from 'html-minifier';
 import { VISIBLE_BY_DESIGN } from './catalog.js';
 import { paint } from './color.js';
-import { METRICS } from './measure.js';
+import { METRICS, NAMES_METRIC } from './measure.js';
 import { MESSAGE_RENDERS_SAME } from './select.js';
 
-export const URL_SCHEMA = 'https://raw.githubusercontent.com/j9t/html-minifier/main/html-minifier.schema.json';
+const URL_SCHEMA = 'https://raw.githubusercontent.com/j9t/html-minifier/main/html-minifier.schema.json';
 
 const MAX_LISTED = 3;
 
 // Render differences, worded so as not to read as errors
-const LABEL_RENDER = 'looks different';
+const LABEL_RENDER = 'Looks different';
 
 /** @param {string} level */
-const labelOf = (level) => level === 'render' ? LABEL_RENDER : level;
+const labelOf = (level) => level === 'render' ? LABEL_RENDER : `${level.charAt(0).toUpperCase()}${level.slice(1)}`;
 
 // Share of the size below which whitespace collapsing suggests minified pages
 const SHARE_MINIFIED = 0.01;
@@ -118,10 +118,10 @@ function condenseReasons(reasons) {
       const pages = [...new Set(run.flatMap((reason) => reason.page ?? []))];
       const count = `${pages.length} page${pages.length === 1 ? '' : 's'}`;
       if (pages.join('\u0000') === pagesRenderedBefore) {
-        lines.push(`${LABEL_RENDER} · same ${count}`);
+        lines.push(`${LABEL_RENDER} on the same ${count}`);
       } else {
         const example = /** @type {import('./select.js').Reason} */ (run.find((reason) => reason.message.includes('“')) ?? run[0]);
-        lines.push(`${LABEL_RENDER} · ${count} (${listShort(pages)}), e.g., ${example.page ? `${example.page}: ` : ''}${example.message}`);
+        lines.push(`${LABEL_RENDER} on ${count} (${listShort(pages)}), e.g., ${example.page ? `${example.page}: ` : ''}${example.message}`);
       }
       pagesRenderedBefore = pages.join('\u0000');
       continue;
@@ -135,7 +135,7 @@ function condenseReasons(reasons) {
       byMessage.set(key, entry);
     }
     for (const { level: levelEntry, message, pages } of byMessage.values()) {
-      lines.push(`${levelEntry}${pages.length ? ` · ${listShort(pages)}` : ''}: ${message}`);
+      lines.push(`${labelOf(levelEntry)}${pages.length ? ` (${listShort(pages)})` : ''}: ${message}`);
     }
   }
   return lines;
@@ -174,12 +174,13 @@ export function formatReport(result, { renderReason = '', color = false, verbose
       ? `${renderedAs}: The recommended configuration looks the same as the original pages`
       : paint('warning', `${renderedAs}: The recommended configuration may not look the same as the original pages (see notes)`, color);
   const lines = [
-    `• Evaluated ${result.pages.length} page${result.pages.length === 1 ? '' : 's'}${dirShared ? ` in ${dirShared}` : ''}, judging gains by ${metric === 'raw' ? 'uncompressed' : metric} size`,
+    `• Evaluated ${result.pagesFound ? `${result.pages.length} of ${result.pagesFound.toLocaleString('en-US')} pages` : `${result.pages.length} page${result.pages.length === 1 ? '' : 's'}`}${dirShared ? ` in ${dirShared}` : ''}${result.pagesFound ? ' (use `--sample` to evaluate more)' : ''}, judging gains by ${NAMES_METRIC[metric]} size`,
     result.rendered
       ? `• ${renderSummary}`
       : `• Not render-checked${renderReason ? `: ${renderReason}` : ''}—options that need a render check aren’t recommended`,
     ...result.rendered && blocked.length ? [`• ${paint('warning', `Rendered without ${blocked.length} file${blocked.length === 1 ? '' : 's'} that symbolic links lead to outside the site (${listShort(blocked)}), which may change how pages look—\`--follow-links\` follows them`, color)}`] : [],
-    result.equivalence ? `• Equivalence-checked with ${result.equivalence}` : '• Not equivalence-checked',
+    // Only in development
+    ...result.equivalence ? [`• Equivalence-checked with ${result.equivalence}`] : [],
     '',
     table([
       ['Option', 'Status', 'Saves alone', 'Contributes', 'Variant'],
@@ -192,12 +193,12 @@ export function formatReport(result, { renderReason = '', color = false, verbose
       ])
     ]),
     '',
-    ...METRICS.map((m) => `${m === 'raw' ? 'Uncompressed' : m === 'gzip' ? 'Gzip' : 'Brotli'}: ${formatChange(result.baseline[m], result.final[m], color)}`)
+    ...METRICS.map((m) => `${m === 'raw' ? 'Uncompressed' : NAMES_METRIC[m]}: ${formatChange(result.baseline[m], result.final[m], color)}`)
   ];
 
   if (result.presets.length) {
     const finalMetric = result.final[metric];
-    lines.push('', `Compared with HTML Minifier’s presets (${metric === 'raw' ? 'uncompressed' : metric}):`, table(result.presets.map((preset) => {
+    lines.push('', `Compared with HTML Minifier’s presets (${NAMES_METRIC[metric]}):`, table(result.presets.map((preset) => {
       const share = finalMetric ? (preset.size / finalMetric - 1) * 100 : 0;
       const size = preset.errors ? `fails on ${preset.errors} of ${result.pages.length}` : formatBytes(preset.size);
       const relative = preset.errors ? '' : Math.abs(share) < 0.05 ? 'same size' : `${Math.abs(share).toFixed(1)}% ${share > 0 ? 'larger' : 'smaller'}`;
@@ -232,12 +233,12 @@ export function formatReport(result, { renderReason = '', color = false, verbose
     for (const evaluation of explained) {
       lines.push(`  ${evaluation.name} (${formatStatus(evaluation.status)})`);
       const reasonLines = verbose
-        ? evaluation.reasons.map((reason) => `${labelOf(reason.level)}${reason.page ? ` · ${reason.page}` : ''}: ${reason.message}`)
+        ? evaluation.reasons.map((reason) => `${labelOf(reason.level)}${reason.page ? ` (${reason.page})` : ''}: ${reason.message}`)
         : condenseReasons(evaluation.reasons);
       condensed ||= reasonLines.length < evaluation.reasons.length;
       if (evaluation.reasons.some((reason) => reason.level === 'render')) {
         const design = VISIBLE_BY_DESIGN[evaluation.name];
-        lines.push(`    why: ${design ? `${design}—that’s how the option works, not an error` : 'Not meant to change how pages look—possibly an HTML Minifier bug worth reporting'}`);
+        lines.push(`    Why: ${design ? `${design}—that’s how the option works, not an error` : 'Not meant to change how pages look—possibly an HTML Minifier bug worth reporting'}`);
       }
       lines.push(...reasonLines.map((line) => `    ${line}`));
     }

package/src/select.js

@@ -2,10 +2,11 @@
 // is measured and checked on its own, then the accepted ones are rendered together, isolating
 // options that break rendering on their own or only in combination
 
-import { minify as minifyHM, presets as PRESETS } from 'html-minifier';
-import { CANDIDATES, FALLBACKS, readSchema, unclassified } from './catalog.js';
+import { presets as PRESETS } from 'html-minifier';
+import { CANDIDATES, checkCandidates, FALLBACKS, readSchema, unclassified } from './catalog.js';
 import { analyzePage, findHazards } from './hazards.js';
-import { sizeOf, sumSizes } from './measure.js';
+import { byMetric, sizeOf, sumSizes } from './measure.js';
+import { minifyInWorkers } from './workers.js';
 
 /**
  * @typedef {object} Reason
@@ -33,8 +34,9 @@ import { sizeOf, sumSizes } from './measure.js';
  * @prop {import('./measure.js').Sizes} final
  * @prop {Evaluation[]} evaluations
  * @prop {string[]} pages
+ * @prop {number | null} pagesFound Pages the sample was taken from, where it left any out
  * @prop {boolean} rendered
- * @prop {string | null} equivalence Source of the equivalence check, if used
+ * @prop {string | null} equivalence Source of the equivalence check, if used (in development)
  * @prop {string[]} notes
  * @prop {PresetComparison[]} presets
  * @prop {boolean} renderedSame Whether the configuration was found to look the same as the original pages, with every file they requested
@@ -61,8 +63,9 @@ import { sizeOf, sumSizes } from './measure.js';
  * @prop {string[]} [keep] Candidates to accept regardless of risks and gains
  * @prop {string[]} [skip] Candidates not to evaluate
  * @prop {import('./render.js').Renderer | null} [renderer]
- * @prop {import('./equivalence.js').Equivalence | null} [equivalence]
+ * @prop {import('./equivalence.js').Equivalence | null} [equivalence] HTML Minifier’s check, in development
  * @prop {(html: string, options: Record<string, unknown>) => Promise<string>} [minify]
+ * @prop {number} [pagesFound] Pages the given ones were sampled from
  * @prop {boolean} [presets] Whether to compare the result with HTML Minifier’s presets
  * @prop {{version: string, options: string[]} | null} [schema] HTML Minifier’s options, by default the installed version’s
  * @prop {(event: import('./progress.js').ProgressEvent) => void} [onProgress]
@@ -97,16 +100,14 @@ export async function customize(pagesGiven, settings = {}) {
     skip = [],
     renderer = null,
     equivalence = null,
-    minify = minifyHM,
+    minify = minifyInWorkers,
+    pagesFound,
     presets = true,
     schema = readSchema(),
     onProgress = () => {}
   } = settings;
 
-  const names = CANDIDATES.map((candidate) => candidate.name);
-  for (const name of [...keep, ...skip]) {
-    if (!names.includes(name)) throw new Error(`Unknown option: ${name} (options that can be kept or skipped: ${names.join(', ')})`);
-  }
+  checkCandidates([...keep, ...skip]);
 
   // By page rather than by ID, as pages from different roots may share one
   /** @type {Map<import('./pages.js').Page, Map<string, Promise<string>>>} */
@@ -163,12 +164,13 @@ export async function customize(pagesGiven, settings = {}) {
   // Pages the minifier can’t read with the carried options are left out
   /** @type {import('./pages.js').Page[]} */
   const pages = [];
-  for (const page of pagesGiven) {
-    try {
-      await run(page, base);
+  const readable = await Promise.allSettled(pagesGiven.map((page) => run(page, base)));
+  for (const [index, page] of pagesGiven.entries()) {
+    const outcome = readable[index];
+    if (outcome?.status === 'fulfilled') {
       pages.push(page);
-    } catch (err) {
-      notes.add(`Left out ${page.id}, which the minifier can’t read: ${messageOf(err)}`);
+    } else {
+      notes.add(`Left out ${page.id}, which the minifier can’t read: ${messageOf(outcome?.reason)}`);
     }
   }
   if (!pages.length) throw new Error(`The minifier can read none of the pages (${[...notes].join('; ')})—\`continueOnParseError\` in a starting configuration may help`);
@@ -231,13 +233,15 @@ export async function customize(pagesGiven, settings = {}) {
     const reasons = [];
     let changed = false;
     let savings = 0;
+    // Started at once, to minify pages in parallel
+    const pairs = pages.map((page) => Promise.all([run(page, ctx), run(page, full)]));
+    for (const pair of pairs) pair.catch(() => {});
     for (const [index, page] of pages.entries()) {
       await onPage(index, page);
       let outBefore;
       let outAfter;
       try {
-        outBefore = await run(page, ctx);
-        outAfter = await run(page, full);
+        [outBefore, outAfter] = await /** @type {Promise<[string, string]>} */ (pairs[index]);
       } catch (err) {
         reasons.push({ level: 'error', page: page.id, message: `Minification failed: ${messageOf(err)}` });
         continue;
@@ -261,7 +265,7 @@ export async function customize(pagesGiven, settings = {}) {
     if (reasons.some((reason) => reason.level === 'error')) return { status: 'rejected', savings, reasons };
     if (keep.includes(name)) return { status: 'kept', savings, reasons };
     if (!changed) return { status: 'no-effect', savings, reasons: [{ level: 'info', message: 'No effect on these pages' }] };
-    if (savings <= 0) return { status: 'no-gain', savings, reasons: [{ level: 'info', message: `Saves nothing ${metric === 'raw' ? 'uncompressed' : `with ${metric}`}` }, ...reasons] };
+    if (savings <= 0) return { status: 'no-gain', savings, reasons: [{ level: 'info', message: `Saves nothing ${byMetric(metric)}` }, ...reasons] };
     return { status: isBlocking(reasons) ? 'rejected' : 'accepted', savings, reasons };
   }
 
@@ -422,29 +426,38 @@ export async function customize(pagesGiven, settings = {}) {
    */
   async function settleContributions() {
     let dropped = false;
-    for (let changed = true; changed;) {
-      changed = false;
+    // After a drop, only options that gained nothing are measured again, as a drop rarely makes
+    // others gain nothing; a full round then confirms
+    /** @type {Set<Evaluation> | null} */
+    let suspects = null;
+    for (;;) {
       const used = inUse();
+      const measured = suspects ? used.filter((evaluation) => suspects?.has(evaluation)) : used;
       const final = await totalOf(combine(used));
-      for (const [index, evaluation] of used.entries()) {
-        onProgress({ phase: 'contribute', detail: evaluation.name, current: index, total: used.length });
-        await yieldTurn();
+      let done = 0;
+      onProgress({ phase: 'contribute', detail: `${measured.length} options`, current: 0, total: measured.length });
+      await Promise.all(measured.map(async (evaluation) => {
         const without = used.filter((other) => other !== evaluation && requirementOf(other.name) !== evaluation.name);
         evaluation.contribution = await totalOf(combine(without)) - final;
-      }
-      const [weakest] = used
+        onProgress({ phase: 'contribute', detail: evaluation.name, current: ++done, total: measured.length });
+      }));
+      const [weakest] = measured
         .filter((evaluation) => evaluation.status === 'accepted' && (evaluation.contribution ?? 0) <= 0)
         .sort((a, b) => (a.contribution ?? 0) - (b.contribution ?? 0));
       if (weakest) {
         weakest.status = 'no-gain';
-        weakest.reasons.push({ level: 'info', message: `Saves nothing alongside the other options (${weakest.contribution} B by ${metric})` });
+        weakest.reasons.push({ level: 'info', message: `Saves nothing alongside the other options (${weakest.contribution} B ${byMetric(metric)})` });
         delete weakest.contribution;
         dropDependents(weakest);
-        changed = true;
         dropped = true;
+        const remaining = inUse().filter((evaluation) => evaluation.status === 'accepted' && (evaluation.contribution ?? 0) <= 0);
+        suspects = remaining.length ? new Set(remaining) : null;
+      } else if (suspects) {
+        suspects = null;
+      } else {
+        return dropped;
       }
     }
-    return dropped;
   }
 
   // 3. What each option contributes; as fewer options may render differently, too, rendering
@@ -460,18 +473,16 @@ export async function customize(pagesGiven, settings = {}) {
   onProgress({ phase: 'presets', detail: presetsCompared.map(([name]) => name).join(', '), current: 0, total: presetsCompared.length });
   /** @type {PresetComparison[]} */
   const comparisons = await Promise.all(presetsCompared.map(async ([name, preset]) => {
+    /** @type {Record<string, unknown>} */
     const optionsPreset = { ...base, ...preset };
     /** @type {{page: import('./pages.js').Page, html: string}[]} */
     const minified = [];
-    let errors = 0;
-    for (const page of pages) {
-      try {
-        minified.push({ page, html: await run(page, optionsPreset) });
-      } catch {
-        errors++;
-      }
-      await yieldTurn();
+    const outcomes = await Promise.allSettled(pages.map((page) => run(page, optionsPreset)));
+    for (const [index, page] of pages.entries()) {
+      const outcome = outcomes[index];
+      if (outcome?.status === 'fulfilled') minified.push({ page, html: outcome.value });
     }
+    const errors = pages.length - minified.length;
     const verdicts = renderer ? await Promise.all(minified.map(async ({ page, html }) => {
       const verdict = await compare(page, html);
       for (const note of verdict.notes) notes.add(`${page.id}: ${note}`);
@@ -484,7 +495,11 @@ export async function customize(pagesGiven, settings = {}) {
       failing: verdicts ? verdicts.filter((verdict) => !verdict.equal && !verdict.inconclusive).length : null,
       inconclusive: verdicts ? verdicts.filter((verdict) => verdict.inconclusive).length : null,
       errors,
-      rejected: Object.keys(preset).filter((key) => /** @type {Record<string, unknown>} */ (preset)[key] && evaluations.get(key)?.status === 'rejected')
+      rejected: Object.keys(preset).filter((key) => {
+        const evaluation = evaluations.get(key);
+        // Or in use only as a gentler variant than the preset’s
+        return optionsPreset[key] && (evaluation?.status === 'rejected' || (evaluation?.variant && Object.keys(evaluation.options).some((option) => !optionsPreset[option])));
+      })
     };
   }));
   const [baseline, final] = await measuring;
@@ -496,6 +511,7 @@ export async function customize(pagesGiven, settings = {}) {
     final,
     evaluations: [...evaluations.values()],
     pages: pages.map((page) => page.id),
+    pagesFound: pagesFound !== undefined && pagesFound > pagesGiven.length ? pagesFound : null,
     rendered: Boolean(renderer),
     equivalence: equivalence?.source ?? null,
     notes: [...notes],

package/src/setup.js

@@ -13,7 +13,7 @@ export const CONFIG_FILE = 'html-minifier.config.json';
 const LOCKFILES = [['pnpm-lock.yaml', 'pnpm'], ['yarn.lock', 'yarn'], ['bun.lock', 'bun'], ['bun.lockb', 'bun'], ['package-lock.json', 'npm']];
 
 /** @type {Record<PackageManager, string[]>} */
-export const INSTALL_ARGS = {
+const INSTALL_ARGS = {
   npm: ['install', '--save-dev'],
   pnpm: ['add', '--save-dev'],
   yarn: ['add', '--dev'],
@@ -200,7 +200,7 @@ export async function offerSetup({ config, dirProject, configPreset = null, cwd,
     const dirInstalled = path.dirname(path.dirname(found.dir));
     const where = dirInstalled === cwd ? '' : ` in ${describeDir(dirInstalled, cwd)}`;
     if (!versionLatest || !isNewer(versionLatest, found.version)) {
-      log(`HTML Minifier ${found.version} is installed${where}${versionLatest ? ' (latest)' : ''}`);
+      log(`HTML Minifier ${found.version}${versionLatest ? ' (latest)' : ''} is installed${where}`);
     } else if (isYes(await ask(`Update HTML Minifier ${found.version}${where} to ${versionLatest} (latest)? (y/N) `))) {
       if (!await install(dirInstalled, detectPackageManager(dirInstalled))) log('Update failed');
     }

package/src/workers.js

@@ -0,0 +1,122 @@
+// Minifies on worker threads, so that pages minify in parallel and the main thread stays free for
+// static checks, rendering, and progress
+
+import os from 'node:os';
+import { Worker } from 'node:worker_threads';
+import { minify } from 'html-minifier';
+
+// One core is left to the main thread; more workers mostly add memory, as each loads the minifier
+const SIZE_DEFAULT = Math.max(1, Math.min(os.availableParallelism() - 1, 8));
+const FILE_WORKER = new URL('./minify-worker.js', import.meta.url);
+
+/**
+ * @typedef {object} Job
+ * @prop {string} html
+ * @prop {Record<string, unknown>} options
+ * @prop {(output: string) => void} resolve
+ * @prop {(err: Error) => void} reject
+ */
+
+/** @typedef {{worker: Worker, job: Job | null}} Slot */
+
+/**
+ * @param {string} html
+ * @param {Record<string, unknown>} options
+ */
+const minifyHere = (html, options) => minify(html, { ...options, log: () => {} });
+
+/**
+ * Workers started as needed, up to `size`, which let the process exit while idle
+ * @param {{size?: number}} [settings]
+ */
+export function createMinifierPool({ size = SIZE_DEFAULT } = {}) {
+  /** @type {Slot[]} */
+  const slots = [];
+  /** @type {Job[]} */
+  const queue = [];
+
+  /** @param {Slot} slot */
+  const release = (slot) => {
+    slot.job = null;
+    slot.worker.unref();
+    pump();
+  };
+
+  const spawn = () => {
+    /** @type {Slot} */
+    // Without the parent’s Node flags, some of which (e.g., `--input-type`) workers can’t take
+    const slot = { worker: new Worker(FILE_WORKER, { execArgv: [] }), job: null };
+    slot.worker.unref();
+    slot.worker.on('message', (/** @type {{output?: string, error?: string}} */ data) => {
+      const { job } = slot;
+      if (data.error === undefined) {
+        job?.resolve(data.output ?? '');
+      } else {
+        job?.reject(new Error(data.error));
+      }
+      release(slot);
+    });
+    // Where a worker fails as a whole (e.g., out of memory), its job fails and others take over
+    slot.worker.on('error', (/** @type {Error} */ err) => {
+      const index = slots.indexOf(slot);
+      if (index !== -1) slots.splice(index, 1);
+      slot.job?.reject(err);
+      slot.job = null;
+      pump();
+    });
+    slots.push(slot);
+    return slot;
+  };
+
+  function pump() {
+    while (queue.length) {
+      const slot = slots.find((candidate) => !candidate.job) ?? (slots.length < size ? spawn() : null);
+      if (!slot) return;
+      const job = /** @type {Job} */ (queue.shift());
+      slot.job = job;
+      slot.worker.ref();
+      slot.worker.postMessage({ html: job.html, options: job.options });
+    }
+  }
+
+  return {
+    size,
+    /**
+     * @param {string} html
+     * @param {Record<string, unknown>} options `log` is left out, as workers log nothing
+     * @returns {Promise<string>}
+     */
+    minify(html, options) {
+      const rest = { ...options };
+      delete rest.log;
+      // Functions, for one, can’t be sent to workers
+      try {
+        structuredClone(rest);
+      } catch {
+        return minifyHere(html, rest);
+      }
+      return new Promise((resolve, reject) => {
+        queue.push({ html, options: rest, resolve, reject });
+        pump();
+      });
+    },
+    async close() {
+      const jobs = [...queue.splice(0), ...slots.flatMap((slot) => slot.job ? [slot.job] : [])];
+      for (const job of jobs) job.reject(new Error('Minifier pool closed'));
+      await Promise.all(slots.splice(0).map((slot) => {
+        slot.job = null;
+        return slot.worker.terminate();
+      }));
+    }
+  };
+}
+
+/** @type {ReturnType<typeof createMinifierPool> | null} */
+let poolShared = null;
+
+/**
+ * Minifies on a pool shared by all runs, which needs no closing
+ * @param {string} html
+ * @param {Record<string, unknown>} options
+ */
+export const minifyInWorkers = (html, options) => (poolShared ??= createMinifierPool()).minify(html, options);