@turntrout/subfont 1.0.0

package/CHANGELOG.md

@@ -0,0 +1,35 @@
+# Changelog
+
+## 1.0.0 -- Hard fork from [Munter/subfont](https://github.com/Munter/subfont)
+
+Published as `@turntrout/subfont`. Based on Munter/subfont v7.2.3.
+
+### Performance
+
+On [TurnTrout.com](https://github.com/alexander-turner/TurnTrout.com) (382 pages), font subsetting dropped from 111 minutes to 28 minutes:
+
+|                                                                                      | Version        | Duration |
+| ------------------------------------------------------------------------------------ | -------------- | -------- |
+| [Before](https://github.com/alexander-turner/TurnTrout.com/actions/runs/23470135763) | Munter/subfont | 111 min  |
+| [After](https://github.com/alexander-turner/TurnTrout.com/actions/runs/23518006824)  | This fork      | 28 min   |
+
+### Breaking changes
+
+- **woff2-only.** Removed `--browsers` and `--formats`. Every browser supports woff2.
+- **Always-on variable font instancing.** Removed `--instance`. If you use weights 400 and 700 from a 100-900 variable font, the subset shrinks to just that range automatically.
+- **Removed legacy flags:** `--skip-source-map-processing`, `--dryrun`/`--dry`/`--canonicalroot`/`--sourceMaps` aliases, and v5 flag validation.
+
+### New features
+
+- **`--cache [dir]`** -- Cache subset results to disk. Speeds up repeat builds.
+- **`--chrome-flags`** -- Custom flags for headless Chrome with `--dynamic`.
+- **`--concurrency N`** -- Control worker thread count for parallel font tracing.
+- **Parallel font tracing** -- Worker pool (up to 8 threads). Pages sharing identical CSS are traced once.
+- **`--root` validation** -- Fails early with a clear error.
+- **Timing summary** -- Printed after every run.
+- **Better `--dry-run`** -- Detailed preview of files, sizes, and CSS changes.
+
+### Bug fixes
+
+- Fixed crash on invalid/corrupt font files during instancing.
+- Fixed incorrect axis range computation for variable fonts.

package/CLAUDE.md

@@ -0,0 +1,53 @@
+# CLAUDE.md — subfont
+
+## Project Overview
+
+subfont is a CLI tool and Node.js library that speeds up initial page paint by automatically subsetting local or Google fonts and loading them optimally. It uses puppeteer to trace font usage across pages and generates optimized font subsets.
+
+## Development Commands
+
+```bash
+pnpm install          # Install dependencies
+pnpm test             # Run mocha tests + lint
+pnpm run lint         # ESLint + Prettier check
+pnpm run coverage     # Run tests with nyc coverage
+pnpm run check-coverage  # Verify coverage thresholds
+```
+
+## Code Style
+
+- **Formatter**: Prettier with single quotes, trailing commas (es5)
+- **Linter**: ESLint via neostandard + eslint-config-prettier
+- **Rules**: `prefer-template`, `prefer-const` (destructuring: all)
+- **Tests**: Mocha with `unexpected` assertion library (not chai/jest)
+- No exclusive tests (`describe.only`, `it.only`) — enforced by eslint-plugin-mocha
+
+## Project Structure
+
+- `lib/` — Source code (entry: `lib/subfont.js`, CLI: `lib/cli.js`)
+- `test/` — Mocha test files
+- `testdata/` — HTML fixtures and font files for tests
+- `cases/` — Additional test case data
+
+## Key Architecture
+
+- Built on **assetgraph** for HTML/CSS asset graph traversal
+- Uses **puppeteer-core** for headless browser font tracing
+- **font-tracer** traces which fonts are used on each page
+- **subset-font** / **harfbuzzjs** for WOFF2 subsetting
+- `lib/subsetFonts.js` — Main orchestration logic
+- `lib/FontTracerPool.js` — Manages puppeteer browser pool for parallel tracing
+
+## Testing Notes
+
+- Tests have a 5-minute timeout (configured in `.mocharc.yml`)
+- Tests use `httpception` for HTTP mocking and `unexpected` for assertions
+- Some tests require puppeteer browser binaries (installed via `pnpm install`)
+- Coverage thresholds are enforced via `nyc check-coverage`
+
+## Conventions
+
+- CommonJS modules (`require`/`module.exports`), not ESM
+- Node.js >= 18 required
+- Use `const` by default; `let` only when reassignment is needed
+- Template literals preferred over string concatenation

package/LICENSE.md

@@ -0,0 +1,7 @@
+Copyright 2017 Peter Brandt Müller
+
+Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

package/README.md

@@ -0,0 +1,93 @@
+# `@turntrout/subfont`
+
+[![Build Status](https://github.com/alexander-turner/subfont/actions/workflows/ci.yml/badge.svg)](https://github.com/alexander-turner/subfont/actions/workflows/ci.yml)
+
+A faster fork of [subfont](https://github.com/Munter/subfont) that subsets web fonts to only the characters used on your pages. Adds parallel tracing, disk caching, woff2-only output, and always-on variable font instancing.
+
+## Performance
+
+On [TurnTrout.com](https://github.com/alexander-turner/TurnTrout.com) (382 pages, 20+ font variants), switching to this fork cut font subsetting from **111 minutes to 28 minutes**:
+
+|                                                                                      |    Version     | Duration |
+| -----------------------------------------------------------------------------------: | :------------: | :------- |
+| [Before](https://github.com/alexander-turner/TurnTrout.com/actions/runs/23470135763) | Munter/subfont | 111 min  |
+|  [After](https://github.com/alexander-turner/TurnTrout.com/actions/runs/23518006824) |   This fork    | 28 min   |
+
+## Install
+
+```
+npm install -g @turntrout/subfont
+```
+
+Requires Node.js >= 18.
+
+## Usage
+
+```bash
+# Optimize build artifacts in-place (recommended)
+subfont path/to/dist/index.html -i
+
+# Preview without writing
+subfont path/to/dist/index.html --dry-run
+
+# Output to a separate directory
+subfont path/to/index.html -o path/to/output
+
+# Crawl all linked pages
+subfont path/to/index.html -i --recursive
+
+# Trace JS-rendered content in headless Chrome
+subfont path/to/index.html -i --dynamic
+
+# Cache subset results between runs
+subfont path/to/index.html -i --cache
+```
+
+## Options
+
+|              Flag | Default | Description                                                  |
+| ----------------: | :-----: | :----------------------------------------------------------- |
+|  `-i, --in-place` |   off   | Modify files in-place                                        |
+|    `-o, --output` |         | Output directory                                             |
+| `-r, --recursive` |   off   | Crawl linked pages                                           |
+|       `--dynamic` |   off   | Trace with headless browser                                  |
+|       `--dry-run` |   off   | Preview without writing                                      |
+|     `--fallbacks` |   on    | Load the full original font for characters not in the subset |
+|  `--font-display` | `swap`  | `auto`/`block`/`swap`/`fallback`/`optional`                  |
+|          `--text` |         | Extra characters for every subset                            |
+|   `--cache [dir]` |   off   | Cache subset results to disk between runs                    |
+| `--concurrency N` |         | Max worker threads for parallel font tracing                 |
+|  `--chrome-flags` |         | Custom Chrome flags for `--dynamic`                          |
+|   `--source-maps` |   off   | Preserve CSS source maps (slower)                            |
+
+Run `subfont --help` for the full list.
+
+To include extra characters in a specific font's subset, add `-subfont-text` to its `@font-face`:
+
+```css
+@font-face {
+  font-family: Roboto;
+  src: url(roboto.woff2) format('woff2');
+  -subfont-text: '0123456789';
+}
+```
+
+## Programmatic API
+
+```js
+const subfont = require('@turntrout/subfont');
+
+const assetGraph = await subfont(
+  {
+    inputFiles: ['path/to/index.html'],
+    inPlace: true,
+  },
+  console
+);
+```
+
+Returns the [Assetgraph](https://github.com/assetgraph/assetgraph) instance.
+
+## License
+
+MIT -- Original work by [Peter Muller (Munter)](https://github.com/Munter/subfont)

package/lib/FontTracerPool.js

@@ -0,0 +1,158 @@
+const pathModule = require('path');
+const { Worker } = require('worker_threads');
+
+/**
+ * Worker pool for running fontTracer in parallel across pages.
+ * Each worker re-parses HTML with jsdom and runs fontTracer independently.
+ */
+class FontTracerPool {
+  constructor(numWorkers) {
+    this._workerPath = pathModule.join(__dirname, 'fontTracerWorker.js');
+    this._numWorkers = numWorkers;
+    this._workers = [];
+    this._idle = [];
+    this._pendingTasks = [];
+    this._taskCallbacks = new Map();
+    this._taskByWorker = new Map(); // track which taskId each worker is processing
+    this._nextTaskId = 0;
+  }
+
+  async init() {
+    const initPromises = [];
+    for (let i = 0; i < this._numWorkers; i++) {
+      const worker = new Worker(this._workerPath);
+      this._workers.push(worker);
+
+      const initPromise = new Promise((resolve, reject) => {
+        const onError = reject;
+        const onMessage = (msg) => {
+          if (msg.type === 'ready') {
+            worker.off('message', onMessage);
+            worker.off('error', onError);
+            worker.on('message', (msg) => this._onWorkerMessage(worker, msg));
+            worker.on('exit', (code) => this._onWorkerExit(worker, code));
+            this._idle.push(worker);
+            resolve();
+          }
+        };
+        worker.on('message', onMessage);
+        worker.on('error', onError);
+      });
+
+      worker.postMessage({ type: 'init' });
+
+      initPromises.push(initPromise);
+    }
+    await Promise.all(initPromises);
+  }
+
+  _onWorkerMessage(worker, msg) {
+    this._taskByWorker.delete(worker);
+    const cb = this._taskCallbacks.get(msg.taskId);
+    if (cb) {
+      this._taskCallbacks.delete(msg.taskId);
+      if (msg.type === 'result') {
+        cb.resolve(msg.textByProps);
+      } else if (msg.type === 'error') {
+        cb.reject(new Error(`Worker error: ${msg.error}\n${msg.stack}`));
+      }
+    }
+    // Worker is now idle, check for pending tasks
+    this._idle.push(worker);
+    this._dispatchPending();
+  }
+
+  _onWorkerExit(worker, code) {
+    // Remove crashed worker from tracking
+    const workerIdx = this._workers.indexOf(worker);
+    if (workerIdx !== -1) {
+      this._workers.splice(workerIdx, 1);
+    }
+    const idleIdx = this._idle.indexOf(worker);
+    if (idleIdx !== -1) {
+      this._idle.splice(idleIdx, 1);
+    }
+
+    if (code !== 0) {
+      // Reject the task that was in-flight on this worker
+      const taskId = this._taskByWorker.get(worker);
+      this._taskByWorker.delete(worker);
+      if (taskId !== undefined) {
+        const cb = this._taskCallbacks.get(taskId);
+        if (cb) {
+          this._taskCallbacks.delete(taskId);
+          cb.reject(new Error(`Worker exited with code ${code}`));
+        }
+      }
+
+      // If no workers remain, reject all pending tasks
+      if (this._workers.length === 0) {
+        for (const task of this._pendingTasks) {
+          const cb = this._taskCallbacks.get(task.message.taskId);
+          if (cb) {
+            this._taskCallbacks.delete(task.message.taskId);
+            cb.reject(
+              new Error('All workers have crashed, no workers available')
+            );
+          }
+        }
+        this._pendingTasks = [];
+      }
+    }
+  }
+
+  _dispatchPending() {
+    while (this._idle.length > 0 && this._pendingTasks.length > 0) {
+      const worker = this._idle.pop();
+      const task = this._pendingTasks.shift();
+      this._taskByWorker.set(worker, task.message.taskId);
+      try {
+        worker.postMessage(task.message);
+      } catch (err) {
+        // postMessage can fail synchronously (e.g. structured clone error).
+        // Return the worker to the idle pool and reject the task.
+        this._taskByWorker.delete(worker);
+        this._idle.push(worker);
+        const cb = this._taskCallbacks.get(task.message.taskId);
+        if (cb) {
+          this._taskCallbacks.delete(task.message.taskId);
+          cb.reject(err);
+        }
+      }
+    }
+  }
+
+  /**
+   * Run fontTracer on the given HTML text + stylesheets in a worker.
+   * Returns a promise that resolves to textByProps.
+   */
+  trace(htmlText, stylesheetsWithPredicates) {
+    const taskId = this._nextTaskId++;
+    // Serialize stylesheets to plain data — asset objects contain DOM/PostCSS
+    // trees that cannot be transferred via structured clone.
+    const serializedStylesheets = stylesheetsWithPredicates.map((entry) => ({
+      text: entry.text || (entry.asset && entry.asset.text) || '',
+      predicates: entry.predicates || {},
+    }));
+    const message = {
+      type: 'trace',
+      taskId,
+      htmlText,
+      stylesheetsWithPredicates: serializedStylesheets,
+    };
+
+    return new Promise((resolve, reject) => {
+      this._taskCallbacks.set(taskId, { resolve, reject });
+      this._pendingTasks.push({ message });
+      this._dispatchPending();
+    });
+  }
+
+  async destroy() {
+    await Promise.all(this._workers.map((w) => w.terminate()));
+    this._workers = [];
+    this._idle = [];
+  }
+}
+
+module.exports = FontTracerPool;

package/lib/HeadlessBrowser.js

@@ -0,0 +1,223 @@
+const urlTools = require('urltools');
+const puppeteer = require('puppeteer-core');
+const pathModule = require('path');
+const {
+  install,
+  uninstall,
+  Browser,
+  detectBrowserPlatform,
+  Cache,
+} = require('@puppeteer/browsers');
+
+async function transferResults(jsHandle) {
+  const results = await jsHandle.jsonValue();
+  for (const [i, result] of results.entries()) {
+    const resultHandle = await jsHandle.getProperty(String(i));
+    const elementHandle = await resultHandle.getProperty('node');
+    result.node = elementHandle;
+  }
+  return results;
+}
+
+async function downloadOrLocatePreferredBrowserRevision(extraArgs = []) {
+  if (process.env.PUPPETEER_EXECUTABLE_PATH) {
+    return puppeteer.launch({
+      executablePath: process.env.PUPPETEER_EXECUTABLE_PATH,
+      args: ['--no-sandbox', '--disable-setuid-sandbox', ...extraArgs],
+    });
+  }
+  const cacheDir = pathModule.resolve(__dirname, '..', 'puppeteer-browsers');
+  const platform = detectBrowserPlatform();
+  const cache = new Cache(cacheDir);
+  const installed = cache.getInstalledBrowsers();
+  let executablePath;
+  const chromeEntry = installed.find((b) => b.browser === Browser.CHROME);
+  if (chromeEntry) {
+    executablePath = chromeEntry.executablePath;
+  } else {
+    // Check the default puppeteer cache (~/.cache/puppeteer) before downloading
+    const defaultCacheDir = pathModule.join(
+      require('os').homedir(),
+      '.cache',
+      'puppeteer'
+    );
+    const defaultCache = new Cache(defaultCacheDir);
+    const defaultInstalled = defaultCache.getInstalledBrowsers();
+    const defaultChromeEntry = defaultInstalled.find(
+      (b) => b.browser === Browser.CHROME
+    );
+    if (defaultChromeEntry) {
+      executablePath = defaultChromeEntry.executablePath;
+    } else {
+      console.log('Downloading Chrome');
+      const result = await install({
+        browser: Browser.CHROME,
+        buildId: 'stable',
+        cacheDir,
+        platform,
+      });
+      executablePath = result.executablePath;
+
+      // Clean up older Chrome versions that may have accumulated from
+      // previous runs with different stable buildIds.
+      const allInstalled = cache.getInstalledBrowsers();
+      for (const entry of allInstalled) {
+        if (
+          entry.browser === Browser.CHROME &&
+          entry.executablePath !== executablePath
+        ) {
+          try {
+            await uninstall({
+              browser: entry.browser,
+              buildId: entry.buildId,
+              cacheDir,
+            });
+            console.log(`Removed old Chrome ${entry.buildId}`);
+          } catch {
+            // Ignore cleanup errors — the old version may be in use or locked
+          }
+        }
+      }
+    }
+  }
+  return puppeteer.launch({
+    executablePath,
+    args: ['--no-sandbox', '--disable-setuid-sandbox', ...extraArgs],
+  });
+}
+
+class HeadlessBrowser {
+  constructor({ console, chromeArgs = [] }) {
+    this.console = console;
+    this._chromeArgs = chromeArgs;
+  }
+
+  _ensureBrowserDownloaded() {}
+
+  _launchBrowserMemoized() {
+    // Make sure we only download and launch one browser per HeadlessBrowser instance
+    return (this._launchPromise =
+      this._launchPromise ||
+      downloadOrLocatePreferredBrowserRevision(this._chromeArgs));
+  }
+
+  async tracePage(htmlAsset) {
+    const assetGraph = htmlAsset.assetGraph;
+    const browser = await this._launchBrowserMemoized();
+    const page = await browser.newPage();
+
+    try {
+      // Make up a base url to map to the assetgraph root.
+      // Use the canonical root if available, so that it'll be
+      // easier to handle absolute and protocol-relative urls pointing
+      // at it, as well as fall through to the actual domain if some
+      // assets aren't found in the graph.
+      const baseUrl = assetGraph.canonicalRoot
+        ? assetGraph.canonicalRoot.replace(/\/?$/, '/')
+        : 'https://example.com/';
+
+      // Intercept all requests made by the headless browser, and
+      // fake a response from the assetgraph instance if the corresponding
+      // asset is found there:
+      await page.setRequestInterception(true);
+      page.on('request', (request) => {
+        const url = request.url();
+        if (url.startsWith(baseUrl)) {
+          let agUrl = url.replace(baseUrl, assetGraph.root);
+          if (/\/$/.test(agUrl)) {
+            agUrl += 'index.html';
+          }
+          const asset = assetGraph.findAssets({
+            isLoaded: true,
+            url: agUrl,
+          })[0];
+          if (asset) {
+            request.respond({
+              status: 200,
+              contentType: asset.contentType,
+              body: asset.rawSrc,
+            });
+          } else {
+            // Asset not in graph — return 404 instead of letting the
+            // request hit the network (baseUrl is synthetic).
+            request.respond({ status: 404, body: '' });
+          }
+          return;
+        }
+        if (url.startsWith('file:')) {
+          request.continue();
+          return;
+        }
+        // External request — abort to avoid hanging on DNS/network.
+        // The requestfailed handler will log it.
+        request.abort('failed');
+      });
+
+      page.on('requestfailed', (request) => {
+        const response = request.response();
+        if (response && response.status() > 400) {
+          this.console.error(
+            `${request.method()} ${request.url()} returned ${response.status()}`
+          );
+        } else {
+          this.console.error(
+            `${request.method()} ${request.url()} failed: ${
+              request.failure().errorText
+            }`
+          );
+        }
+      });
+
+      page.on('pageerror', (err) => {
+        // Puppeteer v24+ passes Error objects; format stack to match v19 style
+        if (err instanceof Error && err.stack) {
+          // Normalize "at <anonymous> (url:line:col)" to "at url:line:col"
+          const normalized = err.stack.replace(
+            /at <anonymous> \((.+)\)/g,
+            'at $1'
+          );
+          this.console.error(normalized);
+        } else if (err instanceof Error) {
+          this.console.error(`${err.name}: ${err.message}`);
+        } else {
+          this.console.error(err);
+        }
+      });
+      page.on('error', this.console.error);
+
+      // Prevent the CSP of the page from rejecting our injection of font-tracer
+      await page.setBypassCSP(true);
+
+      await page.goto(
+        urlTools.resolveUrl(
+          baseUrl,
+          urlTools.buildRelativeUrl(assetGraph.root, htmlAsset.url)
+        )
+      );
+
+      await page.addScriptTag({
+        path: require.resolve('font-tracer/dist/fontTracer.browser.js'),
+      });
+
+      const jsHandle = await page.evaluateHandle(
+        /* global fontTracer */
+        /* istanbul ignore next */
+        () => fontTracer(document)
+      );
+      return await transferResults(jsHandle);
+    } finally {
+      await page.close();
+    }
+  }
+
+  async close() {
+    const launchPromise = this._launchPromise;
+    if (launchPromise) {
+      this._launchPromise = undefined;
+      const browser = await launchPromise;
+      await browser.close();
+    }
+  }
+}
+
+module.exports = HeadlessBrowser;

package/lib/cli.js

@@ -0,0 +1,14 @@
+#!/usr/bin/env node
+
+const { yargs, help, ...options } = require('./parseCommandLineOptions')();
+
+require('@gustavnikolaj/async-main-wrap')(require('./subfont'), {
+  processError(err) {
+    yargs.showHelp();
+    if (err.constructor === SyntaxError) {
+      // Avoid rendering a stack trace for the wrong usage errors
+      err.customOutput = err.message;
+    }
+    return err;
+  },
+})(options, console);