oxlint-plugin-pngdeer 0.2.0 → 0.3.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -65,14 +65,16 @@ pngdeer: 4 flagged images could be 718 kB smaller in all (1.0 MB → 310 kB, 70%
65
65
  src/assets/diagram.png 248 kB → 63 kB 185 kB 90
66
66
  public/photo.jpg 235 kB → 85 kB 150 kB 76
67
67
  src/assets/crystal-ball.png 35 kB → 11 kB 24 kB 87
68
- pngdeer: 1 more image could be 5.1 kB smaller, but no linted code names it (used from CSS or HTML, or not at all):
69
- public/unused-fire.png 13 kB → 7.5 kB 5.1 kB 90
68
+ pngdeer: also 1 image that no linted JS or TS names could be 5.1 kB smaller:
69
+ public/unused-fire.png 13 kB → 7.5 kB 5.1 kB 90 mentioned in README.md
70
70
  pngdeer: quality is SSIMULACRA2, out of 100: 90+ looks identical, 80+ the same side by side, 70+ close, with differences you might spot side by side
71
71
  pngdeer: to shrink the 4 flagged images in place (lossy, so commit first), run: npx oxlint-plugin-pngdeer --flagged
72
+ pngdeer: to shrink it too, run: npx oxlint-plugin-pngdeer public/unused-fire.png
72
73
  ```
73
74
 
74
- - **Each image warns once per file,** on the line that names it, and says how close to the original it would look.
75
- - **The table at the end** counts each image once, biggest saving first. It also lists images the run measured that no linted code names, like a CSS background, which no warning can point at.
75
+ - **Each image warns once per file,** on the line that names it, and says how close to the original it would look. An image named in several files gets the full warning once per run, and a short one after that.
76
+ - **The table at the end** counts each image once, biggest saving first.
77
+ - **Images no JS or TS names** get their own list, since no warning can point at them. Each row says where the image is used instead: `used in src/App.css`, `mentioned in README.md` (a Markdown file that names it), or `nothing names it`, which may mean the image is unused and could be deleted. These aren't in `--flagged`; the list ends with the command for them.
76
78
  - **The table is left out** with `--quiet` and in formats other programs read (`json`, `checkstyle`, `junit`, `sarif`, `gitlab`).
77
79
 
78
80
  Then shrink what it found. There's no autofix (lint fixes can only edit code), so run the command: `--flagged` takes every image the last lint in that folder warned about, or name images yourself.
@@ -89,7 +91,7 @@ pngdeer: 4 images, 1.0 MB → 310 kB (718 kB smaller). Lint again to check.
89
91
 
90
92
  - **It's lossy, and it writes over the file.** Commit first so git is your undo, or write a copy to compare with `-o hero.smaller.png`.
91
93
  - **A warning you turn off with a comment** stays out of the table and out of `--flagged`.
92
- - **It takes a few seconds per image,** one at a time.
94
+ - **Several images run in parallel,** one engine per core, within a quarter of the machine's memory, like the lint's first run.
93
95
  - **`--flagged` uses the `target` each image was measured at.** For images you name, pass it yourself (`--target 90`), as the command in each warning does. `--help` lists the options.
94
96
  - **JPEGs keep their orientation and color profile;** other camera metadata (EXIF, GPS) is dropped.
95
97
  - **It marks each file it writes** with a short text tag (`pngdeer <size>`, in a PNG `tEXt` chunk or a JPEG comment), so the rule doesn't flag it again, on any machine.
@@ -98,7 +100,7 @@ Already have oxlint 1.90 or later? Install only `oxlint-plugin-pngdeer`. If your
98
100
 
99
101
  ## Using ESLint
100
102
 
101
- The same rule runs in ESLint 9 and 10, and shares the cache with oxlint:
103
+ The same rule runs in ESLint 9 and 10, and shares the cache with oxlint. With `--concurrency`, ESLint's threads take turns measuring and print one summary:
102
104
 
103
105
  ```bash
104
106
  npm install --save-dev oxlint-plugin-pngdeer
@@ -157,7 +159,15 @@ Measured on a 16-core M-series Mac ([details](https://github.com/lndgalante/oxli
157
159
 
158
160
  ## In CI
159
161
 
160
- `npm ci` deletes `node_modules`, and the cache with it, so restore the cache after installing, and save it even when the lint fails:
162
+ The simplest way to never start cold: keep the results in a folder you commit, so CI and every clone start with them.
163
+
164
+ ```json
165
+ "settings": { "pngdeer": { "cacheDir": ".pngdeer" } }
166
+ ```
167
+
168
+ Results are small JSON files, one per image version, formatted the way Prettier formats JSON (Biome users: add `!.pngdeer` to `files.includes`). After you add or change images, run `npx oxlint` and commit `.pngdeer` with them. Images pngdeer fails on are remembered on each machine only, never in the committed folder. Upgrading the plugin to a new engine measures everything again; delete the folder first to drop the old results. In a monorepo, packages can share one folder (`"cacheDir": "../../.pngdeer"`).
169
+
170
+ Or let CI cache it. `npm ci` deletes `node_modules`, and the default cache with it, so restore the cache after installing, and save it even when the lint fails:
161
171
 
162
172
  ```yaml
163
173
  - run: npm ci
package/bin/pngdeer.js CHANGED
@@ -2,13 +2,11 @@
2
2
  // oxlint-plugin-pngdeer: makes the images the lint rule flags smaller, with the engine the rule
3
3
  // measures with, so each file it writes is the one the warning promised. Files it writes are marked
4
4
  // as pngdeer's (src/marker.js), so the rule never measures them again, on any machine.
5
- import { chmodSync, readFileSync, renameSync, rmSync, statSync, writeFileSync } from "node:fs";
6
- import { basename, dirname, join } from "node:path";
7
- import { compress } from "../src/engine.js";
8
- import { unsupported } from "../src/images.js";
9
- import { mark, marked } from "../src/marker.js";
5
+ import { readFileSync, realpathSync } from "node:fs";
6
+ import { peakOf, pool } from "../src/batch.js";
7
+ import { plain, shrink } from "../src/shrink.js";
10
8
  import { readFlagged } from "../src/summary.js";
11
- import { count, looks, pathFrom, size, smaller } from "../src/text.js";
9
+ import { count, pathFrom, size } from "../src/text.js";
12
10
 
13
11
  const HELP = `usage: oxlint-plugin-pngdeer IMAGE... [--target auto|T]
14
12
  oxlint-plugin-pngdeer --flagged [--target auto|T]
@@ -27,7 +25,7 @@ pngdeer is lossy: the picture looks the same, the bytes don't, so commit first.
27
25
 
28
26
  Files it writes are marked as pngdeer's, so the lint rule doesn't flag them again.`;
29
27
 
30
- function main(args) {
28
+ async function main(args) {
31
29
  const inputs = [];
32
30
  let output;
33
31
  let target;
@@ -70,90 +68,80 @@ function main(args) {
70
68
  return 0;
71
69
  }
72
70
  // each image, the path to show for it, and its target: a --target given here wins over the lint's
73
- const jobs = fromLint
74
- ? flagged.map((image) => ({
75
- input: pathFrom(process.cwd(), image.file),
76
- target: targeted ? target : image.target,
77
- }))
78
- : inputs.map((input) => ({ input, target }));
71
+ const jobs = once(
72
+ fromLint
73
+ ? flagged.map((image) => ({ input: pathFrom(process.cwd(), image.file), target: targeted ? target : image.target }))
74
+ : inputs.map((input) => ({ input, target })),
75
+ );
79
76
 
80
- // several images take a while, one after another: say what's coming, count, and add it all up
81
- const many = jobs.length > 1;
82
- if (many) console.error(`pngdeer: shrinking ${count(jobs.length, "image")} in place (lossy; commit first so you can undo)`);
77
+ // one image runs here; several run on a pool of engines, one per core, each line printed as it lands
78
+ if (jobs.length === 1) {
79
+ const [{ input, target }] = jobs;
80
+ try {
81
+ console.log(`${input}: ${shrink(input, output ?? input, target).text}`);
82
+ return 0;
83
+ } catch (error) {
84
+ console.error(`${input}: ${plain(error)}`);
85
+ return 1;
86
+ }
87
+ }
88
+ console.error(`pngdeer: shrinking ${count(jobs.length, "image")} in place (lossy; commit first so you can undo)`);
83
89
  let status = 0;
90
+ let finished = 0;
84
91
  let original = 0;
85
92
  let compressed = 0;
86
93
  let shrunk = 0;
87
- for (const [i, { input, target }] of jobs.entries()) {
88
- const step = many ? `[${i + 1}/${jobs.length}] ` : "";
94
+ const report = (input, result) => {
95
+ const step = `[${++finished}/${jobs.length}] ${input}`;
96
+ if (result.error) {
97
+ console.error(`${step}: ${result.error}`);
98
+ status = 1;
99
+ return;
100
+ }
101
+ console.log(`${step}: ${result.text}`);
102
+ if (result.written) {
103
+ shrunk++;
104
+ original += result.original;
105
+ compressed += result.written;
106
+ }
107
+ };
108
+ const work = [];
109
+ for (const { input, target } of jobs) {
110
+ let bytes;
89
111
  try {
90
- const result = shrink(input, output ?? input, target);
91
- console.log(`${step}${input}: ${result.text}`);
92
- if (result.written) {
93
- shrunk++;
94
- original += result.original;
95
- compressed += result.written;
96
- }
112
+ bytes = readFileSync(input);
97
113
  } catch (error) {
98
- console.error(`${step}${input}: ${plain(error)}`);
99
- status = 1;
114
+ report(input, { error: plain(error) });
115
+ continue;
100
116
  }
117
+ work.push({ input, peak: peakOf(bytes), message: { file: input, target, out: input } });
101
118
  }
102
- if (many && shrunk) {
119
+ try {
120
+ await pool(work, { done: (job, result) => report(job.input, result) });
121
+ } catch (error) {
122
+ console.error(`oxlint-plugin-pngdeer: ${plain(error)}`);
123
+ return 1;
124
+ }
125
+ if (shrunk) {
103
126
  console.error(`pngdeer: ${count(shrunk, "image")}, ${size(original)} → ${size(compressed)} (${size(original - compressed)} smaller). Lint again to check.`);
104
127
  }
105
128
  return status;
106
129
  }
107
130
 
108
- /** Writes `input` made smaller to `output`: { text } saying what happened, and the sizes if it wrote. */
109
- function shrink(input, output, target) {
110
- const bytes = readFileSync(input);
111
- const unchanged = (note) => {
112
- if (output !== input) save(output, bytes);
113
- return { text: `${note}; left as is` };
114
- };
115
- if (marked(bytes)) return unchanged("already pngdeer's");
116
- const why = unsupported(bytes);
117
- if (why) throw new Error(why);
118
-
119
- const { output: compressed, outcome, score } = compress(bytes, target);
120
- if (outcome === "miss") {
121
- const wanted = target === undefined ? "the automatic quality" : `--target ${target}`;
122
- return unchanged(`no smaller file reaches ${wanted} (the best reaches quality ${Math.round(score)}/100)`);
123
- }
124
- // the mark costs a few dozen bytes, more than pngdeer saves on some tiny files
125
- const result = outcome === "original" ? null : mark(compressed);
126
- if (!result || result.length >= bytes.length) return unchanged("pngdeer can't make it smaller");
127
- save(output, result);
128
- const quality = outcome === "hit" ? `quality ${Math.round(score)}/100: looks ${looks(score)}` : "lossless: looks identical";
129
- const text = `${size(bytes.length)} → ${size(result.length)} (${smaller(bytes.length, result.length)}% smaller, ${quality})`;
130
- return { text, original: bytes.length, written: result.length };
131
- }
132
-
133
- /** Writes a whole new file or nothing: the image is often written over itself. */
134
- function save(path, bytes) {
135
- const tmp = join(dirname(path), `.${basename(path)}.${process.pid}.tmp`);
136
- try {
137
- writeFileSync(tmp, bytes);
131
+ /** Each image once: overlapping globs or ./ forms would shrink it twice, the second time lossily. */
132
+ function once(jobs) {
133
+ const seen = new Set();
134
+ return jobs.filter(({ input }) => {
135
+ let path = input;
138
136
  try {
139
- chmodSync(tmp, statSync(path).mode & 0o7777);
137
+ path = realpathSync(input);
140
138
  } catch {
141
- // a new file keeps the default mode
139
+ // a missing file is reported when it's read
142
140
  }
143
- renameSync(tmp, path);
144
- } catch (error) {
145
- rmSync(tmp, { force: true });
146
- if (error.code === "ENOENT") throw new Error(`can't write ${path}: its folder doesn't exist`);
147
- throw error;
148
- }
149
- }
150
-
151
- /** An error as one line a person can act on. */
152
- function plain(error) {
153
- if (error.code === "ENOENT") return "no such file";
154
- if (error.code === "EACCES" || error.code === "EPERM") return "permission denied";
155
- if (error.code === "EISDIR") return "a folder, not an image";
156
- return error.message;
141
+ if (seen.has(path)) return false;
142
+ seen.add(path);
143
+ return true;
144
+ });
157
145
  }
158
146
 
159
147
  function fail(message) {
@@ -161,4 +149,4 @@ function fail(message) {
161
149
  return 2;
162
150
  }
163
151
 
164
- process.exitCode = main(process.argv.slice(2));
152
+ process.exitCode = await main(process.argv.slice(2));
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "oxlint-plugin-pngdeer",
3
- "version": "0.2.0",
3
+ "version": "0.3.0",
4
4
  "description": "Lint rule for oxlint and ESLint: flags PNGs and JPEGs your code uses that could be much smaller at about the same visual quality, with the command that shrinks them. Local WebAssembly, no upload.",
5
5
  "keywords": [
6
6
  "oxlint",
package/src/batch.js CHANGED
@@ -1,11 +1,12 @@
1
1
  // Fills the cache for a list of images, on a pool of worker threads (one wasm engine each, so all
2
- // cores work even though the engine itself is single-threaded), within a memory budget.
3
- import { readFileSync, rmSync } from "node:fs";
2
+ // cores work even though the engine itself is single-threaded), within a memory budget. The command
3
+ // shrinks several images on the same pool.
4
+ import { mkdirSync, readFileSync, rmSync, statSync } from "node:fs";
4
5
  import { availableParallelism, totalmem } from "node:os";
5
6
  import { join } from "node:path";
6
7
  import { pathToFileURL } from "node:url";
7
8
  import { Worker } from "node:worker_threads";
8
- import { cacheDir, key, read, write } from "./cache.js";
9
+ import { key, read, readFirst, write } from "./cache.js";
9
10
  import { dimensions, kind, unsupported } from "./images.js";
10
11
  import { marked } from "./marker.js";
11
12
 
@@ -13,18 +14,32 @@ const MAX_WORKERS = 8;
13
14
  // Wasm memory never shrinks, so a worker that measured an image expected to peak above this is
14
15
  // replaced rather than kept.
15
16
  const BIG = 300e6;
17
+ // A batch holding the lock longer than this died with it (sync.js gives up waiting at 15 minutes too).
18
+ const STALE_MS = 15 * 60 * 1000;
16
19
 
17
20
  /**
18
21
  * Computes every file not cached yet; { computed, cached, skipped, failed, ms }. `progress(done,
19
22
  * total)` hears about each image the engine finishes.
20
23
  */
21
- export async function fill({ cwd, target, files }, progress = () => {}) {
24
+ export async function fill({ dir, failures = dir, target, files }, progress = () => {}) {
25
+ // a result in either folder counts: `failures` keeps what pngdeer failed on, on this machine only
26
+ const cached = (k) => readFirst(k, dir, failures);
22
27
  const start = performance.now();
23
- const dir = cacheDir(cwd);
24
28
  // a cache that can't be written fails here, before any image is compressed for nothing
25
- write(dir, "probe", {});
26
- rmSync(join(dir, "probe.json"), { force: true });
29
+ for (const folder of new Set([dir, failures])) {
30
+ write(folder, "probe", {});
31
+ rmSync(join(folder, "probe.json"), { force: true });
32
+ }
33
+ const release = await lock(failures);
34
+ try {
35
+ return await measure({ dir, failures, target, files, cached }, progress, start);
36
+ } finally {
37
+ release();
38
+ }
39
+ }
27
40
 
41
+ /** fill() once it holds the lock: what's cached by then is skipped, the rest goes to the pool. */
42
+ async function measure({ dir, failures, target, files, cached }, progress, start) {
28
43
  const todo = [];
29
44
  const counts = { computed: 0, cached: 0, skipped: 0, failed: 0 };
30
45
  for (const file of new Set(files)) {
@@ -35,12 +50,12 @@ export async function fill({ cwd, target, files }, progress = () => {}) {
35
50
  continue;
36
51
  }
37
52
  const k = key(bytes, target);
38
- if (read(dir, k)) {
53
+ if (cached(k)) {
39
54
  counts.cached++;
40
55
  continue;
41
56
  }
57
+ // pngdeer's own output needs no engine, and no entry: the rule knows it by its mark
42
58
  if (marked(bytes)) {
43
- write(dir, k, { original: bytes.length, outcome: "output" });
44
59
  counts.skipped++;
45
60
  continue;
46
61
  }
@@ -50,12 +65,8 @@ export async function fill({ cwd, target, files }, progress = () => {}) {
50
65
  counts.skipped++;
51
66
  continue;
52
67
  }
53
- const [width, height, frames] = dimensions(bytes, kind(bytes));
54
- todo.push({ file, k, peak: peak(width * height * frames) });
68
+ todo.push({ file, k, peak: peakOf(bytes), message: { file, target } });
55
69
  }
56
- // biggest first, so the slowest image doesn't start last
57
- todo.sort((a, b) => b.peak - a.peak);
58
- const total = todo.length;
59
70
 
60
71
  const record = (job, result) => {
61
72
  // the file changed or went away after it was queued: cache nothing, the next lint measures it again
@@ -64,7 +75,7 @@ export async function fill({ cwd, target, files }, progress = () => {}) {
64
75
  return;
65
76
  }
66
77
  if (result.error) {
67
- write(dir, job.k, { skipped: result.error, failed: true });
78
+ write(failures, job.k, { skipped: result.error, failed: true });
68
79
  counts.failed++;
69
80
  return;
70
81
  }
@@ -76,6 +87,30 @@ export async function fill({ cwd, target, files }, progress = () => {}) {
76
87
  if (output !== input) write(dir, output, { original: entry.compressed, outcome: "output" });
77
88
  };
78
89
 
90
+ const total = todo.length;
91
+ await pool(todo, {
92
+ // the same bytes may have been marked as pngdeer's output since the batch started
93
+ skip: (job) => {
94
+ if (!cached(job.k)) return false;
95
+ counts.cached++;
96
+ return true;
97
+ },
98
+ done: (job, result) => {
99
+ record(job, result);
100
+ progress(counts.computed + counts.failed, total);
101
+ },
102
+ });
103
+ return { ...counts, ms: Math.round(performance.now() - start) };
104
+ }
105
+
106
+ /**
107
+ * Runs `jobs` ({ peak, message }) on a pool of engines (worker.js), biggest first, within the memory
108
+ * budget. `done(job, reply)` gets each reply as it comes; a job `skip(job)` turns down when its turn
109
+ * comes isn't sent. Rejects, with every engine stopped, if one fails or dies.
110
+ */
111
+ export async function pool(jobs, { done, skip = () => false }) {
112
+ // biggest first, so the slowest image doesn't start last
113
+ const todo = [...jobs].sort((a, b) => b.peak - a.peak);
79
114
  const limit = budget();
80
115
  const lanes = Array.from({ length: Math.min(todo.length, MAX_WORKERS, Math.max(1, availableParallelism() - 1)) }, () => ({ worker: null, held: 0, busy: false }));
81
116
  const waiting = [];
@@ -93,16 +128,12 @@ export async function fill({ cwd, target, files }, progress = () => {}) {
93
128
  continue;
94
129
  }
95
130
  const [job] = todo.splice(index, 1);
96
- // the same bytes may have been marked as pngdeer's output since the batch started
97
- if (read(dir, job.k)) {
98
- counts.cached++;
99
- continue;
100
- }
131
+ if (skip(job)) continue;
101
132
  self.busy = true;
102
133
  self.held = Math.max(self.held, job.peak);
103
- self.worker ??= new Worker(new URL("./worker.js", import.meta.url), { workerData: { target } });
134
+ self.worker ??= new Worker(new URL("./worker.js", import.meta.url));
104
135
  try {
105
- record(job, await ask(self.worker, { file: job.file }));
136
+ done(job, await ask(self.worker, job.message));
106
137
  if (job.peak > BIG) {
107
138
  await self.worker.terminate();
108
139
  self.worker = null;
@@ -114,7 +145,6 @@ export async function fill({ cwd, target, files }, progress = () => {}) {
114
145
  for (const l of lanes) l.worker?.terminate();
115
146
  } finally {
116
147
  self.busy = false;
117
- progress(counts.computed + counts.failed, total);
118
148
  wake();
119
149
  }
120
150
  }
@@ -125,7 +155,37 @@ export async function fill({ cwd, target, files }, progress = () => {}) {
125
155
  await Promise.all(lanes.map((l) => l.worker?.terminate()));
126
156
  }
127
157
  if (failure) throw failure;
128
- return { ...counts, ms: Math.round(performance.now() - start) };
158
+ }
159
+
160
+ /**
161
+ * Waits until no other batch measures for this project, then holds that until `release()`: ESLint's
162
+ * --concurrency lints in several threads, and a command-line lint may start while an editor measures.
163
+ * Whoever waits then finds most images cached.
164
+ */
165
+ async function lock(folder) {
166
+ const path = join(folder, "measuring.lock");
167
+ for (;;) {
168
+ try {
169
+ mkdirSync(path);
170
+ return () => rmSync(path, { recursive: true, force: true });
171
+ } catch (error) {
172
+ // a folder where nothing can be locked: measure anyway
173
+ if (error.code !== "EEXIST") return () => {};
174
+ }
175
+ try {
176
+ if (Date.now() - statSync(path).mtimeMs > STALE_MS) rmSync(path, { recursive: true, force: true });
177
+ } catch {
178
+ // released meanwhile
179
+ }
180
+ await new Promise((resolve) => setTimeout(resolve, 250));
181
+ }
182
+ }
183
+
184
+ /** An engine's expected peak memory for an image (by its header), for the pool's budget. */
185
+ export function peakOf(bytes) {
186
+ const type = kind(bytes);
187
+ const [width, height, frames] = type ? dimensions(bytes, type) : [0, 0, 1];
188
+ return peak(width * height * frames);
129
189
  }
130
190
 
131
191
  /** An engine's expected peak memory for an image of this many pixels (over all its frames). */
@@ -157,7 +217,7 @@ function ask(worker, message) {
157
217
  });
158
218
  }
159
219
 
160
- // `node src/batch.js` with { cwd, target, files } as JSON on stdin, for trying it outside a linter
220
+ // `node src/batch.js` with { dir, target, files } as JSON on stdin (dir: the cache folder), for trying it outside a linter
161
221
  if (import.meta.url === pathToFileURL(process.argv[1] ?? "").href) {
162
222
  const request = JSON.parse(readFileSync(0, "utf8"));
163
223
  process.stdout.write(JSON.stringify(await fill(request)));
package/src/cache.js CHANGED
@@ -2,10 +2,13 @@
2
2
  // saving only changes when its bytes, the engine or the target do. One small JSON file per entry,
3
3
  // so parallel lint threads and batch workers never write the same file twice over.
4
4
  import { createHash } from "node:crypto";
5
- import { mkdirSync, readFileSync, renameSync, writeFileSync } from "node:fs";
5
+ import { mkdirSync, readFileSync, renameSync, statSync, writeFileSync } from "node:fs";
6
6
  import { join } from "node:path";
7
+ import { marked } from "./marker.js";
7
8
  import { smaller } from "./text.js";
8
9
 
10
+ const keys = new Map(); // image path -> { size, mtimeMs, target, key, marked }, so images aren't re-read on every keystroke
11
+
9
12
  // The engine's identity: the first 16 hex digits of the wasm's SHA-256 (a few ms for 2 MB). A new
10
13
  // engine invalidates every entry.
11
14
  export const ENGINE = createHash("sha256")
@@ -38,6 +41,15 @@ export function read(dir, k) {
38
41
  }
39
42
  }
40
43
 
44
+ /** The entry for `k` in the first of `dirs` that has one, or null. */
45
+ export function readFirst(k, ...dirs) {
46
+ for (const dir of new Set(dirs)) {
47
+ const entry = read(dir, k);
48
+ if (entry) return entry;
49
+ }
50
+ return null;
51
+ }
52
+
41
53
  /** Whether an entry is a saving the rule reports: smaller at the target, by both thresholds. */
42
54
  export function reportable(entry, { minSavedBytes, minSavedPercent }) {
43
55
  if (!entry || entry.skipped || !["hit", "exact", "lossless"].includes(entry.outcome)) return false;
@@ -45,10 +57,26 @@ export function reportable(entry, { minSavedBytes, minSavedPercent }) {
45
57
  return saved >= minSavedBytes && smaller(entry.original, entry.compressed) >= minSavedPercent;
46
58
  }
47
59
 
60
+ /** An image's size, the cache key of its current bytes and whether pngdeer wrote it; null if it can't be read. */
61
+ export function fileKey(file, target) {
62
+ try {
63
+ const { size, mtimeMs } = statSync(file);
64
+ const memo = keys.get(file);
65
+ if (memo && memo.size === size && memo.mtimeMs === mtimeMs && memo.target === target) return memo;
66
+ const bytes = readFileSync(file);
67
+ const image = { size, mtimeMs, target, key: key(bytes, target), marked: marked(bytes) };
68
+ keys.set(file, image);
69
+ return image;
70
+ } catch {
71
+ return null;
72
+ }
73
+ }
74
+
48
75
  export function write(dir, k, entry) {
49
76
  mkdirSync(dir, { recursive: true });
50
- // written whole and renamed, so a reader never sees half an entry
77
+ // written whole and renamed, so a reader never sees half an entry; formatted the way Prettier
78
+ // would, so a committed cache folder passes `prettier --check`
51
79
  const tmp = join(dir, `${k}.${process.pid}.${Math.random().toString(36).slice(2)}.tmp`);
52
- writeFileSync(tmp, JSON.stringify(entry));
80
+ writeFileSync(tmp, `${JSON.stringify(entry, null, 2)}\n`);
53
81
  renameSync(tmp, join(dir, `${k}.json`));
54
82
  }
package/src/index.d.ts CHANGED
@@ -14,8 +14,10 @@ export interface CompressibleImageOptions {
14
14
  export interface PngdeerSettings {
15
15
  /** Folders where "/x.png" paths are looked up. Default ["public", "static"]. */
16
16
  publicDirs?: string[];
17
- /** Path prefixes and the folders they stand for. Default { "@/": "src/", "~/": "src/" }. */
17
+ /** Path prefixes and the folders they stand for. Default: the project's tsconfig/jsconfig paths, else { "@/": "src/", "~/": "src/" }. */
18
18
  aliases?: Record<string, string>;
19
+ /** Where results are kept, relative to the project root: a folder you can commit. Default node_modules/.cache/pngdeer. */
20
+ cacheDir?: string;
19
21
  }
20
22
 
21
23
  // Shaped like an ESLint plugin, so `plugins: { pngdeer }` type-checks in eslint.config.ts and
package/src/rule.js CHANGED
@@ -1,12 +1,12 @@
1
1
  // pngdeer/compressible-image: on each string in the code that names a local PNG or JPEG (imports,
2
2
  // require, new URL, JSX src and srcSet, plain strings), says how much smaller pngdeer would make it.
3
3
  // oxlint never lints image files themselves, so the reference is where the warning goes.
4
- import { readFileSync, statSync } from "node:fs";
4
+ import { statSync } from "node:fs";
5
5
  import { basename, dirname, join, resolve } from "node:path";
6
- import { cacheDir, key, read, reportable } from "./cache.js";
7
- import { marked } from "./marker.js";
6
+ import { cacheDir, fileKey, readFirst, reportable } from "./cache.js";
8
7
  import { projectImages } from "./scan.js";
9
- import { flag, name, summarize, watch } from "./summary.js";
8
+ import { flag, name, share, summarize, watch } from "./summary.js";
9
+ import { configAliases } from "./tsconfig.js";
10
10
  import { fillInBackground, fillSync } from "./sync.js";
11
11
  import { count, looks, pathFrom, quote, size, smaller } from "./text.js";
12
12
 
@@ -14,23 +14,27 @@ const RULE = "pngdeer/compressible-image";
14
14
  const IMAGE = /\.(png|apng|jpe?g)$/i;
15
15
  const REMOTE = /^[a-z][a-z0-9+.-]*:|^\/\//i;
16
16
  const DEFAULTS = { minSavedBytes: 4096, minSavedPercent: 10, target: "auto", measure: "auto" };
17
+ // when neither the settings nor the project's tsconfig/jsconfig define aliases
18
+ const ALIASES = { "@/": "src/", "~/": "src/" };
17
19
 
18
20
  // Editors lint on every keystroke, so there the rule measures in the background and reports on a
19
21
  // later lint. Everywhere else it waits: a one-off run (the oxlint or eslint command, or a tool calling
20
22
  // ESLint's Node API, like Nx or xo) would exit before a background batch finished. Neither linter
21
- // has an API for this: it's how oxlint's language server (--lsp) and ESLint's (the one VS Code, Zed
22
- // and Neovim use, an Electron child in VS Code) start.
23
+ // has an API for this: it's how oxlint's language server (--lsp), ESLint's (the one VS Code, Zed
24
+ // and Neovim use, an Electron child in VS Code) and the eslint_d daemon that editors call start.
23
25
  const EDITOR =
24
26
  process.argv.includes("--lsp") ||
25
- /eslintServer|language-?server|languageService/i.test(process.argv[1] ?? "") ||
27
+ /eslintServer|language-?server|languageService|eslint_d/i.test(process.argv[1] ?? "") ||
26
28
  process.env.ELECTRON_RUN_AS_NODE === "1";
27
29
 
28
30
  // Per process (the language server's lasts the whole editor session):
29
- const keys = new Map(); // image path -> { size, mtimeMs, target, key, marked }, so images aren't re-read on every keystroke
30
31
  const roots = new Map(); // directory -> its project root
31
- const scanned = new Set(); // `${root}\0${target}` whose images were all measured, or queued, once
32
+ const scanned = new Set(); // `${root}\0${dir}\0${target}` whose project images were all measured, or queued, once
32
33
  const failed = new Set(); // outside editors: roots whose batch failed, not tried again this run
33
34
  const told = new Set(); // outside editors: images already named on stderr
35
+ // outside editors: `${image}\0${target}` -> the linted file that got the image's full warning; other
36
+ // files get a short one. A file linted again (eslint --fix passes, watch modes) keeps its full one.
37
+ const warned = new Map();
34
38
 
35
39
  export default {
36
40
  meta: {
@@ -42,6 +46,8 @@ export default {
42
46
  },
43
47
  messages: {
44
48
  compressible: "{{name}} could be {{percent}}% smaller ({{before}} → {{after}}) and look {{looks}} (quality {{quality}}/100). Run: {{command}}",
49
+ // the same image again, in another file of the same run: its first warning said the rest
50
+ again: "{{name}} could be {{percent}}% smaller ({{before}} → {{after}}), as warned where it's first used. Run: {{command}}",
45
51
  lossless: "{{name}} could be {{percent}}% smaller ({{before}} → {{after}}), losslessly. Run: {{command}}",
46
52
  },
47
53
  schema: [
@@ -64,11 +70,18 @@ export default {
64
70
  create(context) {
65
71
  const { minSavedBytes, minSavedPercent, target: option, measure } = { ...DEFAULTS, ...context.options[0] };
66
72
  const target = option === "auto" ? undefined : option;
67
- const { publicDirs = ["public", "static"], aliases = { "@/": "src/", "~/": "src/" } } = context.settings.pngdeer ?? {};
73
+ const settings = context.settings.pngdeer ?? {};
68
74
  const from = context.physicalFilename ?? context.filename;
69
75
  // public dirs, aliases and the cache belong to the package, wherever the linter was started
70
76
  const root = projectRoot(dirname(from), context.cwd);
71
- const dir = cacheDir(root);
77
+ const { publicDirs = ["public", "static"] } = settings;
78
+ const aliases = settings.aliases ?? configAliases(root) ?? ALIASES;
79
+ // a folder the team commits (settings.pngdeer.cacheDir) keeps CI and fresh clones warm
80
+ const local = cacheDir(root);
81
+ const dir = settings.cacheDir ? resolve(root, settings.cacheDir) : local;
82
+ // pngdeer failing on an image is kept in `local` only: the failure may be this machine's alone
83
+ // (out of memory), and in a committed folder it would stick for the whole team
84
+ const lookup = (k) => readFirst(k, dir, local);
72
85
  const find = (path) => locate(path, from, root, publicDirs, aliases);
73
86
  // a path as the person running the linter would type it
74
87
  const shown = (file) => pathFrom(context.cwd, file);
@@ -77,7 +90,7 @@ export default {
77
90
 
78
91
  const reported = new Set(); // images warned about in this file: once each, at their first mention
79
92
  let disabled; // (line) => whether a comment turns this rule off there, read on first use
80
- if (!EDITOR) watch(root, { minSavedBytes, minSavedPercent, target });
93
+ if (!EDITOR) watch(root, { minSavedBytes, minSavedPercent, target, dir });
81
94
 
82
95
  function check(node, text) {
83
96
  for (const { file, part } of imagesIn(text, find)) {
@@ -102,17 +115,20 @@ export default {
102
115
  if (!reportable(result, { minSavedBytes, minSavedPercent }) || reported.has(file)) return;
103
116
  // The linter hides a warning a comment turns off by itself, but the image mustn't count as
104
117
  // flagged either, or --flagged would rewrite an image someone chose to keep.
105
- disabled ??= disabledLines(context.sourceCode);
118
+ disabled ??= disabledLines(context.sourceCode, context.id ?? RULE);
106
119
  if (disabled((loc?.start ?? node.loc.start).line)) return;
107
120
  reported.add(file);
108
- if (!EDITOR) flag(file, { original: result.original, compressed: result.compressed, score: result.score, target });
121
+ if (!EDITOR) flag(file, from, { original: result.original, compressed: result.compressed, score: result.score, target });
109
122
  // this package's own command (bin/pngdeer.js), at the target measured, so it writes exactly the
110
123
  // file this promises; named by the package, so npx can't fetch someone else's
111
124
  const command = `npx oxlint-plugin-pngdeer ${quote(shown(file))}${target === undefined ? "" : ` --target ${target}`}`;
125
+ const seen = `${file}\0${target}`;
126
+ if (!EDITOR && !warned.has(seen)) warned.set(seen, from);
127
+ const repeat = !EDITOR && warned.get(seen) !== from;
112
128
  context.report({
113
129
  node,
114
130
  loc: loc ?? undefined,
115
- messageId: result.outcome === "hit" ? "compressible" : "lossless",
131
+ messageId: repeat ? "again" : result.outcome === "hit" ? "compressible" : "lossless",
116
132
  data: {
117
133
  name: basename(file),
118
134
  percent: smaller(result.original, result.compressed),
@@ -139,37 +155,45 @@ export default {
139
155
  TemplateLiteral(node) {
140
156
  if (node.expressions.length === 0) check(node, node.quasis[0].value.cooked ?? "");
141
157
  },
142
- // What this file names is measured, then reported. The first time, every image in the
143
- // project goes in the same parallel batch, so the files linted next find theirs cached.
158
+ // What this file names is measured, then reported. The first file of a project in a run also
159
+ // takes every image in it, named or not, in the same parallel batch: the files linted next find
160
+ // theirs cached, and the summary can tell about images only CSS or HTML use.
144
161
  "Program:exit"() {
145
- if (found.length === 0) return;
146
- const missing = measure === "never" ? [] : [...new Set(found.filter(({ k }) => !read(dir, k)).map(({ file }) => file))];
147
- if (missing.length) {
148
- const group = `${root}\0${target}`;
149
- const files = scanned.has(group) ? missing : [...new Set([...missing, ...projectImages(root, minSavedBytes)])];
150
- scanned.add(group);
151
- if (!wait) fillInBackground({ cwd: root, target, files });
162
+ const group = `${root}\0${dir}\0${target}`;
163
+ const first = measure !== "never" && !scanned.has(group);
164
+ if (found.length === 0 && !first) {
165
+ if (!EDITOR) share();
166
+ return;
167
+ }
168
+ scanned.add(group);
169
+ // what the engine has yet to measure: pngdeer's own output (marked) needs none, and gets no entry,
170
+ // so the editor and the command line leave the same files in a cache folder someone commits
171
+ const unmeasured = (file) => {
172
+ const image = fileKey(file, target);
173
+ return image && !image.marked && !lookup(image.key);
174
+ };
175
+ const named = measure === "never" ? [] : found.map(({ file }) => file);
176
+ const files = [...new Set(first ? [...named, ...projectImages(root, minSavedBytes)] : named)].filter(unmeasured);
177
+ if (files.length) {
178
+ if (!wait) fillInBackground({ dir, failures: local, target, files });
152
179
  else if (!failed.has(root)) measureNow(files);
153
180
  }
154
181
  for (const ref of found) {
155
- const result = read(dir, ref.k);
182
+ const result = lookup(ref.k);
156
183
  tell(ref.file, result);
157
184
  report(ref, result);
158
185
  }
186
+ if (!EDITOR) share();
159
187
  },
160
188
  };
161
189
 
190
+ /** Measures `files` (none cached yet) while the lint waits, saying what the wait is for. */
162
191
  function measureNow(files) {
163
- // how many need the engine, to say what the wait is for (the hashes are kept for later lookups)
164
- const fresh = files.filter((file) => {
165
- const image = fileKey(file, target);
166
- return image && !image.marked && !read(dir, image.key);
167
- }).length;
168
192
  // on a terminal the batch rewrites this line as a count (fill-worker.js), and it's cleared after
169
- const tty = Boolean(fresh && process.stderr.isTTY);
170
- if (fresh) process.stderr.write(`pngdeer: measuring ${count(fresh, "image")} not in the cache yet...${tty ? "" : "\n"}`);
193
+ const tty = Boolean(process.stderr.isTTY);
194
+ process.stderr.write(`pngdeer: measuring ${count(files.length, "image")} not in the cache yet...${tty ? "" : "\n"}`);
171
195
  try {
172
- const { computed, ms } = fillSync({ cwd: root, target, files, progress: tty });
196
+ const { computed, ms } = fillSync({ dir, failures: local, target, files, progress: tty });
173
197
  if (tty) process.stderr.write("\r\x1b[K");
174
198
  if (computed) process.stderr.write(`pngdeer: measured ${count(computed, "image")} in ${(ms / 1000).toFixed(1)} s; later runs read the cache\n`);
175
199
  } catch (error) {
@@ -206,9 +230,10 @@ function imagesIn(text, find) {
206
230
 
207
231
  /**
208
232
  * Whether a comment turns this rule off at a line: `oxlint-` or `eslint-disable-next-line` above it,
209
- * `-disable-line` on it, or a `-disable` block around it, for this rule or for all rules.
233
+ * `-disable-line` on it, or a `-disable` block around it, for this rule (`id`, as the config names
234
+ * the plugin) or for all rules.
210
235
  */
211
- function disabledLines(source) {
236
+ function disabledLines(source, id) {
212
237
  const lines = new Set();
213
238
  const blocks = [];
214
239
  let open = null;
@@ -217,7 +242,7 @@ function disabledLines(source) {
217
242
  if (!directive) continue;
218
243
  const [, kind, rest] = directive;
219
244
  const rules = rest.split("--")[0].split(",").map((rule) => rule.trim()).filter(Boolean);
220
- if (rules.length && !rules.includes(RULE)) continue;
245
+ if (rules.length && !rules.includes(id) && !rules.includes(RULE)) continue;
221
246
  if (kind === "disable-next-line") lines.add(comment.loc.end.line + 1);
222
247
  else if (kind === "disable-line") lines.add(comment.loc.start.line);
223
248
  else if (kind === "disable") open ??= comment.loc.start.line;
@@ -252,21 +277,6 @@ function projectRoot(dir, fallback) {
252
277
  return roots.get(dir);
253
278
  }
254
279
 
255
- /** An image's size, the cache key of its current bytes and whether pngdeer wrote it; null if it can't be read. */
256
- function fileKey(file, target) {
257
- try {
258
- const { size, mtimeMs } = statSync(file);
259
- const memo = keys.get(file);
260
- if (memo && memo.size === size && memo.mtimeMs === mtimeMs && memo.target === target) return memo;
261
- const bytes = readFileSync(file);
262
- const image = { size, mtimeMs, target, key: key(bytes, target), marked: marked(bytes) };
263
- keys.set(file, image);
264
- return image;
265
- } catch {
266
- return null;
267
- }
268
- }
269
-
270
280
  function isFile(path) {
271
281
  try {
272
282
  return statSync(path).isFile();
package/src/scan.js CHANGED
@@ -1,8 +1,9 @@
1
- // Every PNG and JPEG in a project, to measure them all in one parallel batch the first time the
2
- // rule meets an image it hasn't seen (later references then find them cached). Folders are skipped
3
- // by name, not by reading .gitignore: dot folders, dependencies, build output, and test output such
4
- // as visual-regression snapshots, which are images too but never shipped. A folder with its own
5
- // package.json is another project, with its own cache, and is left to it.
1
+ // Every PNG and JPEG in a project, to measure them all in one parallel batch on a run's first file
2
+ // (later references then find them cached), and the other files the summary searches for images no
3
+ // linted code names. Folders are skipped by name, not by reading .gitignore: dot folders,
4
+ // dependencies, build output, and test output such as visual-regression snapshots, which are images
5
+ // too but never shipped. A folder with its own package.json is another project, with its own cache,
6
+ // and is left to it.
6
7
  import { readdirSync, statSync } from "node:fs";
7
8
  import { extname, join } from "node:path";
8
9
  import { EXTENSIONS } from "./images.js";
@@ -15,6 +16,11 @@ const SKIP = new Set([
15
16
 
16
17
  /** The project's images of at least `minSize` bytes. */
17
18
  export function projectImages(root, minSize = 0) {
19
+ return projectFiles(root, EXTENSIONS, minSize);
20
+ }
21
+
22
+ /** The project's files with one of `extensions`, from `minSize` to `maxSize` bytes. */
23
+ export function projectFiles(root, extensions, minSize = 0, maxSize = Infinity) {
18
24
  const found = [];
19
25
  const walk = (dir) => {
20
26
  let entries;
@@ -30,8 +36,9 @@ export function projectImages(root, minSize = 0) {
30
36
  if (entry.isDirectory()) {
31
37
  // Playwright keeps a test's snapshots in "<test file>-snapshots"
32
38
  if (!SKIP.has(entry.name) && !entry.name.endsWith("-snapshots")) walk(path);
33
- } else if (entry.isFile() && EXTENSIONS.includes(extname(entry.name).toLowerCase()) && size(path) >= minSize) {
34
- found.push(path);
39
+ } else if (entry.isFile() && extensions.includes(extname(entry.name).toLowerCase())) {
40
+ const bytes = size(path);
41
+ if (bytes >= minSize && bytes <= maxSize) found.push(path);
35
42
  }
36
43
  }
37
44
  };
package/src/shrink.js ADDED
@@ -0,0 +1,63 @@
1
+ // Making one image smaller on disk: what the command does to each image it's given, in its own
2
+ // thread for one image or on the pool's engines for several (worker.js).
3
+ import { chmodSync, readFileSync, renameSync, rmSync, statSync, writeFileSync } from "node:fs";
4
+ import { basename, dirname, join } from "node:path";
5
+ import { compress } from "./engine.js";
6
+ import { unsupported } from "./images.js";
7
+ import { mark, marked } from "./marker.js";
8
+ import { looks, size, smaller } from "./text.js";
9
+
10
+ /**
11
+ * Writes `input` made smaller to `output`, marked as pngdeer's: { text } saying what happened, plus
12
+ * { original, written } sizes when it wrote a smaller file. Throws when it can't read or compress.
13
+ */
14
+ export function shrink(input, output, target) {
15
+ const bytes = readFileSync(input);
16
+ const unchanged = (note) => {
17
+ if (output !== input) save(output, bytes);
18
+ return { text: `${note}; left as is` };
19
+ };
20
+ if (marked(bytes)) return unchanged("already pngdeer's");
21
+ const why = unsupported(bytes);
22
+ if (why) throw new Error(why);
23
+
24
+ const { output: compressed, outcome, score } = compress(bytes, target);
25
+ if (outcome === "miss") {
26
+ const wanted = target === undefined ? "the automatic quality" : `--target ${target}`;
27
+ return unchanged(`no smaller file reaches ${wanted} (the best reaches quality ${Math.round(score)}/100)`);
28
+ }
29
+ // the mark costs a few dozen bytes, more than pngdeer saves on some tiny files
30
+ const result = outcome === "original" ? null : mark(compressed);
31
+ if (!result || result.length >= bytes.length) return unchanged("pngdeer can't make it smaller");
32
+ save(output, result);
33
+ const quality = outcome === "hit" ? `quality ${Math.round(score)}/100: looks ${looks(score)}` : "lossless: looks identical";
34
+ const text = `${size(bytes.length)} → ${size(result.length)} (${smaller(bytes.length, result.length)}% smaller, ${quality})`;
35
+ return { text, original: bytes.length, written: result.length };
36
+ }
37
+
38
+ /** An error as one line a person can act on. */
39
+ export function plain(error) {
40
+ if (error?.code === "ENOENT") return "no such file";
41
+ if (error?.code === "EACCES" || error?.code === "EPERM") return "permission denied";
42
+ if (error?.code === "EISDIR") return "a folder, not an image";
43
+ return String(error?.message ?? error);
44
+ }
45
+
46
+ /** Writes a whole new file or nothing: the image is often written over itself. */
47
+ function save(path, bytes) {
48
+ // unique per writer: the pool's engines share the process, and one image may come twice
49
+ const tmp = join(dirname(path), `.${basename(path)}.${process.pid}.${Math.random().toString(36).slice(2)}.tmp`);
50
+ try {
51
+ writeFileSync(tmp, bytes);
52
+ try {
53
+ chmodSync(tmp, statSync(path).mode & 0o7777);
54
+ } catch {
55
+ // a new file keeps the default mode
56
+ }
57
+ renameSync(tmp, path);
58
+ } catch (error) {
59
+ rmSync(tmp, { force: true });
60
+ if (error.code === "ENOENT") throw new Error(`can't write ${path}: its folder doesn't exist`);
61
+ throw error;
62
+ }
63
+ }
package/src/summary.js CHANGED
@@ -1,26 +1,35 @@
1
1
  // What a lint flagged, in all: a table at the end of the run, biggest saving first, and the list that
2
2
  // `npx oxlint-plugin-pngdeer --flagged` shrinks, so nobody copies paths from warnings one by one.
3
- // The table also names images the run measured but no linted code names (used from CSS or HTML, or
4
- // not at all), which no warning can point at.
5
- import { existsSync, mkdirSync, readFileSync, writeFileSync } from "node:fs";
6
- import { dirname, join } from "node:path";
7
- import { cacheDir, key, read, reportable } from "./cache.js";
8
- import { projectImages } from "./scan.js";
9
- import { count, pathFrom, size, smaller } from "./text.js";
3
+ // The table also lists images the run measured that no linted JS or TS names, which no warning can
4
+ // point at, each with where it is used instead (a stylesheet, a page, a template), if anywhere.
5
+ import { existsSync, mkdirSync, readdirSync, readFileSync, rmSync, statSync, writeFileSync } from "node:fs";
6
+ import { basename, dirname, join } from "node:path";
7
+ import { isMainThread, threadId } from "node:worker_threads";
8
+ import { cacheDir, fileKey, read, reportable } from "./cache.js";
9
+ import { projectFiles, projectImages } from "./scan.js";
10
+ import { count, pathFrom, quote, size, smaller } from "./text.js";
10
11
 
11
12
  // Rows shown per table; the rest are counted, and --flagged still takes them all.
12
13
  const ROWS = 10;
13
14
  const UNNAMED_ROWS = 5;
14
15
  // Formats other programs read: the table on stderr would only get in the way.
15
16
  const MACHINE = new Set(["json", "json-with-metadata", "checkstyle", "junit", "sarif", "gitlab"]);
17
+ // Files the linter doesn't read but that use images, most telling first: styles, pages and
18
+ // templates, then content and data. Markdown comes last, and only "mentions" an image: a README
19
+ // that names one may just be describing it.
20
+ const PAGES = [".css", ".scss", ".sass", ".less", ".styl", ".html", ".htm", ".vue", ".svelte", ".astro", ".mdx", ".json", ".yml", ".yaml"];
21
+ const DOCS = [".md"];
22
+ const PAGE_BYTES = 1e6; // bigger ones are build output or lockfiles, not where people name images
16
23
 
17
- const flagged = new Map(); // image path -> { original, compressed, score, target }
24
+ const flagged = new Map(); // image path -> { original, compressed, score, target, files: the linted files warning about it }
18
25
  const named = new Set(); // every image linted code names, flagged or not
19
26
  const roots = new Map(); // project root -> the rule's options there
27
+ const SHARE_HOURS = 1; // a thread's share older than this was left by a run that died
20
28
 
21
- /** Notes an image a warning was reported for. */
22
- export function flag(file, entry) {
23
- flagged.set(file, entry);
29
+ /** Notes an image a warning in linted file `from` was reported for: one warning per file, however often it's linted. */
30
+ export function flag(file, from, entry) {
31
+ const files = flagged.get(file)?.files ?? new Set();
32
+ flagged.set(file, { ...entry, files: files.add(from) });
24
33
  }
25
34
 
26
35
  /** Notes an image linted code names. */
@@ -47,8 +56,58 @@ export function readFlagged(cwd) {
47
56
  }
48
57
  }
49
58
 
50
- /** Prints the table on stderr and leaves the list in `cwd`; once, as the lint exits. */
59
+ /**
60
+ * ESLint's --concurrency lints in worker threads, each with its own copy of this module: each leaves
61
+ * what it saw for the main thread, after every file it lints (a thread may be stopped without warning),
62
+ * and only the main thread prints. oxlint runs every rule on its main thread.
63
+ */
64
+ export function share(cwd = process.cwd()) {
65
+ if (isMainThread || roots.size === 0) return;
66
+ try {
67
+ const dir = join(cacheDir(cwd), "threads");
68
+ mkdirSync(dir, { recursive: true });
69
+ const state = { flagged: [...flagged].map(([file, entry]) => [file, { ...entry, files: [...entry.files] }]), named: [...named], roots: [...roots] };
70
+ writeFileSync(join(dir, `${process.pid}-${threadId}.json`), JSON.stringify(state));
71
+ } catch {
72
+ // that thread's images go untold
73
+ }
74
+ }
75
+
76
+ /** Takes in what this process's lint threads left, and drops shares a dead run left behind. */
77
+ function gather(cwd) {
78
+ const dir = join(cacheDir(cwd), "threads");
79
+ let names;
80
+ try {
81
+ names = readdirSync(dir);
82
+ } catch {
83
+ return;
84
+ }
85
+ for (const name of names) {
86
+ const path = join(dir, name);
87
+ try {
88
+ if (name.startsWith(`${process.pid}-`)) {
89
+ const state = JSON.parse(readFileSync(path, "utf8"));
90
+ for (const [file, entry] of state.flagged) {
91
+ const files = new Set([...(flagged.get(file)?.files ?? []), ...entry.files]);
92
+ flagged.set(file, { ...entry, files });
93
+ }
94
+ for (const file of state.named) named.add(file);
95
+ for (const [root, options] of state.roots) roots.set(root, options);
96
+ } else if (Date.now() - statSync(path).mtimeMs < SHARE_HOURS * 3600e3) continue;
97
+ rmSync(path, { force: true });
98
+ } catch {
99
+ // a share half written or already gone
100
+ }
101
+ }
102
+ }
103
+
104
+ /** Prints the table on stderr and leaves the list in `cwd`; once, as the lint exits, on the main thread. */
51
105
  export function summarize(cwd = process.cwd()) {
106
+ if (!isMainThread) return;
107
+ gather(cwd);
108
+ // a process that linted nothing (ESLint's main thread, when worker threads did the linting and
109
+ // left nothing) has nothing to say, and mustn't empty the list the last lint left
110
+ if (roots.size === 0) return;
52
111
  const list = flaggedList(cwd);
53
112
  const images = [...flagged].sort(([, a], [, b]) => b.original - b.compressed - (a.original - a.compressed));
54
113
  // a clean run replaces an older list, but doesn't create one
@@ -64,11 +123,19 @@ export function summarize(cwd = process.cwd()) {
64
123
 
65
124
  const unnamed = unnamedImages();
66
125
  if (images.length === 0 && unnamed.length === 0) return;
126
+ const shownUnnamed = unnamed.slice(0, UNNAMED_ROWS);
67
127
  const row = ([file, { original, compressed, score }]) => [pathFrom(cwd, file), size(original), size(compressed), size(original - compressed), String(Math.round(score))];
68
- const rows = [["image", "now", "after", "saves", "quality"], ...images.slice(0, ROWS).map(row), ...unnamed.slice(0, UNNAMED_ROWS).map(row)];
69
- const width = (column) => Math.max(...rows.map((cells) => cells[column].length));
70
- const line = ([path, before, after, saved, score], arrow = "→") =>
71
- ` ${path.padEnd(width(0))} ${before.padStart(width(1))} ${arrow} ${after.padStart(width(2))} ${saved.padStart(width(3))} ${score.padStart(width(4))}`;
128
+ const header = ["image", "now", "after", "saves", "quality"];
129
+ // the unnamed images say where they're named, as no warning can point there
130
+ const where = ([file, , root]) => {
131
+ const page = namedIn(root, file);
132
+ if (!page) return "nothing names it";
133
+ return `${page.endsWith(".md") ? "mentioned" : "used"} in ${pathFrom(cwd, page)}`;
134
+ };
135
+ const rows = [header, ...images.slice(0, ROWS).map(row), ...shownUnnamed.map((image) => [...row(image), where(image)])];
136
+ const width = (column) => Math.max(...rows.map((cells) => (cells[column] ?? "").length));
137
+ const line = ([path, before, after, saved, score, place], arrow = "→") =>
138
+ ` ${path.padEnd(width(0))} ${before.padStart(width(1))} ${arrow} ${after.padStart(width(2))} ${saved.padStart(width(3))} ${score.padStart(width(4))}${place === undefined ? "" : ` ${place}`}`.trimEnd();
72
139
  const total = (list) => {
73
140
  const original = list.reduce((sum, [, image]) => sum + image.original, 0);
74
141
  const compressed = list.reduce((sum, [, image]) => sum + image.compressed, 0);
@@ -77,14 +144,19 @@ export function summarize(cwd = process.cwd()) {
77
144
  const lines = [];
78
145
  if (images.length) {
79
146
  const { saved, detail } = total(images);
80
- lines.push(`pngdeer: ${count(images.length, "flagged image")} could be ${saved} smaller in all (${detail}):`, line(rows[0], " "));
147
+ // an image named in many files warns in each: say so, or the counts look wrong
148
+ const warnings = images.reduce((sum, [, image]) => sum + image.files.size, 0);
149
+ const often = warnings > images.length ? ` (${count(warnings, "warning")})` : "";
150
+ lines.push(`pngdeer: ${count(images.length, "flagged image")}${often} could be ${saved} smaller in all (${detail}):`, line(header, " "));
81
151
  lines.push(...rows.slice(1, 1 + Math.min(images.length, ROWS)).map((cells) => line(cells)));
82
152
  if (images.length > ROWS) lines.push(` …and ${count(images.length - ROWS, "more image")}, ${total(images.slice(ROWS)).saved} in all`);
83
153
  }
84
154
  if (unnamed.length) {
155
+ // no warning can point at these: their only mention is in CSS, HTML and the like, or nowhere
85
156
  const { saved } = total(unnamed);
86
- lines.push(`pngdeer: ${count(unnamed.length, "more image")} could be ${saved} smaller, but no linted code names ${unnamed.length === 1 ? "it" : "them"} (used from CSS or HTML, or not at all):`);
87
- if (!images.length) lines.push(line(rows[0], " "));
157
+ const lead = images.length ? "also" : "no warnings, but";
158
+ lines.push(`pngdeer: ${lead} ${count(unnamed.length, "image")} that no linted JS or TS names could be ${saved} smaller:`);
159
+ if (!images.length) lines.push(line(header, " "));
88
160
  lines.push(...rows.slice(1 + Math.min(images.length, ROWS)).map((cells) => line(cells)));
89
161
  if (unnamed.length > UNNAMED_ROWS) lines.push(` …and ${count(unnamed.length - UNNAMED_ROWS, "more image")}`);
90
162
  }
@@ -92,28 +164,50 @@ export function summarize(cwd = process.cwd()) {
92
164
  if (images.length) {
93
165
  lines.push(`pngdeer: to shrink the ${images.length === 1 ? "flagged image" : `${images.length} flagged images`} in place (lossy, so commit first), run: npx oxlint-plugin-pngdeer --flagged`);
94
166
  }
167
+ if (unnamed.length) {
168
+ const paths = shownUnnamed.map(([file]) => quote(pathFrom(cwd, file))).join(" ");
169
+ const which = unnamed.length === 1 ? "it" : "them";
170
+ lines.push(`pngdeer: to shrink ${images.length ? `${which} too` : `${which} in place (lossy, so commit first)`}, run: npx oxlint-plugin-pngdeer ${paths}`);
171
+ if (shownUnnamed.some((image) => !namedIn(image[2], image[0]))) {
172
+ lines.push("pngdeer: an image nothing names may be unused (or its path is built at runtime): deleting it saves it all");
173
+ }
174
+ }
95
175
  process.stderr.write(`${lines.join("\n")}\n`);
96
176
  }
97
177
 
98
178
  /**
99
179
  * Measured images in the linted projects that no linted code names, and that would clear the rule's
100
- * thresholds: [[path, entry]], biggest saving first. Reads only results already cached.
180
+ * thresholds: [[path, entry, root]], biggest saving first. Reads only results already cached.
101
181
  */
102
182
  function unnamedImages() {
103
183
  const found = [];
104
184
  for (const [root, options] of roots) {
105
185
  for (const file of projectImages(root, options.minSavedBytes)) {
106
186
  if (named.has(file) || flagged.has(file)) continue;
107
- let entry;
187
+ const image = fileKey(file, options.target);
188
+ const entry = image && read(options.dir, image.key);
189
+ if (reportable(entry, options)) found.push([file, entry, root]);
190
+ }
191
+ }
192
+ return found.sort(([, a], [, b]) => b.original - b.compressed - (a.original - a.compressed));
193
+ }
194
+
195
+ const pages = new Map(); // project root -> [{ path, text }] of its PAGES, then its DOCS, read once
196
+
197
+ /** The most telling file the linter doesn't read (a stylesheet, a template, a doc) that names `image`; or null. */
198
+ function namedIn(root, image) {
199
+ if (!pages.has(root)) {
200
+ const files = [...PAGES, ...DOCS].flatMap((extension) => projectFiles(root, [extension], 0, PAGE_BYTES));
201
+ pages.set(root, files.flatMap((path) => {
108
202
  try {
109
- entry = read(cacheDir(root), key(readFileSync(file), options.target));
203
+ return [{ path, text: readFileSync(path, "utf8") }];
110
204
  } catch {
111
- continue;
205
+ return [];
112
206
  }
113
- if (reportable(entry, options)) found.push([file, entry]);
114
- }
207
+ }));
115
208
  }
116
- return found.sort(([, a], [, b]) => b.original - b.compressed - (a.original - a.compressed));
209
+ const name = basename(image);
210
+ return pages.get(root).find(({ text }) => text.includes(name))?.path ?? null;
117
211
  }
118
212
 
119
213
  /** The linter's output format, from its command line: `-f x`, `--format x` or `--format=x`. */
package/src/sync.js CHANGED
@@ -31,10 +31,10 @@ export function fillSync(request) {
31
31
 
32
32
  // The editor's background batches: one at a time per process, whatever arrives meanwhile goes in
33
33
  // the next one, so fill()'s pool and memory budget hold for the whole language server. Files are
34
- // keyed by project and target, as a batch has one of each.
35
- const queued = new Map(); // `${cwd}\0${target}` -> { cwd, target, files: Set }
36
- const pending = new Set(); // `${cwd}\0${target}\0${file}`, queued or being measured
37
- const failedAt = new Map(); // `${cwd}\0${target}` -> when its last batch failed
34
+ // keyed by cache folder and target, as a batch has one of each.
35
+ const queued = new Map(); // `${dir}\0${target}` -> { dir, target, files: Set }
36
+ const pending = new Set(); // `${dir}\0${target}\0${file}`, queued or being measured
37
+ const failedAt = new Map(); // `${dir}\0${target}` -> when its last batch failed
38
38
  const RETRY_MS = 5 * 60 * 1000;
39
39
  let running = false;
40
40
 
@@ -57,10 +57,10 @@ function drain() {
57
57
  }
58
58
 
59
59
  /** fill(request) in the background, for the editor: nothing waits, nothing is reported late. */
60
- export function fillInBackground({ cwd, target, files }) {
61
- const group = `${cwd}\0${target}`;
60
+ export function fillInBackground({ dir, failures, target, files }) {
61
+ const group = `${dir}\0${target}`;
62
62
  if (Date.now() - (failedAt.get(group) ?? -Infinity) < RETRY_MS) return;
63
- const batch = queued.get(group) ?? { cwd, target, files: new Set() };
63
+ const batch = queued.get(group) ?? { dir, failures, target, files: new Set() };
64
64
  for (const file of files) {
65
65
  if (pending.has(`${group}\0${file}`)) continue;
66
66
  pending.add(`${group}\0${file}`);
@@ -0,0 +1,132 @@
1
+ // Path aliases a project defines in tsconfig.json or jsconfig.json (compilerOptions.paths), resolved
2
+ // the way TypeScript does, so "@/assets/hero.png" finds the file the build would: `extends` (later
3
+ // entries win, ".json" optional), `paths` and `baseUrl` each from the nearest config that sets them
4
+ // (`null` unsets), targets relative to `baseUrl` or else to the config that set `paths`, and
5
+ // `${configDir}` as the folder of the config being resolved. A config without paths falls back to
6
+ // its `references` (Vite keeps them in tsconfig.app.json). Configs from packages are skipped: they
7
+ // don't point into the project.
8
+ import { readFileSync, statSync } from "node:fs";
9
+ import { dirname, join, relative, resolve, sep } from "node:path";
10
+
11
+ const found = new Map(); // project root -> { aliases, seen: Map(config path -> mtime, or -1 if missing) }
12
+ const told = new Set(); // configs that couldn't be read, already said so
13
+
14
+ /** { "@/": "src/" }-style aliases from the project's config, as folders relative to `root`; null if none. */
15
+ export function configAliases(root) {
16
+ const memo = found.get(root);
17
+ // a config edited since (an editor runs for hours; .svelte-kit and .nuxt regenerate theirs)
18
+ if (memo && [...memo.seen].every(([file, mtime]) => mtimeOf(file) === mtime)) return memo.aliases;
19
+ const seen = new Map();
20
+ let aliases = null;
21
+ try {
22
+ aliases = projectAliases(root, seen);
23
+ } catch {
24
+ // whatever a config holds, the rule goes on with the default aliases
25
+ }
26
+ found.set(root, { aliases, seen });
27
+ return aliases;
28
+ }
29
+
30
+ /** The aliases of the project's tsconfig.json (or jsconfig.json, without one), or of its references. */
31
+ function projectAliases(root, seen) {
32
+ const tsconfig = join(root, "tsconfig.json");
33
+ const leaf = mtimeOf(tsconfig) >= 0 ? tsconfig : join(root, "jsconfig.json");
34
+ const options = effective(leaf, dirname(leaf), seen, new Set());
35
+ if (options?.paths !== undefined) return aliasesOf(options, root, dirname(leaf));
36
+ for (const reference of load(leaf, seen)?.references ?? []) {
37
+ if (typeof reference?.path !== "string") continue;
38
+ const file = configFile(resolve(dirname(leaf), reference.path));
39
+ if (!file) continue;
40
+ const referenced = effective(file, dirname(file), seen, new Set());
41
+ if (referenced?.paths !== undefined) return aliasesOf(referenced, root, dirname(file));
42
+ }
43
+ return null;
44
+ }
45
+
46
+ /**
47
+ * A config's `paths` (with `pathsDir`, the folder of the config that set them) and `baseUrl` (a
48
+ * folder), after its `extends`; a key is present only if some config in the chain set it.
49
+ */
50
+ function effective(file, configDir, seen, chain) {
51
+ if (chain.has(file)) return {};
52
+ chain.add(file);
53
+ const config = load(file, seen);
54
+ if (!config) return null;
55
+ let options = {};
56
+ const extended = typeof config.extends === "string" ? [config.extends] : Array.isArray(config.extends) ? config.extends : [];
57
+ for (const spec of extended) {
58
+ const base = typeof spec === "string" && (spec.startsWith(".") || spec.startsWith("/") || spec.startsWith("${configDir}")) ? configFile(resolve(dirname(file), expand(spec, configDir))) : null;
59
+ if (base) options = { ...options, ...effective(base, configDir, seen, chain) };
60
+ }
61
+ const own = config.compilerOptions ?? {};
62
+ if ("baseUrl" in own) options.baseUrl = typeof own.baseUrl === "string" ? resolve(dirname(file), expand(own.baseUrl, configDir)) : undefined;
63
+ if ("paths" in own) {
64
+ options.paths = own.paths && typeof own.paths === "object" ? own.paths : null;
65
+ options.pathsDir = dirname(file);
66
+ }
67
+ return options;
68
+ }
69
+
70
+ /** Folder aliases from effective options; null when none is a folder pattern ("@/*": ["./src/*"]). */
71
+ function aliasesOf({ paths, pathsDir, baseUrl }, root, configDir) {
72
+ const aliases = {};
73
+ for (const [pattern, targets] of Object.entries(paths ?? {})) {
74
+ const target = Array.isArray(targets) ? targets.find((t) => typeof t === "string") : undefined;
75
+ // a bare "*" would match every path
76
+ if (pattern === "*" || !pattern.endsWith("*") || !target?.endsWith("*")) continue;
77
+ const folder = relative(root, resolve(baseUrl ?? pathsDir, expand(target, configDir).slice(0, -1))).split(sep).join("/");
78
+ aliases[pattern.slice(0, -1)] = folder ? `${folder}/` : "";
79
+ }
80
+ return Object.keys(aliases).length ? aliases : null;
81
+ }
82
+
83
+ /** A config's contents, or null if it doesn't exist; a config that can't be parsed is named on stderr once. */
84
+ function load(file, seen) {
85
+ seen.set(file, mtimeOf(file));
86
+ let text;
87
+ try {
88
+ text = readFileSync(file, "utf8");
89
+ } catch {
90
+ return null;
91
+ }
92
+ try {
93
+ return parse(text);
94
+ } catch (error) {
95
+ if (!told.has(file)) {
96
+ told.add(file);
97
+ process.stderr.write(`pngdeer: couldn't read path aliases from ${file}: ${error.message}\n`);
98
+ }
99
+ return null;
100
+ }
101
+ }
102
+
103
+ /** The config file a path in `extends` or `references` names: as is, with ".json", or a folder's tsconfig.json. */
104
+ function configFile(path) {
105
+ for (const file of [path, `${path}.json`, join(path, "tsconfig.json")]) {
106
+ try {
107
+ if (statSync(file).isFile()) return file;
108
+ } catch {
109
+ // not this one
110
+ }
111
+ }
112
+ return null;
113
+ }
114
+
115
+ /** JSON with a BOM, comments and trailing commas, as tsconfig files are written. */
116
+ function parse(text) {
117
+ const bare = text.replace(/^\uFEFF/, "").replace(/"(?:\\.|[^"\\])*"|\/\/[^\n]*|\/\*[\s\S]*?\*\//g, (match) => (match.startsWith('"') ? match : ""));
118
+ return JSON.parse(bare.replace(/,(\s*[}\]])/g, "$1"));
119
+ }
120
+
121
+ /** TypeScript 5.5's ${configDir}: the folder of the config being resolved, not of the one that says it. */
122
+ function expand(path, configDir) {
123
+ return path.replace(/^\$\{configDir\}/, configDir);
124
+ }
125
+
126
+ function mtimeOf(file) {
127
+ try {
128
+ return statSync(file).mtimeMs;
129
+ } catch {
130
+ return -1;
131
+ }
132
+ }
package/src/worker.js CHANGED
@@ -1,31 +1,41 @@
1
- // One pngdeer engine per worker thread: compresses the files it's sent and reports sizes, the cache
2
- // key of the bytes it actually read (the file may have changed since it was queued), and the key of
3
- // its own output, so that file is recognized later if someone writes it to disk.
1
+ // One pngdeer engine per worker thread, doing what it's sent: { file, target } measures a file and
2
+ // reports sizes, the cache key of the bytes it actually read (the file may have changed since it was
3
+ // queued), and the key of its own output, so that file is recognized later if someone writes it to
4
+ // disk; { file, target, out } makes the file smaller on disk, as the command does (shrink.js).
4
5
  import { readFileSync } from "node:fs";
5
- import { parentPort, workerData } from "node:worker_threads";
6
+ import { parentPort } from "node:worker_threads";
6
7
  import { key } from "./cache.js";
7
8
  import { compress } from "./engine.js";
8
9
  import { mark } from "./marker.js";
10
+ import { plain, shrink } from "./shrink.js";
9
11
 
10
- const why = (error) => String(error?.message ?? error);
11
-
12
- parentPort.on("message", ({ file }) => {
12
+ parentPort.on("message", ({ file, target, out }) => {
13
+ if (out !== undefined) {
14
+ let reply;
15
+ try {
16
+ reply = shrink(file, out, target);
17
+ } catch (error) {
18
+ reply = { error: plain(error) };
19
+ }
20
+ parentPort.postMessage(reply);
21
+ return;
22
+ }
13
23
  const start = performance.now();
14
24
  let bytes;
15
25
  try {
16
26
  bytes = readFileSync(file);
17
27
  } catch (error) {
18
- parentPort.postMessage({ input: null, error: why(error) });
28
+ parentPort.postMessage({ input: null, error: plain(error) });
19
29
  return;
20
30
  }
21
- const input = key(bytes, workerData.target);
31
+ const input = key(bytes, target);
22
32
  try {
23
- const { output, outcome, score } = compress(bytes, workerData.target);
33
+ const { output, outcome, score } = compress(bytes, target);
24
34
  const ms = Math.round(performance.now() - start);
25
35
  // the size the command will write, mark included, so the warning and the command agree
26
36
  const compressed = outcome === "original" ? bytes.length : mark(output).length;
27
- parentPort.postMessage({ input, original: bytes.length, compressed, outcome, score, ms, output: key(output, workerData.target) });
37
+ parentPort.postMessage({ input, original: bytes.length, compressed, outcome, score, ms, output: key(output, target) });
28
38
  } catch (error) {
29
- parentPort.postMessage({ input, error: why(error) });
39
+ parentPort.postMessage({ input, error: plain(error) });
30
40
  }
31
41
  });